A little bit of functional analysis wouldn't hurt. Knowing your fourier / laplace operators could help clarify some of the logic underlying CNNs / Neural ODEs.
Group theory seems to be ramping up with invariance / equivariant properties of some specialized neural networks (i.e. rotation equivariant nets, gauge equivariant nets).
Causal inference is another emerging hot topic - but it is a bit scatterbrained, since there are multiple competing schools of thought surround this topic.
Maybe for silicon neural networks, but what about quantum neural networks??