Generally, models with stronger inductive biases (like CNNs) work better at small scales - as long as those biases are correct for the kind of data you're working with.
currentscurrents
joined 1 year ago
You can train CNNs on unlabeled data too. Unsupervised learning works with any model type, and diffusion models or VAEs are often CNN-based.
The other advantage is multimodality, you can tokenize anything.
Maybe it's less about having as many parameters as the human brain, and more about having datasets as rich and diverse as the real world.
What are you trying to use SVMs for in 2023? Most of their use cases have been taken over by either neural networks or XGBoost.