The lottery ticket hypothesis crystallised: large networks succeed not by learning complex solutions, but by providing more opportunities to find simple ones.
Wouldn't training a lot of small networks work as well then?
this post was submitted on 18 Aug 2025
11 points (100.0% liked)
Technology
4267 readers
637 users here now
Which posts fit here?
Anything that is at least tangentially connected to the technology, social media platforms, informational technologies and tech policy.
Post guidelines
[Opinion] prefix
Opinion (op-ed) articles must use [Opinion] prefix before the title.
Rules
1. English only
Title and associated content has to be in English.
2. Use original link
Post URL should be the original link to the article (even if paywalled) and archived copies left in the body. It allows avoiding duplicate posts when cross-posting.
3. Respectful communication
All communication has to be respectful of differing opinions, viewpoints, and experiences.
4. Inclusivity
Everyone is welcome here regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
5. Ad hominem attacks
Any kind of personal attacks are expressly forbidden. If you can't argue your position without attacking a person's character, you already lost the argument.
6. Off-topic tangents
Stay on topic. Keep it relevant.
7. Instance rules may apply
If something is not covered by community rules, but are against lemmy.zip instance rules, they will be enforced.
Companion communities
!globalnews@lemmy.zip
!interestingshare@lemmy.zip
Icon attribution | Banner attribution
If someone is interested in moderating this community, message @brikox@lemmy.zip.
founded 2 years ago
MODERATORS
That’s pretty much just AI agents.
Quite possibly, yes. But how much is "a lot?" A wide network acts like many permutations.
Probing the space with small networks and brief training sounds faster, but that too is recreated in large networks. They'll train for a bit, mark any weights near zero, reset, and zero those out.
What training many small networks would be good for is experimentation. Super deep and narrow, just five big dumb layers, fewer steps with more heads, that kind of thing. Maybe get wild and ask a question besides "what's the next symbol."