iantimmis

joined 10 months ago
sorted by: new top controversial old
[D] Skill Creep in ML/DL Roles - is the field getting not just more competitive, but more difficult? in c/machinelearning@academy.garden
[โ€“] iantimmis@alien.top 1 points 10 months ago

Damn this post resonates so hard.

Are LLMs at a practical limit for layer stacking? [D] in c/machinelearning@academy.garden
[โ€“] iantimmis@alien.top 1 points 10 months ago

It becomes very expensive compute-wise, but where we are actually running up to the edge is the scale of the data. They've discovered "scaling laws" (see chinchilla paper) that determines how big your model should be given the amount of data you have. We could go bigger but there's no reason to use a multi-trillion parameter model for example because it's just wasted capacity.