Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

[D] How do you keep up ? (alien.top)

submitted 11 months ago by CursedCrystalCoconut@alien.top to c/machinelearning@academy.garden

32 comments fedilink hide all child comments

I started my PhD in NLP a year or so before the advent of Transformers, and finished it just as ChatGPT was unveiled (literally defended a week before). Halfway through, I felt the sudden acceleration of NLP, where there was so much everywhere all at once. Before, knowing one's domain, and the state-of-the-art GCN, CNN or Bert architectures, was enough.

Since, I've been working in a semi-related area (computer assisted humanities) as a data engineer/software developer/ML engineer (it's a small team so many hats). Not much in terms of latest news, so I tried recently to get up to speed with the recent developments.

But there are so many ! Everywhere. Even just in NLP, not considering all the other fields such as reinforcement learning, computer vision, all the fundamentals of ML etc. It is damn near impossible to gather an in-depth understanding of a model as they are so complex, and numerous. All of them are built on top of other ones, so you also need to read up on those to understand anything. I follow some people on LinkedIn who just give new names every week or so. Going to look for papers in top conferences is also daunting as there is no guarantee that a paper with an award will translate to an actual system, while companies churn out new architectures without the research paper/methodology being made public. It's overwhelming.

So I guess my question is two fold : how does one get up to speed after a year of not being too much in the field ? And how does one keep up after that ?

you are viewing a single comment's thread
view the rest of the comments

[–] Gramious@alien.top 1 points 11 months ago (1 children)

My tactic is to start by checking the papers that actually GET IN to major conferences (Neurips, ICLR, ICML are a good start). This narrows the search considerably. Doing a Google scholar search, for example, will just yield an insurmountable number of papers. This is, in part, due to the standard "make public before it is accepted" methodology (arXiV preprints are fantastic but they also increase the noise level dramatically).

Now, having been burnt by the chaos of the review processes of the aforementioned conferences, I am certainly aware that their publications are by no means the "Gold standard" but the notion of peer review, including the intended outcomes of improvement therethrough, is powerful nonetheless.

[–] CursedCrystalCoconut@alien.top 1 points 11 months ago

That helps narrow it down. Though, many discoveries are not published anymore. Reminds me of Mikolov, who was rejected pretty much everywhere and word vectors ended up being such a big deal. Or that OpenAI does not publish their models.