Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

[D] Can LLM trained on Testset? (alien.top)

submitted 1 year ago by No_Oilve_6577@alien.top to c/machinelearning@academy.garden

3 comments fedilink hide all child comments

For the public leaderboard in LLM, they tested of MLMU, ARC that kinds of dataset. What happen if I simply train my LLM on test set, how do you know I did that? I will get a model that rank high in the public leaderboard right?

top 3 comments

sorted by: hot top controversial new old

[–] wazis@alien.top 1 points 1 year ago

Yeah you can also hard code answer and score 100% on any test. But what's the point?

[–] AltruisticCoder@alien.top 1 points 1 year ago

There is a fair chance that a lot of LLMs already do this, just not only the test set but also other data.

[–] saintshing@alien.top 1 points 1 year ago

Pretraining on the Test Set Is All You Need