Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

[D] How do you handle the lack of data in a research project for testing? (alien.top)

submitted 11 months ago by I_will_delete_myself@alien.top to c/machinelearning@academy.garden

8 comments fedilink hide all child comments

The thing I have is randomly sampling days from the year however this runs into flaws of the day just not being optimal. Other research papers in related work has two years one for training and another for testing. I only have one year and asked the professor for another year of data and he said it isn’t available with no explanation behind it. I tried searching online for the data since it’s supposedly public data but can’t find it. It’s kind of hard to get an accurate display of evaluation when you don’t have a good test environment. Instead of complaining I have to figure out something. This is in the works in trying to get published to a smaller journal. Not sure if any of you all had this so curious how you would handle such situations?

At least an idea

Sample months instead of just time samples and create a pseudo environment and run it through there. Picking diverse set that has similar yearly trends.

I don’t have much experience so open ears when you all had similar limitations and how you all overcame it.

you are viewing a single comment's thread
view the rest of the comments

[–] Altruistic-Skill8667@alien.top 1 points 11 months ago (2 children)

When I hear: not a lot of data, i immediately think: overfitting danger.

If it’s a Reinforcement Learning algorithm, then maybe pretraining with synthetic data that’s similar to the real one, so that you already have some rough Q values. And then k-fold crossvalidation. Train on a subset and test on another, and then rotate through.

[–] I_will_delete_myself@alien.top 1 points 11 months ago (1 children)

It has about 18 thousand samples.

[–] Altruistic-Skill8667@alien.top 1 points 11 months ago

Sounds like a decently sized dataset.