No magic hyperparameter is going to do the trick here.
Your only real option is to try to find a dataset that approaches the real distribution, or to try to somehow augment your data to make it look more like it (although this is very challenging to do without introducing even more bias).
Hyperparameter tuning is out. Its much harder to overfit anything nowadays unless you re trying to using something like yolov8.
Its always been about the dataset.
What are the differences between the training objects and the real targets?
Generally speaking you're not going to bridge the gap with the hyperparameters, you may do so with augmentations or synthetic data, share more of the issue.
Yeah I thought that might be the case.
The projects goal is the following:
The problem is the following:
I have annotated images of multiple growth stages of fish, but the average growth stage of the fish in the training data will almost always be either smaller or bigger than the ones im measuring.
So when I'm training a model on all data I have and then running the model in a tank of fish that are at the upper end of growth, than the model will detect the smaller fish inside that tank more often, because most fish in the training data are smaller then the fish in the tank.
Does that make sense?
These values are just to show what I mean (Expecting that the model is always trained on all 5k samples)
| growth stage | 1 month | 2 months | 3 months | 4 months | 5 months |
|---|---|---|---|---|---|
| training samples | 1000 | 1000 | 1000 | 1000 | 1000 |
| length error on real live data | + 10% | + 5 % | 0 % | - 5% | -10% |
If that’s not something that you already do, use data augmentation. Especially scaling.
It might not help for features related to age (such as color changes and the likes), but it can definitely help remove the relative size bias, especially if your dataset was created in a single scenario with a fixed camera distance.
If you do know some features of "an older fish", you can also apply those transformations on masks of younger fish from your first trained but biaised model. Somewhat like a "semi-synthetic" data augmentation.
For example, if older fishes are browner, you can skew the hue of the masks by a certain amount to get brownish young fishes.