If that’s not something that you already do, use data augmentation. Especially scaling.
It might not help for features related to age (such as color changes and the likes), but it can definitely help remove the relative size bias, especially if your dataset was created in a single scenario with a fixed camera distance.
If you do know some features of "an older fish", you can also apply those transformations on masks of younger fish from your first trained but biaised model. Somewhat like a "semi-synthetic" data augmentation.
For example, if older fishes are browner, you can skew the hue of the masks by a certain amount to get brownish young fishes.
If that’s not something that you already do, use data augmentation. Especially scaling.
It might not help for features related to age (such as color changes and the likes), but it can definitely help remove the relative size bias, especially if your dataset was created in a single scenario with a fixed camera distance.
If you do know some features of "an older fish", you can also apply those transformations on masks of younger fish from your first trained but biaised model. Somewhat like a "semi-synthetic" data augmentation.
For example, if older fishes are browner, you can skew the hue of the masks by a certain amount to get brownish young fishes.