Icko_

You could just add an extra channel with the extra information. E.g. you want to use a pretrained model. You take (and freeze) everything but the last few layers. Let's say the frozen model reduces the image to 224x224x100. Just add a few more channels (with constant value for all pixels), e.g. 10 channels for color, 20 for product type, etc. This will work if the you have training data. If not, maybe something like what u/saintshing said might be the way to go.

For some datasets, it's not possible to go to 90%. Just figure out some reasonable ranges for your most important hyperparameters (you don't have to optimize ALL of them), and do grid search for those values. That's all there is to it.