The field is broad, with no easy answer and many nuances, you could fill countless PostDocs and Phds with work here.
Judging by the way you phrased your question, very generically and without essentially any detail, I'm going to take a wild guess and say that you're either a beginner in ML ot haven't read/studied anything on the subject.
I'd encourage you to start by reading recent and not-so-recent papers dealing with inherently multimodal tasks, like scene text recognition, VQA, and the like. The big problem in the field is that the models will, generally speaking, try to overfit on the modality that's mostly information-dense between V and L for your task. In current SoTa methods, the best way to mitigate this seems to be fusing the two modalities via a gradual mechanism, for example the gated tanh attention of Flamingo.
Happy reading!
Just download more RAM! https://downloadmoreram.com