Edit: The extension now lets you see lemmy comments on any website!
A couple weeks ago I reached out to the dev that makes the Reddit Comments for Youtube extension and asked him to enable support for Lemmy. The developer was already rewriting the extension and decided to implement Lemmy support as well! Here's the GitHub and FireFox links. It also available for Chrome iirc.
The comments appear after you select an instance in the extension's menu on any youtube video that has been posted to any Lemmy instance with a full url without a timecode. Invidious links and Lemmy comments on invidious instances still aren't supported but since its supported by the Reddit comments it should be possible if someone submits a PR.
Seems like it’s not something they want to work on. they might accept a PR though