Selfhosted

Yeah, but my wife and kid also use it, and they're not going to be happy if I change things.

Yep, VPN will allow you to be on the same local network, and they're only pay walling remote play.

realistic security concerns

If you're running a binary installation of Jellyfin on your server and exposing it to the public internet, you can face significant risks:

• Remote execution vulnerabilities might allow attackers to exploit bugs to run malicious code on your server.

• Buffer overflows. Poorly handled data can let attackers manipulate memory, Bypass logins, touch things in the host that aren't meant to be twiddled with

• Network exposure. If compromised, the server could become a launchpad for attacks on your network.

There might not be any vulnerabilities at this moment, but they might come in a future release. And we might not even know they exist. It's a small team of volunteers, and they'll do their best. This is just what is reasonably possible when installing the server as an application on your OS and exposing it to the Internet.

You can minimize risk with a safer setup, as someone else in the comments here mentioned (and I think they even linked to their setup)

Using a Docker container version of the app significantly reduces your attack surface. This isolates the app from your host system. If they get in, they only get into the container and whatever that container is allowed to do.

Mount your media files as read-only to prevent accidental modifications or potential malicious changes. Now that container can't do any real harm do your data.

Avoid making the container privileged. A privileged container can interact with the host system in risky ways.

Use reasonable unique usernames and passwords. If the container does manage to get compromised, they will likely be able to read usernames and passwords stored in the container.

Regularly update your container – Ensures you have the latest security patches.

Short of some massive Docker vulnerability, (which is on you to keep updated) the worst case should be public enumeration of your media, exposure of your JF users/passwords, and denial of service. Which IMO isn't very serious.

For even tighter access control, don't whitelist the entire world.

Whitelist specific IP addresses. Have users visit WhatIsMyIP to get their IP, then configure port forwarding to allow only trusted addresses. This allows the clients at their houses in without any serious hinderance, but would block them from accessing your media when they're not at their house.

If they're accessing you through a phone or PC, setup headscale or tailscale or any VPN and allow them to get to you through VPN

I would be very interested in an answer to this as well. Also any how to guides that would be useful for a guy whose technical high-water mark was getting mint set up on my laptop.