Many people talking about using subdomains, but that's only really a thing if you actually have a domain. Just last year the domain .internal was reserved for internal use, so that's what I've set up all my domains to use. E.g. https://pihole.internal/, https://proxmox.internal/.
To make this work I use pihole's local dns records to rewrite any *.internal domain to point to my reverse proxy Caddy's ip.
As for the certificates, I created my own CA, which I install on all my and my family's devices. Then, for each new url I set up, I create a new certificate and sign it with my CA certificate, then have my reverse proxy serve it.
This all sounds like a lot of work, and it is, but using OPNsense for both reverse proxy and certificates makes it well integrated and certificates are trivial to renew. With that said, if you have your own domain, go the let's encrypt subdomain route instead imo. It saves you a lot of manual labor with setting up your CA on every device you own and creating new certificates for each site.