Programming

You may want to ask this in a selfhosted community, not in programming. With that out of the way:

I don't think hosting your own DNS server on VPS will help much, for several reasons:

• As you noticed, if you connect to the VPS by name then you can't resolve that name with the DNS server you will only reach after connecting.
• Your hosted DNS server will still need to get its information from somewhere, meaning it will query other DNS servers, meaning information about which domains you visit will still "leak".

DNS "leakage" happens in two ways:

1. The DNS server directly upstream from you knows what domains you want and can associate them with your identity. This applies to scenarios like you using your ISP's DNS server from the home being served by that ISP, you using your VPS provider's DNS from a VPS issued by that provider, using Google's DNS if you use any Google services on any devices (they spy on a tremendous amount of things, even your IP is enough to associate you) etc.
2. You connect to a random DNS server that doesn't know anything about you or doesn't care, but the DNS protocol is not natively encrypted, meaning anybody on the way can spy on your queries. Which brings us back to the usual suspects: your ISP, your VPS provider etc.

To fix DNS leaks you need to do two things:

a) Use a DNS service that has an explicit mission statement of protecting the users' privacy. Here's a good start.
b) You need to connect using encrypted DNS. The most widespread form is DoH (DNS over HTTPS) which uses port 443 and is virtually indistinguishable from regular web traffic (aside from the fact it connects to known public DNS servers). You can also use DoT (DNS over TLS) on port 853 (as opposed to unencrypted DNS on 53).

You can set up DoH or DoT with the address of a privacy-respecting public DNS service on a wide variety of apps and devices:

• On Android or iOS you can set it up directly on the device, and force all DNS queries from that device to always use that service.
• You can set it up in recent versions of Firefox.
• You can set it up on your router, for example on OpenWRT if you run that, and force all devices on the LAN to resolve through that encrypted service.

There are also downsides to DoH/DoT. For example, you can't coerce LAN devices or apps that use a hardcoded DoH/DoT server to use the one you want. You could hijack their name resolution to the server name but you can't satisfy their TLS certificate, especially if it's also hardcoded and doesn't rely on a central store (like the Android or iOS certificate store). This is often the case with Chinese ioT devices who like to phone home. Google has also started to do this with Chrome on mobile, to prevent DNS-based adblocking.

Use https://www.dnsleaktest.com/ to test what you leak.

Just use the IP address of your VPS?

Keep in mind that using your own VPS as a VPN doesn't bring anonymity. You're simply replacing one IP tied to your name (your ISP) with another one (your VPS).

You hide your traffic from your ISP, and delegate it to your VPS provider.

This will be the same for your DNS. If you want true anonymity regarding DNS, you should use someone else's service, preferably over encrypted channels, eg. cyberia.is DoT.

I personally use it as a forwarder from a box inside my home (along with others), and use this box as the local DNS when I'm home. This way I know that all DNS traffic is encrypted, and doesn't leak anything to my ISP or VPS or whatever.

DNS is handled by peer - what kind of leak are you experiencing?

You don't have to set up your own resolver. It is enough to configure route to 1.1.1.1 via WireGuard peer. If you already use it as a default gateway, your DNS requests don't leak (I mean, Cloudflare is unable to associate them with your local IP address). To be sure, check traceroute 1.1.1.1 (on *nix system) or tracert 1.1.1.1 (on Windows), you should see your WG peer address in the output.

Random VPN service cannot determine if your DNS server trusted or not, it only checks if the server is provided by that service. When using your own WG server, such checks are useless.

It’s not completely clear what you mean, but I’m guessing you’re only routing a subset of your traffic through wireguard, probably only IPv4, and there may be some IPv6 traffic that is not being routed over your wireguard connection.

You can specify any IPs you want for DNS with wireguard, and if your allowed IPs include those addresses, then it should flow over your VPN.

I do this with Pihole at home, and it blocks ads while I’m away.

With whatever test you’re running that says stuff is “leaking,” keep in mind that the website is going to report any traffic that originates from your VPS as “unprotected” because it’s not their system, and even if you run your own DNS server, it’s still got to query upstream to a public DNS. All they’re really doing is demonstrating which upstream DNS server you have configured, and it’s up to you if you want your VPS’s IP to be connected to the query history of that upstream DNS provider.

You will usually need a hostname in DNS for your VPN server to make it easy to find/connect, which will use your normal DNS resolution. Once connected, if you have it set up correctly, new dns queries should route through your VPN connection. Just keep in mind that various results can be cached on your system and in web browsers, so you should quit and reopen your browser after you connect to the VPN before you run your “leak” test.