You'll have to use nginx's ACL feature. Like you've discovered once someone has access to the proxy itself, by its very nature it's acting on behalf of the client making the request so firewall rules won't help much there. You'd have to get into packet inspection, and deal with https man in the middle stuff and all that, likely not worth it.
I'm not familiar with nginx proxy manager which you seem to be using but in the config files it'd look like this:
nginx.conf
location / {
include /etc/nginx/acl_file_name.acl;
deny all;
# rest of location config
}
the deny all; at the end means everything that wasn't listed in the ACL file is denied. You could alternatively put the deny all directly in the acl file at the end of the file.
the acl file itself is just a list, i create it in a file for easy re-use, and made up the ".acl" extension because it makes sense.
acl_file_name.acl
allow 192.168.1.0/24;
allow 192.168.2.101;