If you are all linux, perhaps you'd like Borg. It's pretty easy to set up, and perhaps it offers the de-duplication you crave. It's also got some GUIs you can run, if you are into that kind of thing. If you don't feel inclined to use any backup software, then using rsync over ssh (scp, yes) instead of SFTP is a solid way to go.
emuspawn
I personally run a ProxMox cluster. I run both Windows and Linux servers. I perform local full-VM backups using the hypervisor to a USB disk. That gives me a fast way to restore VMs if I need to. I also run Veeam, which handles the offsite copy and provides granular file restores. It's nice because the community edition supports hardened disk immutability, which can help prevent ransomware attacks and Unfortunate Incidents. That just runs over SSH, and installs a Veeam agent/repo on the remote linux box.
Tell us a bit about your environment! Are you all linux or do you have Windows as well? Are you running a hypervisor like Proxmox or VMWare or using containers? Are you just making complete backups, or can you forsee yourself needing granular file restores? There are a number of ways you could go, depending on your setup.
Thanks! These were experimental so I'm glad I got some eatable ones! A shame about the beetles.