You're not off the mark. It's a great idea for a low traffic server that doesn't guzzle electricity.
I suggest an SBC with 4GB RAM (or more) and 4 cores, though you could probably get by with a bit less. If you use a Raspberry Pi, make it version 4. If you're going to use mechanical hard drives (good value for bulk storage) consider a board with native SATA or PCIe for lower interrupt overhead and better SMART access than a USB bridge would provide.
Debian is a great server OS. There are derivative distributions (e.g. Armbian & Raspbian) for boards that aren't supported by mainline Debian.
Software will depend on your needs as a user and your preferences/experience as an admin. If you end up wanting something like Docker, you might want to try Podman instead, as it's less resource-hungry.
Also, check out the self-hosting forums. You're likely to find more people doing the same sort of thing.