Selfhosted

I’m in the same kinda situation as you, I need some storage but need it to be expandable, want to run some docker stuff, while I could (and have in the past) build and maintain something like that from scratch, I don’t want it to take over my life and I want it to be easy to maintain. My previous NAS was fully set up from scratch on FreeBSD, it was pretty good but was a lot of work to get it right.

So I set up an Unraid server on a parts-bin server as a kinda compromise between a fully DIY and just buying a NAS. Meant I could use some old stuff I had and some cheap components rather than paying out hundreds for a NAS. Slapped in some shucked drives and some old NVMe drives (took the opportunity to upgrade my gaming machine, so used the old stuff for this), now got 42Tb of storage and 2Tb cache.

I have to say it’s bloody fantastic. Was a bit on the fence about a paid OS but it’s cheap, the UI is solid, and thus far totally worth the money.

Alongside about a dozen services running in containers, I’ve got an Arch VM to satiate my DIY cravings, which suits me fine because I can do what I want with that without messing up my file storage/services/etc.

If you are running a media server, depends what you going to do with it. Storing media without transcoding you don't need much, ex-gov computer from last 10 years x86_64 CPU that could store a HDD, you could use openmediavault (Linux), Unraid (paid - linux) or freenas (freeBSD) in a JBOD config and a ssd for cache (so you can serve databases, and metadata quickly, like plex or Jellyfin) and it will be much faster than MyBook Cloud.

Transcoding media, that is converting media from a format to another format so you can be compatible is a different story. You will need at bare minimum intel with quicksync or a fast CPU for software encoding.

Generally with DIY NAS software, you load a docker, point the docker directory (fake) to your jbod directory and it will just deploy on the webUI port you assigned it too.

Generally I will do this imo. Cheap intel PC (eg. Ex-gov) with a HDD > upgrade to SSD for a cache drive > DIY PC with plenty of SATA ports > upgrade to NVME > Cheap GPU for transcoding OR SAS card for more harddrives.

If the data is really important, make sure you have a parity drive and a backup solution.

I can't answer all of it, but much of what you're asking for can be accomplished with a simple samba share. If you can handle nextcloud, you can set up samba.

It's perfectly reasonable to use the same device to run your web services and as your NAS. There's no reason you can't divide them up later if you want.

You'll need to pick a file system, I suggest either BTRFS or ZFS though there are several options. BTRFS is neat because it's flexible - you can make huge changes without ever dismounting. You'll want to plan for a multiple-drive solution, and you'll need to decide how you'd like to balance performance, space efficiency, and failure tolerance. Whatever you do, pick one single drive size and stick with it -- different disk sizes xan work, but there are restrictions and they complicate things.

A good backup is automatic, versioned, and encrypted. You preferably want one offsite and one onsite for anything irreplaceable. Restic is a good tool, as is Borg backup, as are many other options. Personally I run a restic job nightly, with backblaze b2 as a destination. I also have a local backup on normally-unplugged drives that I run manually every couple weeks.

For plumbing, tailscale is really nice. Easy to set up, and you get remote access to everything with minimal config and no holes in your firewall.

Regarding hardware, you have many options. Old laptops actually make great homelabs:

• energy efficient
• built-in UPS
• no need to drag a keyboard/monitor/mouse over when you can't access via SSH for whatever reason
• usually plenty performant for the task

Their biggest drawback is a lack expandability/upgradability, though you can get pretty good USB drive bays to partially address that. Another option is the intel NUC family and its competitors, basically tiny desktops built out of laptop parts. A third option is to build a normal desktop PC, either into a normal case or a rack-mount one if you have the space. The off-the-shelf options work, but are limiting in my experience. That said, they're the way to go if you don't want to do a lot of tinkering.

Whichever solution you go with, personally I wouldn't start with any less than 4 drive bays. More is better, you can't have too many. You should be able to shuck your old drives and put them into any 3.5" drive bay.

For reference, my setup is an intel NUC with a thunderbolt 10 drive bay plugged in. I have a mishmash of disks, ~48tb total in BTRFS raid 1giving me 24tb usable. Running a good handful of docker containers and a samba share, all accessible over tailscale.

Sorry for the info dump, happy to answer questions.

Does it make sense to mix my uses, i.e. media server, open project, etc. co-existing with file server for my docs and general files. Can I segregate portions for only local access?

Yes and yes.

I don't have tons of time to maintain this. Nextcloud hasn't been a pain, I log in here and there and make sure everything is updated (nextcloud and the server) and I run the NextCloud security scan to make sure I get an A+. Does it make sense to go for something like the better Synology NASs that can run docker images or would it provide better affordability/functionality to use a mini-pc or a FBmarketplace/craigslist slim pc hooked up to a drive enclosure or something else frankenstein-y. I don't mind doing basic maintenance, but I can't afford to spend every other weekend rebuilding things.

Most of the time, setting up for the first time and tweaking for maybe couple weeks top is a pita. I have not spent more than an hour a month after the initial setup phase. Realistically, you wouldn't need more than what I spent.

I had one time that my OMV fucked up my docker containers but spent about 30 min and it was back to normal. All the other problems were network problems which will happen even if I used Synology.

I have a dead WD MyBook Live and MyBook Cloud on my shelf. WD never updated them to fix the critical security issues, I missed the 40% off upgrade window, and they're not safe to run with network access. They also sucked even when they were new. I want to avoid products doomed to become dead-end abandonware before I'm ready to upgrade. Are there NAS brands that are known to be better/worse with this? How does homemade NAS fare as far as hardware support and having to upgrade/rebuild when OS versions change.

In the end, if you go with the premade NAS, you will want Synology. There are no real competitors in the market atm.

Homemade NAS has been constantly getting better. Trunas is amazing. OMV is nice although I'm using it just because I'm too lazy to upgrade to Truenas.

I can't say anything about other NAS brands but homemade NAS is going to get basically the best and the fastest updates there is, depending on the OS of uour choice. They will be painless for most of the time.

Can I purchase/build a simple NAS that I use for storage and serve the files for my media server through a different device like my laptop? Is this better/worse than just streaming from the NAS itself or will I not notice in most cases?

If you're using homemade NAS, that will be more than enough. If you're using something like Synology, you'll be fine but also want something better depending on your usecase.

It sounds like some of the pre-built machines can use drives of different sizes which would allow me to re-use the barely used drives inside of the WD devices. Do any of the self-build solutions allow for this

Check out Unraid. I personally would not do that though.

I would LOVE some book/media/community recommendations for digital hygiene and how to handle store, backup, maintain the deluge of information in our modern lives.

Ask here!

Feel free to ask more/correct me.

I had been self-hosting stuff on my QNAP NAS for years before it died due to the infamous Intel clock drift issue and now I am in the process of making a DIY NAS (last few parts are coming in this weekend). I don't have answers to all your questions but I'll try my best with the experience that I have.

1. It is absolutely possible to mix your usecases on one machine, with the caveat that if you're running on less-powerful hardware (like a off-the-shelf NAS), some of your services might be competing with each other for resources. CPU usage and disk access times (especially with a RAID 5 HDD array) can all impact performance. My QNAP NAS did start to bog down a few times with both Jellyfin and Nextcloud running at full tilt, but it was generally pretty usable.

2. Most NAS products support docker images so I wouldn't worry too much about NAS vs PC in this case. Also, docker-compose is your friend. Write your yaml file once and it will make for easy setup and upgrading.

3. Dude, I am with you on dead-end products. The death of my QNAP NAS has caused me lots of headache and I basically swore off products that I can't upgrade and fix myself. The problem is price. The cheapest x86 PC that I personally think will handle multiple usecases (media server, Nextcloud, SAMBA, maybe a Valheim server or a VM when I need it) costs roughly around $650-$750 depending on your build. You can probably find a Synology or QNAP NAS for about $500-$550. Granted, they most likely aren't going to be anywhere near as powerful as a DIY x86 PC, so I think its worth going the DIY route. Those prices do NOT include the drives either, so be sure to factor that into your calculation. If you're curious, here's one of the cheaper builds I was considering building: https://pcpartpicker.com/list/rtqDbK. Ultimately I decided to go for a crazier build because I did not want slow HDDs anymore: https://pcpartpicker.com/list/Lm92Kp

4. You mean running a media server on your laptop, but pointing the media libraries to a Samba share on a NAS? I did that for years with my QNAP NAS and a little Intel NUC running Plex. The only issue is that you won't get incremental media library updates whenever you add new files into the Samba folder. Usually, Plex (and Jellyfin) can detect file changes if the media library is local and automatically process only those files instead of rescanning the entire media library. Over Samba, there's no such automatic detection so whenever you add a file, you have to manually trigger a full rescan in order for it to pop up in your media library.

5. I believe Unraid does this. I have not tried it myself and I plan on going with ZFS for my DIY NAS.

6. I don't have any resource recommendations, but personally I've taken the docker-compose approach which helps quite a bit for isolation. For media servers, you only need to give read-only access to the volumes hosting your media storage. It is also recommended to put media servers like Jellyfin behind a reverse Nginx proxy because Nginx has been battle-tested in terms of security and Jellyfin's web server has not. You can use docker-compose to easily spin up a Nginx proxy alongside your media server and have them contained in their own isolated network.

Do not open any more ports than is necessary to host your services. This means even remote administration should not be available via your public IP. Learn how to setup Wireguard so that if you're away from home, you can quickly VPN into your network and do remote administration. If you're using SSH, make sure you disable password authentication and only rely on SSH keys. I am sure other people can add more, this is just the basics.

Hope this helps!

I have multiple devices split across two locations and I end up having to use hard drives to periodically move files back over to my main desktop for sorting and archiving. If I want to access older files, I have to copy them from my main storage on the desktop to a hard drive, my NextCloud, or whatever device I want to access them on. I would like to avoid this drudgery by moving my file storage to a NAS

A NAS is a good idea but do note that this sort of setup can work aswell with the correct tooling.

don't really even need access outside the network, though it could be useful if I understood it enough to keep it secure

I can highly recommend Tailscale for this purpose.

run a few docker images for things like media server, open project, restyaboard, etc. I'm not sure if it makes sense to do this on the NAS or just get a simple NAS and do this stuff in a VM on my laptop or with a Rasberry Pie.

Depends. Many people host such things on their NAS since the NAS is always on anyways and barely does anything most of the time, so it's perfectly valid to do that.

Can I purchase/build a simple NAS that I use for storage and serve the files for my media server through a different device like my laptop?

Yes but in a home setting, it usually makes sense to keep the services running on the same device which stores the service's data.

It sounds like some of the pre-built machines can use drives of different sizes which would allow me to re-use the barely used drives inside of the WD devices. Do any of the self-build solutions allow for this.

Sure. Unless you need assistance setting up a Linux system (I doubt you do) or building a computer, self-built is almost always better. I use a low-power Intel single-board-computer (Celeron J4105) in a small PC case for this purpose.

For pooling different sized drives, I use btrfs but the same could be achieved using ZFS or even LVM.

Do note that, unless you have specific uptime requirements, RAID is pretty wasteful in a home setting in both hardware and time thinking about it.
When it comes to digital hygiene, figure out backups first. 3-2-1!

I would LOVE some book/media/community recommendations for digital hygiene and how to handle store, backup, maintain the deluge of information in our modern lives.

I've found git-annex for myself. It's quite a rabbit hole and takes a lot of effort to understand and really use well but it's an incredible tool that has greatly aided simplifying my storage setup.

One of the best things about it is that it separates data from metadata. You always have the metadata but you don't need to hold the data all in one place.
This means you can re-organise files on your laptop without those files actually being present on the laptop. They could be anywhere; on a hard disk sitting unplugged on a shelf, in the cloud, on some other machine that's turned off, on the NAS etc. but you can move them around in the filesystem as if they were there. If you needed some file's content content, you can ask git-annex where the file's content is stored, i.e. plug in the hard drive and then ask it to copy the data over.

That's the gist of it (git-annex can do a bunch of other cool stuff) but this really helped my get my shit together w.r.t. storage hygiene.

I found that running just a NAS is no good because it lacks performance. And running just a server is no good because of data safety. So I run both.

I run a Synology NAS for storage, and an Ubuntu Box for services (Paperless, Mattermost chat, Photoprism gallery, Gitea for Code, picoshare, and lots other small stuff). All services run on Docker (plus Portainer for handling them, plus NPM for certificate handling and subdomains).