this post was submitted on 15 Oct 2024
156 points (98.8% liked)

Linux

47730 readers
834 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

Can you please share your backup strategies for linux? I'm curious to know what tools you use and why?How do you automate/schedule backups? Which files/folders you back up? What is your prefered hardware/cloud storage and how do you manage storage space?

(page 2) 50 comments
sorted by: hot top controversial new old
[–] Kongar@lemmy.dbzer0.com 7 points 1 day ago (1 children)

Synology NAS. I really love that thing. I use their synology drive software to backup the Linux home folder, as well as windows PCs, iPads, iPhones etc. I use their photos mobile software to automatically backup phone photos and videos. I also synchronize a few select folders between PCs so certain in-use files are always up to date. I set the NAS to keep 30 old versions of every file. This works great for my college kids - dad has a copy of everything in case they nuke a paper or something (which has happened).

I stopped cloning drives long ago. Now I just reinstall the os and packages. With Linux, this is honestly faster than deploying a backup - a single pacman command installs everything I want. Then I just log into things as I open them. Ya I might have to futz around with some settings or redownload some big games on steam - but the eye candy and games can wait - I can be productive pretty quickly after an install.

I DO use btrfs with automatic snapshots (snapper and btrfs assistant). This saves me from myself when I bork an update (which I’ve done more than once). If I make a mistake, I just rollback a snapshot, and try again without my stupid mistakes. This has saved my install 3 or 4 times now.

Lastly, I sneaker net an external hard drive to my office. On it is a manual backup of the NAS. I do this once per month. This protects from catastrophic failures like my house burning down. I might lose a month or so of pictures in the worst case scenario, but I still have my 25+ years of pictures of my kids, wedding videos, etc.

In the end, the only thing that really matters is not losing my lifetime of family pictures and the good memories they provoke.

[–] bad_news@lemmy.billiam.net 1 points 1 day ago

I've been using btrfs for x years now and it's never occurred to me to use the snapshots 🤦

[–] FrostyPolicy@suppo.fi 3 points 1 day ago

Daily rsync to a local nas and weekly backups to offsite with pika-backup.

[–] JustEnoughDucks@feddit.nl 3 points 1 day ago* (last edited 1 day ago)

321

Kopia backup to secondary HDD

  • Pictures (phone photos backed up to my server via immich)
  • workspace (git repos, ECAD, MCAD, firmware, etc...)
  • qmk layout
  • Documents
  • vim folder with bundles
  • ebooks

KDE vaults stores on secondary HDD

Soon I will set up kopia to also back up every via SSH to my server and then small size essentials and important docs via google drive

I need to set server cloud backups too, but haven't had the time...

[–] Vincent@feddit.nl 5 points 1 day ago
[–] sloppy_diffuser@sh.itjust.works 5 points 1 day ago (1 children)

I use immutable nixos installs. Everything to redeploy my OS is tracked in git including most app configurations. The one exception are some GUI apps I'd have to do manually on reinstall.

I have a persistence volume for things like:

  • Rollbacks
  • Personal files
  • Git repos
  • Logs
  • Caches / Games

I have 30 days (or last 5 minimum) of system rollbacks using BTRFS volumes.

The personal files are backed up hourly to a local server which then backs up nightly to B2 Backblaze using rclone in an encrypted volume using my private keys. The local server has a mishmash of drives in a mirrored LVM setup. While it works well for having mixed drives, I'll warn I haven't had a drive failure yet so I'm not sure the difficulty of replacing a drive.

My phone uses the same flow with RoundSync (rclone + GUI).

Git repos are backed up in git.

Logs aren't backed up. I just persist them for debugging and don't want them lost after every reboot.

Caches/Games are persisted but not backed up. Nixos uses symlinks and BTRFS to be immutable. That paradigm doesn't work well for this case. The one exception is a couple game folders are part of my personal files. WoW plugin folder, EvE online layouts, etc.

I used to use Dropbox (with rclone to encrypt). It was $20/mo for 2Tb. It is cheaper on paper. I don't backup nearly that much. Backblaze started at $1/mo for what I use. I'm now up to $2/mo. It will be a few years before I need to clean up my backups for cost reasons.

The local server is a PC in a case with 8 drive bays plus some NVME drives for fast storage. It has a couple older drives and for the last couple years I typically buy a pair of drives on sale (black Friday, prime day, etc). I have a little over 30TB mirrored, so slightly over 60TB in total. NVME is not counted in that. One NVME is for the system, the others are a caching layer (monero node) or temp storage (transcoding as it also my media server).

I like the case, but if I were to do it again, I'd probably get a rack mountable case.

[–] ikidd@lemmy.world 2 points 1 day ago* (last edited 1 day ago) (2 children)

You seem pretty organized in your strategy, I would suggest you just pull a drive in your LVM to check how that goes for you. I've had issues in JBOD style LVM volumes with drive swaps, but YMMV.

Frankly, I use ZFS now in anything that I would have use LVM in before. The feature set is way more robust. Also, an offsite ZFS replication to zfs.rent is a good backup of a backup. But Backblaze is pretty solid too.

[–] sloppy_diffuser@sh.itjust.works 1 points 1 day ago (1 children)

Good call on a simulated failure. When I first set it up, it was LVM/BTRFS or ZFS as my top choices. It was a coin toss at the time because I hadn't built this sort of setup before.

[–] ikidd@lemmy.world 1 points 1 day ago

Yah, an untested raid is like an untested backup: suspect.

[–] 0x0@programming.dev 1 points 1 day ago

Hetzner's storage boxes have caught my eye but i haven't tried them yet.

[–] JubilantJaguar@lemmy.world 5 points 1 day ago (1 children)

Here's one that probably nobody else here is doing. The backup goes on my mobile device. Yes, the thing in my pocket.

  • Mount it over SSHFS on the local network
  • Unlock a LUKS container in the form of a 30GB sparse file on the device
  • rsync the files across
  • Lock, unmount

The backup is incremental but the container file never changes size, no matter what's in it. Your data is in two places and always under your physical control. But the key is never stored on the remote device, so you could also do this with a VPS.

Highly recommended.

[–] tonyn@lemmy.ml 1 points 1 day ago (1 children)
[–] JubilantJaguar@lemmy.world 1 points 1 day ago (1 children)
[–] tonyn@lemmy.ml 2 points 1 day ago (1 children)

If your local machine dies, and you have a backup on your phone which you cannot unlock... aren't you screwed?

[–] JubilantJaguar@lemmy.world 1 points 1 day ago

Good question. No, but at a small cost in security. The key I generated using sha512sum using a very solid memorized passphrase. This means I can regenerate the key in the scenario you describe.

[–] LemmyBe@lemmy.world 3 points 1 day ago* (last edited 1 day ago)

I use Bluebuild to create a reproducible system, plus a post-install script to handle other post-install tasks such as setting up initial preferences.

Also Vorta to backup files and settings to external HD, plus OneDrive Linux client to sync files and settings to cloud.

[–] clif@lemmy.world 1 points 22 hours ago* (last edited 22 hours ago)

Internal RAID1 as first line of defense. Rsync to external drives where at least one is always offsite as second. Rclone to cloud storage for my most important data as the third.

Backups 2 and 3 are manual but I have reminders set and do it about once a month. I don't accrue much new data that I can't easily replace so that's fine for me.

[–] oscardejarjayes@hexbear.net 3 points 1 day ago* (last edited 1 day ago)

restic to a local server and to cloud storage. it varies by device, but usually just everything in /home/. The rest of the operating system should be reproducible, whether through images, ansible, nix, or guix, given the information in /home/.

scheduling is done through systemd, usually (or the non-systemd equivalent). I use BackBlaze now, but I switch around occasionally. restic has policy based snapshot removal, and a prune option.

[–] synapse1278@lemmy.world 4 points 1 day ago

I use Duplicity to backup my home directory, excluding Steam and Downloads folders. It is setup to backup weekly to my NAS mounted as NFS. The NAS has a weekly cron task to upload the backups to pCloud using rclone. I backup this way, several computers (2 desktop, 2 laptop, the NAS as well). The files included in this strategy are essentially my photos, documents and configs. My software installations, games, media library are not backed up.

[–] CrabAndBroom@lemmy.ml 4 points 1 day ago

Currently I use Borg Backup with Vorta as a GUI. I don't really do anything automated/scheduled, I just back it up manually to an external SSD every few days or so. I pretty much do my whole /home folder, except for a couple of subfolders that aren't really necessary (and Videos, which I back up separately.)

I do eventually want to upgrade to a NAS, but I'm waiting until we move to start setting that up. Also I don't really have an off-site plan yet which I know is bad, but I need to figure that out.

[–] smeg@feddit.uk 4 points 1 day ago (2 children)

The important stuff is in cloud storage using Cryptomator (I'm hoping that rclone should make sync simple), I should probably set up time shift in case things do go wrong

load more comments (2 replies)

Nightly rsync to two NAS boxes in the house (TrueNAS Scale and a Synology). Docs go in NextCloud, hosted on a VM in my basement, which is also backed up to the Synology by Proxmox. Also backing up my main machine (Pop!_OS) and my wife’s laptop (ThinkPad E595, also Pop!_OS) using Spideroak One.

[–] seaQueue@lemmy.world 3 points 1 day ago* (last edited 1 day ago)

I leverage btrfs or ZFS snapshots. I take rolling system level snapshots on a schedule (daily, weekly, monthly and separately before any package upgrades or installs) and user data snapshots every couple of hours. Then I use btrbk to sync those snapshots to an external drive at least once a week. When I have all of my networking gear and home services setup I also sync all of this to storage on my NAS. Any hosts on the network keep rolling snapshots stored on the NAS as well.

Important data also gets shoveled into a B2 bucket and/or Google drive if I need to be able to access it from a phone.

I keep snapshots small by splitting data up into well defined subvolumes, anything that can be reacquired from the cloud (downloads, package caches, steam libraries, movies, music, etc) isn't included in the backup strategy. If I download something and it's hard to find or important I move it out of downloads and into a location that is covered by my backups.

[–] m4m4m4m4@lemmy.world 3 points 1 day ago (1 children)

Not only because third world issues, but because I like adrenaline, I don't have any backup strategy but an old external HDD where I haven't copied stuff since 2018.

When I could afford a new PC and tried to rsync my data from my old crappy laptop, much of it was lost.

That being said, I had a backup strategy back in the day that was burning CDs. I used to have a second HDD (a IDE one) but they were so freaking bad all of them went bad after a year or so, so I have like 3 or 4 of them stored without any chance to recover their data.

[–] smallpatatas@lemm.ee 1 points 1 day ago

After having recently restored some stuff from an aging external hdd, i'm seriously considering getting a few dvdr discs and burning the important things every now and then.

I know they don't last forever either, but - just as a random example that has definitely never happened to me hahaha - you can drop them from a height of 3 feet and still get files off them!

[–] ikidd@lemmy.world 3 points 1 day ago

Keep everything on Nextcloud and back that up via Proxmox Backup Server.

Nuke and pave takes me less time to reconfigure Plasma and install NC client than bothering to back anything up directly.

[–] Minty95@lemm.ee 3 points 1 day ago* (last edited 1 day ago)

Timeshift for the system, works perfectly, if you screw up the system, bad update for instance just start it, and you'll be back up running in less than ten minutes. Simple Cron backups for data, documents etc, just in case you delete a folder, document, image etc . Both of these options to a second internal HD

[–] xlash123@sh.itjust.works 2 points 1 day ago

For my home server, I use Restic and a cronjob to weekly take snapshots of all my services. It then gets synced to a Backblaze B2 bucket (at $6/TB/mo). It's pretty neat, only saving the difference between the previous and current snapshot, removes older snapshots, and encrypts everything.

[–] savvywolf@pawb.social 3 points 1 day ago

Firstly, for my dotfiles, I use home-manager. I keep the config on my git server and in theory I can pull it down and set up a system the way I like it.

In terms of backups, I use Pika to backup my home directory to my hard disk every day, so I can, in theory, pull back files I delete.

I also push a core selection of my files to my server using Pika, just in case my house burns down. Likewise, I pull backups from my server to my desktop (again with Pika) in case Linode starts messing me about.

I also have a 2TiB ssd I keep in a strongbox and some cloud storage which I push bigger things to sporadically.

I also take occasional data exports from online services I use. Because hey, Google or Discord can ban you at any time for no reason. :P

I have my important folders synced to my Nextcloud and create nightly snapshots of that to a different drive using borg.

One thing I still need to do, is offsite encrypted backups using rsync.

[–] b34n5@hexbear.net 3 points 1 day ago

I really make backups only a few times. I have the configuration files of my systems on my GitHub and Codeberg. The rest, I don't need; the only things I keep are books and music that I download from the internet, which I have on a 1TB external hard drive.

When I have made a backup for a specific reason, I have done it with rsync. It's a tool that works quite well and is for the command line.

[–] hallettj@leminal.space 3 points 1 day ago

When I researched this previously I concluded that there are two very good options for regular backups: Borg and Restic. These are especially efficient at backing up a diff of what has changed since the last backup. So you get snapshots of your filesystem state at each backup point without using a huge amount of space. You can mount any snapshot as a virtual directory. After the initial backup, incremental backups take a minute or two.

I use Borg, and I back up to cloud storage on Borgbase. I use Vorta as a GUI for Borg. I have Vorta start automatically when I start my window manager, and I have it set up for daily backups. I set up the same thing on my kid's computer.

I back up my home directory. I have some excluded directories like ~/.cache, and Steam's data directory. I use Baobab to find large directories that I don't want backed up.

I use the "exclude caches" option in the Borg "create archive" settings. That automatically excludes Rust target/ directories because they follow the Cache Directory Tagging Specification. Not all programming languages' tooling follows that spec so I also use directory name pattern excludes. For example I have an exclude pattern for .*/node_modules/.*

I use NixOS, and I keep my system config in a git repo so I don't need backups for anything outside my home directory.

[–] drwho@beehaw.org 2 points 1 day ago

All of my servers make local dumps of their databases and config files to directories owned by unprivileged users. This includes file paths, permissions, and ownerships (so I know how to put them back).

My primary research server at home uses rsync to pull copies of those local backups from my servers.

My primary research server uses Restic to make a daily incremental backup to Backblaze's B2 service.

[–] spacemanspiffy@lemmy.world 1 points 1 day ago

Dotfiles are handled by GNU Stow and git. I have this on all my devices.

Projects like in git.

Media is periodically rsynced from my server to an external drive.

Been meaning to put all my docker-composes into git as well...

I don't back up too much else.

[–] GustavoM@lemmy.world 2 points 1 day ago

.dotfiles on github

Big/critical files on an external HD

simple as

[–] shadowtofu@discuss.tchncs.de 2 points 1 day ago

I use syncthing to sync almost everything across my computer, laptop (occasional usage), server (RAID1), old laptop (powered up once every month or so), and a few other devices (that only get a small subset of my data, though). On the computer, laptop, and server, I have btrfs snapshots (snapper). Overall, this works very well, I always have 4+ copies of my data in 2+ geographical locations.

[–] traches@sh.itjust.works 2 points 1 day ago* (last edited 1 day ago)

Software & Services:

Destinations:

  • Local raspberry pi with external hdd, running restic REST server
  • RAID 1 NAS at parents' house, connected via tailscale, also running restic REST

I've been meaning to set up a drive rotation for the local backup so I always have one offline in case of ransomware, but I haven't gotten to it.

Edit: For the backup set I back up pretty much everything. I'm not paying per gig, though.

[–] neo@hexbear.net 2 points 1 day ago (1 children)

Pika Backup for /home/ to an external drive. It's an automatic solution with a simple GUI that serves as a front end to Borg iirc. Lets you easily browse and mount old backups. Anything outside of my actual personal files can be recreated or restored trivially, so I don't care to back them up.

I also have a manual dump of /etc/ but i change it so infrequently that it doesn't really need looking after.

[–] bad_news@lemmy.billiam.net 1 points 1 day ago

This is what I've been using but I've never actually had to recover using it...

[–] qwerty@discuss.tchncs.de 1 points 1 day ago

Pendrive for the important stuff, paper for the really important stuff and brain for everything else.

[–] TomBombadil@hexbear.net 1 points 1 day ago

My backup is begging my computer to implode so I can experience the sweet relief of getting offline.

But also I use external discs and make copies of important files I can't recreate. Don't care too much about config as I am happy enough to distro hop and set things up anew.

Timeshift for configs to a locally attached drive. Home partition to cloud with rsync

[–] Veraxis@lemmy.world 1 points 1 day ago

For system files/configuration on my machines, timeshift set to run once a week.

For family photos and shared files, I built a pair of SFTP servers made from old HP thin-client PCs at two different geographic locations which automatically sync to each other once a day via cron job using vsftpd and lftp. Each one has both an NVMe and SATA SSD which run in a software RAID 1 configuration.

For any other files, a second local server also using vsftpd and two SSDs in USB enclosures. I manually back them up using rsync on an irregular basis.

[–] Wanderer@r.nf 0 points 1 day ago (1 children)

The glorious life of openSUSE, defaults to btrfs on install and everything is preconfigured with snapper out of the box. Easy life, nothing to worry about.

[–] breakcore@discuss.tchncs.de 3 points 1 day ago

Yea, that's snapshotting, what do you do about back ups?

load more comments
view more: ‹ prev next ›