this post was submitted on 04 Jun 2024
30 points (96.9% liked)

Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ

54627 readers
772 users here now

⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.

Rules • Full Version

1. Posts must be related to the discussion of digital piracy

2. Don't request invites, trade, sell, or self-promote

3. Don't request or link to specific pirated titles, including DMs

4. Don't submit low-quality posts, be entitled, or harass others



Loot, Pillage, & Plunder

📜 c/Piracy Wiki (Community Edition):


💰 Please help cover server costs.

Ko-Fi Liberapay
Ko-fi Liberapay

founded 1 year ago
MODERATORS
 

Hey Folks, I have a bit of a conundrum that I'm hoping the hive mind can assist with.

I am in the process of learning docker to prep for my migration to Linux, but I have some questions about my filesystem structure. Currently my media files of all types live on a single file-based iSCSI LUN hosted on a QNAP which I connect to from a Windows machine. In my research to see if this would be consistent with best practice, I came to the conclusion that I should create independent NFS shares that the docker containers would connect to individually, rather than serving the files to the containers through the host and it's iSCSI connection.

This leads to my problem.

I can't seem to find any way to directly copy data from the LUN to one of my newly created NFS shares. With the volume of data I'll need to copy I'm trying to avoid as much overhead as possible, and using my Windows machine to connect to the new NFS share, then transferring the files from the iSCSI share, would be ludicrously inefficient.

As I'm able to SSH into my NAS, my first thought was to try and mount the iSCSI file locally and rsync the contents directly to the NFS share. After finding the home of the iSCSI file in the NAS filesystem, I discovered that it is not stored as a single, mountable file, but broken up into 1TB chunks. This leaves me unable to mount it, even in part, as each of the files lack an identifiable filesystem. Further, this is my largest partition, and so I don't (currently) have the space to attempt to concatenate the files into a single file (assuming that would even work, no idea).

After giving up on this approach, I decided to try and log into it's own external iSCSI target (from the NAS), then mount the LUN as I would from an external client. I thought I might be in the clear, as the login was successful, and both iscsiadm and the NAS GUI showed the active session to itself. But no matter where I looked I could see no evidence of a newly available partition, only those that were there from before I connected to the iSCSI target.

At this point the next step seems to be shrinking the partition and trying to concatenate the iSCSI files as I mentioned earlier. I have the space to play with, but I'll need to convert the volume to thin-provisioned, then shrink the volume, which would likely take foreverrrrrrr. But really, even this option sucks, because I'd prefer to avoid jeopardizing my primary storage volume in changing the provisioning style.

So anyway, after banging my head on it for the last few hours, I decided to step away and do some "rubber ducky debugging" with you guys.

So here are my questions: Is migrating to NFS worth the effort? Would the file concatenation method even work? COULD the loopback iSCSI method work if I do something differently? Any other tricks, or maybe something in the QNAP App Marketplace?

Any assistance welcome, thanks for reading!

you are viewing a single comment's thread
view the rest of the comments
[–] ultratiem@lemmy.ca 10 points 5 months ago

You really painted yourself in a corner with the QNAP setup unfortunately. There’s no way to migrate over to NFS without headache. Moreover, if you want to go the Docker on Linux route, you’re headed for the same headache but different. You can’t really treat iSCSI/LUN as a run of the mill filesystem (as you’ve discovered).

The issue I see is you’re virtualizing everything. Docker was essentially built to negate the OS. iSCSI virtualizes the filesystem. Doesn’t quite matter what you end up running them on really, you’re a slave to each in a sense.

I’ve always favoured running on the rails with this type of thing to maximize compatibility and limit overhead and headache when it comes to potential migration and connectivity to your NAS. Ubuntu server on ZFS running SMB gives you the broadest compatibility and today probably the best performance even on Linux. Moving to NFS doesn’t seem worth at all imo, it’s just a lateral move with a costly bill at the end.