this post was submitted on 26 Aug 2024
30 points (85.7% liked)

Selfhosted

40183 readers
801 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

After upgrading my internet connection I immediatelly noticed that my HDD tops 40 MB/s and bottlnecking download speed in qbittorrent. Is it possible to use SSD drive as a catch drive for 12 TB HDD so it uses SSD speeds when downloading and moves files to HDD later on? If yes, does it make sense? Anyone using anything simmilar? Would 512 GB be enough or could I benefit from 2TB SSD?

HDD is just for jellyfin (movies/shows), not in raid, dont need backup for that drive, I can afford risking data if that matters at all

All suggestions are welcome, Thx in advance

EDIT: I obviously have upset some of you, wasn't my intention, I'm sorry about that. I love to tinker and learn new things, but I could live with much lower speeds tho... Please don't hate me if I couldn't understand your comment or not being clear with my question.

HDD being bottleneck at 40 MB/s was wrong assumption (found out in meantime). I'm still trying to figure out what was the reason for download to be that slow, but I'm interested in learning about the main question anyway. I just thought I'm experiencing the same issue like many people today, having faster internet than storage. Some of you provided solutions I will look into, but need time for that and also have to fix whatever else I'm having issue with.

Keep this community awesome because it is <3

you are viewing a single comment's thread
view the rest of the comments
[–] braindefragger@lemmy.world 9 points 2 months ago (22 children)

Yes. It’s part of the application and well documented. What did you try and not work?

[–] rambos@lemm.ee 0 points 2 months ago (2 children)

Are you also talking about incomplete directory in qbit? Doesnt make it faster afaik, but I might be wrong. I havent tried anything yet, wanted to check is it something usual or not worth at all. Got zero experience with using SSD as catch drive, it just made sense to me

[–] braindefragger@lemmy.world 7 points 2 months ago* (last edited 2 months ago) (2 children)

Yes, if the temporary directory where the files are being downloaded (incomplete folder) is on the SSD, then it will be faster, especially if you’ve identified a cheap HDD as your bottleneck.

Unless you are incorrect about the HDD being the bottleneck.

[–] rambos@lemm.ee 0 points 2 months ago (1 children)

Yeah it will be faster, but its extra step before the files get available on HDD.

Even if my HDD is super fast and healthy it would still be a bottleneck for 2Gbps fiber? Ill deffo play with HDD more to find max speeds, wasnt paying attention before because it felt normal to me

[–] braindefragger@lemmy.world 5 points 2 months ago* (last edited 2 months ago) (1 children)

Of course it’s an additional step. But it will download faster. Which was what you asked for, specifically in your post above.

If you write directly to your HDD it will take longer to download. If you write to your ( faster?) SSD the download will be faster but yeah, processing has another step of copying.

I’m sorry, but I have no idea what you’re asking.

Best of luck.

[–] rambos@lemm.ee -1 points 2 months ago

Yeah feels like that lol. Thx anyway, have a nice day dude

[–] acosmichippo@lemmy.world -4 points 2 months ago* (last edited 2 months ago) (1 children)

what OP wants is to download the file to a SSD, be able to use it on the SSD for a time, and then have the file moved to spinning disk later when they don't need to wait for it.

this is just adding an extra step to the process before the file can be available to use. you're just saving the copying to the HDD until the very end of the torrent.

[–] braindefragger@lemmy.world 5 points 2 months ago* (last edited 2 months ago) (1 children)

Yeah, of course it is. Because that’s what OP asked for. I don’t see ( use it for a bit first and then automatically copy it over ).

I see:

Is it possible to use SSD drive as a catch drive for 12 TB HDD so it uses SSD speeds when downloading and moves files to HDD later on?

I assumed OP wanted Faster Download Speeds > Time to Access File

You know what. I don’t care. This whole post is ridiculous.

[–] acosmichippo@lemmy.world -4 points 2 months ago (1 children)

what is the point of faster download if you just have to do another entire copy after that?

[–] braindefragger@lemmy.world 4 points 2 months ago (1 children)
[–] acosmichippo@lemmy.world -4 points 2 months ago (1 children)

or you could, you know, think about it for a second from their point of view. and they have already clarified this in other comments.

[–] braindefragger@lemmy.world 3 points 2 months ago* (last edited 2 months ago) (1 children)

Man you’re slow.

I don’t need to think about anything. I’ve built out millions upon millions of dollars worth of infrastructure over the past 20 years. I understand the options.

I answered the question asked. Others hinted at raid solutions. OP went out of his way to let us know how he’s a newb so good luck with that.

Long term solution is 100% setting up a raid.

[–] acosmichippo@lemmy.world -3 points 2 months ago (1 children)

wtf does raid have to do with anything here? yeah, sure, I'm the slow one.

[–] braindefragger@lemmy.world 3 points 2 months ago (1 children)

Wow. Good luck with life bro.

[–] acosmichippo@lemmy.world -2 points 2 months ago* (last edited 2 months ago) (1 children)

the dude asks about SSD cash for torrents and your multimillion-dollar answer is "raid". lol

as people have already pointed out multiple times, what OP wants is something like mergerfs or unraid which can handle files on SSD cash and then move to spinning disks later.

[–] zelifcam@lemmy.world 2 points 2 months ago* (last edited 2 months ago)

It seems that the commenter’s intention was clear to everyone except you. The commenter acknowledged the need for RAID software or a specific file system, mentioning that it had already been addressed. Understood the budget and OP being an newb.

Although their tone may have been blunt, they stayed focused on their original point.

But you just kept nagging. lol

Either way OP was helped and now you can sleep knowing you did your part. A true internet hero.

[–] DaGeek247@fedia.io 4 points 2 months ago (1 children)

Yeah, I use the incomplete folder location as a cache drive for my downloads as well. works quite nicely. It also keeps the incomplete ISOs out of jellyfin until they're actually ready to watch, so, bonus.

If it's not going faster for you there's probably something else that's broke.

[–] rambos@lemm.ee 2 points 2 months ago (1 children)

It will download faster to SSD, but then I have to wait the files to be moved to HDD before getting them imported in media server. Im not after big numbers in qbit, I just want to start watching faster if possible. Sorry Im probably not explaining well and Im not sure if Im asking for something that even make sense

[–] DaGeek247@fedia.io 2 points 2 months ago (2 children)

qbittorrent moves the completed files to the assigned literally as soon as it is done.

[–] acosmichippo@lemmy.world 1 points 2 months ago* (last edited 2 months ago) (2 children)

but if the disk is actually bottlenecking at 40MB/s it will still take time to copy from the SSD. That plus the initial download to SSD will just end up being more time than downloading to the spinning disk at 40MB/s in the first place.

[–] DaGeek247@fedia.io 2 points 2 months ago

That's not how hard drives work, and doesn't take into account that OP might want to download more than one thing at a time.

Hard drives are fastest when they are moving large single files. SSDs are way better than hard drives at lots of small random reads/writes.Setting qbittorrent up so that all the random writes inherent to downloading a torrent go to a small ssd, and then moving that file over to the big hard drive with a single long writer operation is how you make both devices perform to their best.

[–] theterrasque@infosec.pub 2 points 2 months ago

I doubt the disk will bottleneck at 40mb/s when doing sequential write. Torrent downloads are usually heavy random writes, which is the worst you can do to a HDD.

[–] rambos@lemm.ee 1 points 2 months ago

Im doing more research, but will defo test this

load more comments (19 replies)