LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

Storing LLM models on external SSD: Is the SSD speed important? Samsung T7 w/ USB 3.2 (read: 1050MB/s) vs. Fantom w/ Thunderbolt 3/4 (read: 2800MB/s) (alien.top)

submitted 10 months ago by nderstand2grow@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

(title)

you are viewing a single comment's thread
view the rest of the comments

[–] Susp-icious_-31User@alien.top 1 points 10 months ago

I store all mine on slow drives because no matter where you load it, RAM or VRAM, it gets fully loaded and the original file is forgotten about. And it’s not like the read speed of huge files is terrible, even on a spinning disk. Even if you overload your RAM and swap to disk, you’ll still be using your designated pagefile/swap drive rather than your LLM files drive.