this post was submitted on 28 Nov 2023
1 points (100.0% liked)

Machine Learning

1 readers
1 users here now

Community Rules:

founded 1 year ago
MODERATORS
 

I have a good PC, but the problem is with my RAM. The model is LLM, which makes the memory requirement understandable, but I can't afford to get that much RAM right now so I'm looking for a roundabout. Any help?

you are viewing a single comment's thread
view the rest of the comments
[–] KingsmanVince@alien.top 1 points 11 months ago (4 children)
[–] theonewhoask11@alien.top 1 points 11 months ago (3 children)

That is exactly the reason, I'm literally looking at my options and from the comments, it definitely isn't looking good 😭

Let's say I want to work on Noon (https://huggingface.co/Naseej/noon-7b), how much would I actually need?

[–] The-Protomolecule@alien.top 1 points 11 months ago

Not trying to be rude, but you’re also like saying you want to participate in a car race and you don’t have a fast car kind of a prerequisite unfortunately.

You could look at things that offload your model to disk but they’re going to be slow as hell.

load more comments (2 replies)
load more comments (2 replies)