LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Is some tiny (1GB) model available through pip install? (alien.top)

submitted 10 months ago by GermanK20@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

I was looking into Galactica the other day, and it's available as a "pip install", that on first use downloads the model (260GB download LOL) and sets up everything. I got a slight headache by looking for hours through different small models, what I need to download etc, has any other model, a really small one for my ssh-only server without CUDA, been packaged like that? Though I wouldn't mind a model that fits in my 8GB laptop RTX either.

top 2 comments

sorted by: hot top controversial new old

[–] mcmoose1900@alien.top 1 points 10 months ago

You can use hf hub, see: https://huggingface.co/docs/hub/models-downloading

[–] MrEloi@alien.top 1 points 10 months ago

I doubt 1GB will give you much in the way of conversational skills from the AI.

I suspect that 7B is about usable with 13B getting there.