LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

I am very impressed by Claude.AI. What kind of hardware and models do i need to replicate it? (alien.top)

submitted 1 year ago by herozorro@alien.top to c/localllama@poweruser.forum

2 comments fedilink hide all child comments

im running M1/16 gig. Id like to get the speed and understanding that claude ai provides. I can throw it some code and documentation and it writes back very good advice.

What kind of models and extra hardware do i need to replicate the experience locally? I am using mistral 7b right now

top 2 comments

sorted by: hot top controversial new old

[–] AutomataManifold@alien.top 1 points 1 year ago

Claude has two big things: very long context length and high understanding (or whatever we want to call it).

The context length is the hardest part at the moment, I think. Though understanding is hard to measure.

[–] BalorNG@alien.top 1 points 1 year ago

There is no way it has "undiluted" 100k context. https://news.ycombinator.com/item?id=36374936

But yea, it IS impressive.