No-Activity-4824

joined 11 months ago
sorted by: new top controversial old
Hardware for Meta Llama2 65b for a Web App? in c/localllama@poweruser.forum
[โ€“] No-Activity-4824@alien.top 1 points 11 months ago
  1. Does it work well with other consumer graphics cards?
  2. Is the 15-20 t/s output or input?
  3. Regarding the fine tuning, Meta is working on it anyway, so hopefully another release at the beginning of 2024 of the same platforms but finetuned.
1
Hardware for Meta Llama2 65b for a Web App? (alien.top)
 

I need a bit more info from people who installed Llama2 locally and using it to support web apps, or just local information.

  • What is the ideal hardware for the 65b version?
  • How many tokens can this hardware process per second, input, and output?
  • Regarding safety, since it is used for business, what is the change that this model will end up arguing with the customer ๐Ÿ˜Š ?
Are there any data cleaning focused LLMs? [also, rant] in c/localllama@poweruser.forum
[โ€“] No-Activity-4824@alien.top 1 points 11 months ago

You need an AI like GPT 4 not an LLM

China is retrofitting consumer RTX4090s with 2 slot blower for ML in c/localllama@poweruser.forum
[โ€“] No-Activity-4824@alien.top 1 points 11 months ago

The entire country of China? The 1 billion people ๐Ÿ˜?

Sam Altman out as CEO of OpenAI. Mira Murati is the new CEO. in c/localllama@poweruser.forum
[โ€“] No-Activity-4824@alien.top 1 points 11 months ago (2 children)

I guess Microsoft did not like the guy, so they replaced him, I guess.

I hate Executives, in all companies, it is always how to push the next guy under the bus.