this post was submitted on 23 Nov 2023
1 points (100.0% liked)

LocalLLaMA

3 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago
MODERATORS
communick@poweruser.forum
1
Yi-34B and Yi-34B-Chat are out (alien.top)
submitted 11 months ago by hackerllama@alien.top to c/localllama@poweruser.forum
29 comments fedilink hide all child comments
 

Yi is a series of LLMs trained from scratch at 01.AI. The models have the same architecture of Llama, making them compatible with all the llama-based ecosystems. Just in November, they released

  • Base 6B and 34B models
  • Models with extended context of up to 200k tokens
  • Today, the Chat models

With the release, they are also releasing 4-bit quantized by AWQ and 8-bit quantized by GPTQ

Things to consider:

  • Llama compatible format, so you can use across a bunch of tools
  • License is not commercial unfortunately, but you can request commercial use and they are quite responsive
  • 34B is an amazing model size for consumer GPUs
  • Yi-34B is at the top of the OS Leaderboard, making it a very strong base model for a chat one
you are viewing a single comment's thread
view the rest of the comments
[–] a_beautiful_rhind@alien.top 1 points 11 months ago (7 children)

I like the base yi and the yi tunes that were made. I predict the chat model will be aligned too much for me to use it.

I gave the demo a shot, and while it wasn't super oppressive, still think I'm gonna pass and use other tunes of yi to chat instead.

[–] Hatter_The_Mad@alien.top 1 points 11 months ago (5 children)

Can you give a example of such a model?

[–] a_beautiful_rhind@alien.top 1 points 11 months ago (3 children)

nous-capybara, tess

[–] reddithotel@alien.top 1 points 11 months ago (1 children)

I cannot load that one :(. Dolphin does work for me, but I cannot change the output writing style.

[–] a_beautiful_rhind@alien.top 1 points 11 months ago

Sucks, all the ones I d/l work so far but I'm using exl2.

Those are actually 2 different 34b chat models but there is a merge of them, nous-tess. They were the first that came to mind. If you search 34b there are others.

load more comments (1 replies)
load more comments (2 replies)
load more comments (3 replies)