this post was submitted on 17 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
Introducing Tess: Tess-M with 200K Context Length (alien.top)
submitted 10 months ago by migtissera@alien.top to c/localllama@poweruser.forum
29 comments fedilink hide all child comments
 

I'm blown away. See for yourself.

https://migel.substack.com/p/a-conversation-with-tess

Tess, welcome to the world!

Model is Open Source with 200K context length.

Available at: https://huggingface.co/migtissera/Tess-M-v1.0

​

https://preview.redd.it/qehogf32mx0c1.jpg?width=800&format=pjpg&auto=webp&s=333356e7c95e1a1d8fc32dca1d7829720ad2cc84

you are viewing a single comment's thread
view the rest of the comments
[–] ReMeDyIII@alien.top 1 points 10 months ago (1 children)

According to TheBloke the Sequence Length is 8192 ctx, so I'm assuming 8192 ctx is its default and it can extend up to 200k ctx via alpha_scale?

[–] migtissera@alien.top 1 points 10 months ago

No, the base model itself is 200K: https://huggingface.co/01-ai/Yi-34B-200K