LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

LLM for organizing long transcriptions into outlines? Latest and greatest? (alien.top)

submitted 10 months ago by Brad12d3@alien.top to c/localllama@poweruser.forum

3 comments fedilink hide all child comments

I found a post from several months ago asking about this and this was recommended : lmsys/longchat-13b-16k · Hugging Face

but I wanted to check and see if there are any other recommendations? I am wanting an LLM I can run locally that can search long transcriptions of interviews, brainstorm sessions, etc. and organize it into outlines without leaving out important info.

I have an RTX 4090 24gb and 128 DDR5 ram.

you are viewing a single comment's thread
view the rest of the comments

[–] FullOf_Bad_Ideas@alien.top 1 points 10 months ago (1 children)

Check out yi-34B 200K fine-tunes. You can load up to about 43K tokens on rtx 4090 if you use quantized version, 4.0bpw exllama v2 i believe.

[–] Wooden-Potential2226@alien.top 1 points 10 months ago

Yi-34-200k is trained for summarization and does it really well