LocalLLaMA

1 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago

MODERATORS

communick@poweruser.forum

Break the Sequential Dependency of LLM Inference Using Lookahead Decoding (alien.top)

submitted 10 months ago by Thistleknot@alien.top to c/localllama@poweruser.forum

4 comments fedilink hide all child comments

https://lmsys.org/blog/2023-11-21-lookahead-decoding/

you are viewing a single comment's thread
view the rest of the comments

[–] CorporationFlayer@alien.top 1 points 10 months ago (1 children)

Maybe not for speed, but do you think this approach could be well suited for environments where you have complex tasks that require knowledge on my different multidisciplinary fronts?

Aka Complex system building task creates many different fast models with different initializations in different directions and then aggregates?

[–] lone_striker@alien.top 1 points 10 months ago

I'm not sure how this would be applicable in those other scenarios you've mentioned; anything is possible. There may be other uses for this novel decoding method. But being touted as being X percent faster than transformers in a useful way isn't one of them.