this post was submitted on 25 Nov 2023
1 points (100.0% liked)

LocalLLaMA

1 readers
1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 10 months ago
MODERATORS
communick@poweruser.forum
1
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding (alien.top)
submitted 10 months ago by Thistleknot@alien.top to c/localllama@poweruser.forum
4 comments fedilink hide all child comments
 

https://lmsys.org/blog/2023-11-21-lookahead-decoding/

you are viewing a single comment's thread
view the rest of the comments
[–] lone_striker@alien.top 1 points 10 months ago

I'm not sure how this would be applicable in those other scenarios you've mentioned; anything is possible. There may be other uses for this novel decoding method. But being touted as being X percent faster than transformers in a useful way isn't one of them.