LocalLLaMA

3 readers

1 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 1 year ago

MODERATORS

communick@poweruser.forum

Break the Sequential Dependency of LLM Inference Using Lookahead Decoding (alien.top)

submitted 11 months ago by Thistleknot@alien.top to c/localllama@poweruser.forum

4 comments fedilink hide all child comments

https://lmsys.org/blog/2023-11-21-lookahead-decoding/

you are viewing a single comment's thread
view the rest of the comments

[–] lone_striker@alien.top 1 points 11 months ago

I'm not sure how this would be applicable in those other scenarios you've mentioned; anything is possible. There may be other uses for this novel decoding method. But being touted as being X percent faster than transformers in a useful way isn't one of them.

permalink
fedilink
source
parent