Machine Learning

1 readers

1 users here now

Community Rules:

Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.

founded 1 year ago

MODERATORS

communick@academy.garden

Adjusting Probability distribution Using Speculative Decoding [D] (alien.top)

submitted 11 months ago by 1azytux@alien.top to c/machinelearning@academy.garden

1 comments fedilink hide all child comments

Hi, as Speculative Decoding runs a small model and a large model at the same time with a sampler in between, but in this instance the sampler's job is to NOT skew the probability distributions while doing so. There's a fairly simple python implementation of this idea here. Is there a way we can adjust the probability distributions of either the small model or the large model for the task of generation?

top 1 comments

sorted by: hot top controversial new old

[–] CatalyzeX_code_bot@alien.top 1 points 11 months ago

No relevant code picked up just yet for "Fast Inference from Transformers via Speculative Decoding".

Request code from the authors or ask a question.

If you have code to share with the community, please add it here 😊🙏

To opt out from receiving code links, DM me.