esotericloop

joined 1 year ago
sorted by: new top controversial old
Questions on Attention Sinks and Their Usage in LLM Models in c/localllama@poweruser.forum
[–] esotericloop@alien.top 1 points 1 year ago

See, you're attending to the initial token across all layers and heads. :P