esotericloop

joined 10 months ago
sorted by: new top controversial old
Questions on Attention Sinks and Their Usage in LLM Models in c/localllama@poweruser.forum
[–] esotericloop@alien.top 1 points 10 months ago

See, you're attending to the initial token across all layers and heads. :P