LocalLLaMA

4 readers

4 users here now

Community to discuss about Llama, the family of large language models created by Meta AI.

founded 2 years ago

MODERATORS

communick@poweruser.forum

GPT-4's 128K context window tested (alien.top)

submitted 2 years ago by Ok_Relationship_9879@alien.top to c/localllama@poweruser.forum

6 comments fedilink hide all child comments

This fella tested the new 128K context window and had some interesting findings.

* GPT-4’s recall performance started to degrade above 73K tokens

* Low recall performance was correlated when the fact to be recalled was placed between at 7%-50% document depth

* If the fact was at the beginning of the document, it was recalled regardless of context length

Any thoughts on what OpenAI is doing to its context window behind the scenes? Which process or processes they're using to expand context window, for example.

He also says in the comments that at 64K and lower, retrieval was 100%. That's pretty impressive.

https://x.com/GregKamradt/status/1722386725635580292?s=20

you are viewing a single comment's thread
view the rest of the comments

[–] ArtifartX@alien.top 1 points 2 years ago

If the fact was at the beginning of the document, it was recalled regardless of context length

Lol at OpenAI adding a cheap trick like this, since they know the first thing people will test at high context lengths is recall from the beginning.