KV cache/working memory

In-context learning

The context window acts as working memory while model weights act as long-term hazy recollection, implying a memory hierarchy.

1 source videos