What is an AI context window? Why assistants forget

You paste in a long document, talk it through for an hour, and then the assistant contradicts something you told it at the start. It feels like forgetting. Most of the time, it is the context window at work.

What a context window is

A context window is the amount of text a language model can take into account when it writes one reply. It is measured in tokens, which are small pieces of text, often a short word or part of a longer one. A rough rule of thumb for English is that a token is about three-quarters of a word, though the exact count depends on the model's tokenizer. For more on how text is counted, see what a token is.

Everything the model needs to see has to fit inside that window at the same time:

• your current message and every earlier message in the conversation
• the assistant's own earlier replies
• hidden instructions from the app, such as a system prompt
• documents, files, or search results the app attaches
• the reply the model is writing right now

The window is a fixed limit for each model. It is not a measure of how clever the model is, and it is not the same thing as long-term memory.

Why assistants seem to forget

A model has no built-in memory of your conversation. Each time you send a message, the app sends the relevant text to the model again and the model reads it from scratch. Continuity is something the app creates by resending history.

When that history grows past the window, the app has to leave something out. The common choices are:

• Drop the oldest messages. Early instructions silently disappear.
• Summarize older parts of the conversation. The gist stays and the detail goes.
• Retrieve only passages that look relevant to your latest question. This works well when the search is good and badly when it is not.

None of these is free. Whichever one an app uses, some detail from earlier is no longer in front of the model, and the model cannot tell you about text it cannot see. It may still answer with confidence.

A bigger window helps, but not for free

Larger windows let more of a conversation fit, and they come with trade-offs:

• Cost and speed. Processing a longer prompt generally takes more computation, so long contexts tend to be slower and more expensive to serve.
• Uneven use of the text. Research on long inputs, including the 2023 paper "Lost in the Middle," found that models often did best when the relevant information sat at the beginning or end of the input, and worse when it was buried in the middle.
• Noise. Adding everything you have can bury the one paragraph that matters under pages that do not.

So the goal is not simply the biggest number. It is the right information, in front of the model, at the moment it answers.

Practical ways to work with a context window

• Put what matters first, and restate it. Keep the task, constraints, and definitions near the top, and repeat them when a conversation gets long.
• Start a new conversation for a new task. Old, unrelated text takes up space and can pull answers off course.
• Carry a summary forward. At a natural stopping point, ask for a short summary of decisions and open questions, and paste it into the next chat.
• Split large documents. Ask about one section at a time, or ask the assistant to quote the passage it is relying on so you can check it.
• Watch for drift. If an answer contradicts something from earlier, assume the earlier text may no longer be in view and restate it.

Where NAMELESS fits

NAMELESS is built on our own architecture so that context has no ceiling. It does not reread the whole conversation to continue it, which is why a long conversation does not slow it down or run up its cost, and it is built to carry what you share forward across conversations. NAMELESS is still in training and is not available yet; slots open to the waitlist as capacity allows.

If you want to follow along, you can join the early-access waitlist. To see how memory and context differ, read AI memory vs. context.