A context window is the working memory an AI uses to answer you. It's everything the model can "see" at once: your prompt, the back-and-forth so far, and any text you pasted. It's measured in tokens, which are roughly chunks of words. When a chat or a document runs longer than that window can hold, the model starts losing track of the earlier parts. That's why your AI seems to forget.
Key Takeaways
- A context window is the AI's working memory: your prompt, the conversation, and pasted text, all at once.
- It's measured in tokens, and when a chat or document exceeds it, the model drops the earliest details.
- Models don't read long context evenly; info buried in the middle gets used least reliably.
- You can fight this: put key facts first or last, paste only what matters, and keep a short context recap to reuse.
What is a context window, in plain English?
Picture a desk. The model can only work with the papers it has spread out in front of it right now. The context window is the size of that desk. Your prompt goes on it, the running conversation goes on it, and anything you paste goes on it too. When the desk fills up, older papers slide off the edge to make room for new ones. The model isn't choosing to ignore them. They're simply no longer in view.
"Tokens" are the unit that measures the desk. A token is a piece of text, often a word or part of a word, so a long email might be a few hundred tokens and a long report many thousands. Every model has a maximum window, and the exact size shifts with each release, so it isn't worth memorizing a number. What matters is the idea: there's a limit, and once you cross it, something has to go.
This is also why two people get different results from the same tool, a pattern we cover in why AI gives different answers. One person pastes a wall of text and buries the request. The other puts the ask up top and trims the rest. Same model, different desk.
Why does the AI forget what I said earlier?
Two things happen as a chat grows. First, you can run past the window entirely. Once the conversation is longer than the window, the oldest turns fall out, so a detail you gave at the start may simply be gone by message forty. The model answers from what's left, which can look like it changed its mind or lost the thread.
Second, and less obvious, the model doesn't use a long window evenly even when everything still fits. In a 2025 study across 18 models, performance grew increasingly unreliable as the input got longer, a pattern the researchers called "context rot" (Chroma, 2025). More text in the window isn't the same as more attention on the part you care about. The model can technically "see" your detail and still under-weight it.
So "forgetting" is really two failures wearing one coat: the detail dropped out of the window, or the detail is still there but the model isn't leaning on it. Both get worse as the conversation or document gets longer, which is why long chats tend to drift.
What is the "lost in the middle" effect?
Where you put information inside the window matters as much as whether it fits. In a 2023 study, models did best when the key information sat at the very beginning or the very end of the input, and performance dropped noticeably when that same information was buried in the middle of a long context (Liu et al., Stanford / TACL, 2023). The researchers named it "lost in the middle."
The practical read: think of the window as having strong edges and a soft center. If you paste a long document and the one fact you need is on page nine of twenty, the model is most likely to skim right past it. Move that fact to the top of your message or restate it at the bottom, and you've put it where the model actually looks.
This single habit, edges over middle, fixes a surprising number of "the AI ignored my instruction" moments. The instruction wasn't ignored. It was parked in the worst seat in the room.
Keep your best prompts one click away
Promptly saves the prompts and recaps you reuse, across every AI assistant.
How do I stop my AI from forgetting?
You can't change the window's size, but you can change what goes in it and where. Each of these is something you can do on your next message.
- Put the important part first or last. Lead with the instruction or the key fact, and if it's a long message, repeat the ask in one line at the end. You're placing it on the strong edges, not the soft middle.
- Paste only what matters. Don't dump a whole document when three paragraphs carry the answer. Less filler means the relevant text takes up more of the window's attention.
- Start a fresh chat for a new task. A long thread carries baggage that competes for space. When you switch topics, open a clean conversation so the model isn't half-remembering the last one.
- Ask the model to summarize before you continue. When a chat gets long, say "summarize what we've decided so far in five bullets." That compresses pages of history into a few lines you can keep going from.
- Keep a reusable context recap. Maintain one short block that states who you are, the goal, and the constraints, then paste it at the top of new chats. It re-seeds the desk with the essentials in seconds.
None of this requires technical skill. It's the same instinct as briefing a busy colleague: say the important thing first, keep it short, and don't make them reconstruct last week's meeting from memory. If you want the broader version of this skill, our guide to prompt engineering walks through the building blocks.
A copy-paste context recap you can reuse
A recap is just a tight summary of the stuff the model keeps needing. Save one, edit the brackets, and paste it at the start of a new chat or whenever a long one starts to wander:
Context recap (paste at the top of a new chat):
- Who I am: [your role, e.g. a marketing lead at a small B2B SaaS company]
- What we're doing: [the goal, e.g. drafting a launch email sequence]
- Key facts so far: [3-5 bullets the model must not forget]
- Constraints: [tone, length, things to avoid, e.g. no jargon, under 150 words]
- What I want next: [the single ask for this message]Because the recap lives at the top of the message, it sits on a strong edge of the window, exactly where the model reads best. And because it's reusable, you're not rebuilding context from scratch every time. This is part of why so many people keep a small library of go-to prompts and snippets: the work you did once keeps paying off. As of 2025, the share of U.S. employees using AI in their role at least a few times a year has nearly doubled, from 21% to 40% (Gallup, 2025), so recaps you save now get reused more often, not less.
Recaps also travel. If you move a project from one assistant to another, a saved recap re-establishes the context in one paste. That's handy when you switch between AI assistants or want to export AI conversations and pick up the thread somewhere else.
Putting it together
A context window is the desk the model works on, measured in tokens, and it has a hard edge. Cross it and old details fall off. Even before you cross it, the model reads the edges better than the middle and uses long input less reliably as it grows. So the fix isn't a bigger desk. It's a tidier one: lead with what matters, paste only what counts, start fresh for new tasks, and keep a recap you can drop in anytime. Do that and the AI stops "forgetting," because you stopped asking it to hold more than it can see.
Frequently asked questions
What exactly is a context window?
It's the AI's working memory: everything the model can consider at once, including your prompt, the running conversation, and any text you paste. It's measured in tokens, which are roughly chunks of words. Once your input exceeds the window, the oldest parts drop out and the model answers from what remains.
Why does my AI forget what I told it earlier?
Two reasons. The detail may have fallen out of the window because the chat got longer than the window can hold. Or it's still in the window but the model under-weights it, since models use long input less reliably as it grows. Both get worse the longer the conversation runs.
What is the 'lost in the middle' effect?
A 2023 Stanford study found models perform best when key information sits at the very start or end of the input and noticeably worse when it's buried in the middle of a long context. The practical takeaway: put your most important fact or instruction first or last, not somewhere in the center.
Should I start a new chat or keep one long thread?
Start a fresh chat when you switch tasks. A long thread carries old context that competes for space and can pull answers off course. For a genuinely new task, a clean conversation keeps the window focused. If you need continuity, paste a short recap instead of relying on the old thread.
How do I keep important context without retyping it?
Keep a short, reusable context recap: who you are, the goal, the key facts, the constraints, and your current ask. Paste it at the top of a new chat so it lands on a strong edge of the window. Saving it once means you re-seed context in seconds across any assistant.
Sources
- Liu et al. (Stanford / TACL). Lost in the Middle: How Language Models Use Long Contexts (2023). https://arxiv.org/abs/2307.03172, retrieved 2026-06-16.
- Chroma. Context Rot: How Increasing Input Tokens Impacts LLM Performance (2025). https://www.trychroma.com/research/context-rot, retrieved 2026-06-16.
- Gallup. AI Use at Work Has Nearly Doubled in Two Years (2025). https://www.gallup.com/workplace/691643/work-nearly-doubled-two-years.aspx, retrieved 2026-06-16.
- Hero image: Md Jawadur Rahman via Pexels.