What is chain-of-thought prompting?

Chain-of-thought prompting means asking an AI model to reason step by step and show its work before it gives you a final answer. Instead of jumping straight to a result, the model lays out the intermediate steps, the same way you'd work through a math problem on scratch paper. That one change has an outsized effect on hard questions. In a 2022 study from Google Research, chain-of-thought prompting raised the PaLM 540B model's accuracy on the GSM8K math benchmark from 17.9% with standard prompting to 56.9% (Wei et al., 2022). That's more than triple, from one tweak to the prompt.

This guide covers what chain-of-thought actually is, when it helps and when it's overkill, a copy-paste prompt you can use today, and why newer reasoning models may do all of this for you without being asked.

Key Takeaways

• Chain-of-thought prompting asks the model to reason step by step before answering, which sharply improves accuracy on math, logic, and multi-step problems.
• In 2022, it raised PaLM's GSM8K math score from 17.9% to 56.9%, and adding self-consistency pushed it to 74%.
• It's overkill for simple lookups and one-step questions, where it just adds clutter.
• Newer reasoning models do chain-of-thought internally, so you often no longer need to ask for it by hand.

What is chain-of-thought prompting, exactly?

Chain-of-thought prompting is a technique where you instruct the model to think through a problem in steps rather than answering in one shot. You're asking it to "show its work." The model writes out a short reasoning trace, then arrives at the answer, and that visible reasoning is what makes the answer more reliable.

Here's the intuition. When a model answers a hard question instantly, it's committing to a result before it has worked through the parts. When it reasons step by step, each step builds on the last, so it's less likely to skip a constraint or fumble the arithmetic. The steps act like guardrails.

The effect shows up clearly on benchmarks. On GSM8K, a set of grade-school math word problems, standard prompting got PaLM 540B to 17.9%. Adding chain-of-thought reasoning took it to 56.9% (Wei et al., 2022). You can stack a second technique on top called self-consistency, which samples several reasoning chains and takes the majority answer. With self-consistency added, the same Google Research work pushed GSM8K accuracy to 74% (Google Research, 2022). The lesson is that the reasoning steps, not just the final guess, carry most of the value.

Chain-of-thought is one of a handful of core techniques in prompt engineering. It pairs well with others, like few-shot prompting, where you show the model worked examples first.

When does chain-of-thought prompting actually help?

It helps most when a question has multiple steps and getting one step wrong wrecks the answer. Reach for it on these:

• Math and word problems. Anything with numbers, units, or a sequence of calculations.
• Logic puzzles. Questions where you have to track conditions or rule out options.
• Multi-step planning. "Plan a 3-day trip given this budget and these constraints."
• Reasoning over rules. Applying a policy, eligibility check, or set of instructions to a specific case.

The common thread is that the right answer depends on getting the intermediate steps right. When you let the model lay out those steps, it has a place to catch its own mistakes before committing.

It's overkill, though, for simple questions. If you're asking "What's the capital of Japan?" or "Convert this date to ISO format," there are no intermediate steps to show. The answer is a lookup or a one-step transformation. Asking for step-by-step reasoning there just pads the response with filler and slows you down. A good rule: if you could answer it in one step yourself, you don't need chain-of-thought.

How do I write a chain-of-thought prompt?

The simplest version is a single instruction you add to your question. Copy this:

Work through this step by step, then give the final answer:
[your question here]

That's it. The phrase "step by step" is the trigger that tells the model to externalize its reasoning instead of guessing. Here's what changes in practice.

Before (standard prompt):

A store sells pens at 3 for $2. I buy 12 pens and pay with a $20 bill.
How much change do I get?

A model answering in one shot might rush this and return a wrong number, because it collapses three calculations (cost per pen, total cost, change) into a single guess.

After (chain-of-thought prompt):

Work through this step by step, then give the final answer:
A store sells pens at 3 for $2. I buy 12 pens and pay with a $20 bill.
How much change do I get?

Now the model writes out: 12 pens is 4 groups of 3; 4 groups at $2 each is $8; $20 minus $8 is $12 change. Each step is visible, so an error is easy to spot, and the final number is far more likely to be right.

One useful refinement: ask for the final answer on its own line, separate from the reasoning. Something like "After your reasoning, write 'Final answer:' followed by just the result." That keeps your output clean and easy to scan, especially if you're going to copy the answer into something else. You get the accuracy benefit of the reasoning without having to wade through it every time.

If you want to go deeper on phrasing prompts that produce reliable output, see our guide to writing better AI prompts.

Do reasoning models change anything?

Yes, and this is the part that trips people up. A newer class of "reasoning" models does chain-of-thought internally before showing you an answer. You don't have to ask for the steps; the model runs them on its own, often hidden, and gives you the worked-out result.

The payoff is large. According to a 2024 overview, OpenAI's o1, a step-by-step reasoning model, solved 83% of problems on the 2024 AIME math exam, against 13% for the non-reasoning GPT-4o (OpenAI o1, via Wikipedia, 2024). That gap shows how much built-in reasoning matters on genuinely hard problems.

So what should you do? It depends on the model in front of you:

• On a reasoning model, you usually don't need to add "step by step." The model already does it. Adding the instruction rarely hurts, but it's often redundant.
• On a standard model, the explicit chain-of-thought prompt still earns its keep, especially on math and logic.

When you're not sure which kind you're using, the safe move is to keep the step-by-step instruction in your back pocket and add it whenever an answer comes back wrong on a multi-step question. Developers building these prompts into apps will find more depth in our prompt engineering for developers guide.

How do you keep chain-of-thought prompts handy?

The technique is only useful if you actually use it, and the friction is remembering to. The fix is to save the prompt once and reuse it. Keep a short reusable version, the "Work through this step by step, then give the final answer:" line, somewhere you can drop into any chat in a click.

That's the same habit that makes any prompting technique stick: store the parts that repeat, fill in the parts that change. A reasoning prompt you've saved is one you'll remember to reach for the next time a model gets a math problem wrong, instead of re-typing it or forgetting it exists.

Sources

• Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, arXiv (2022). https://arxiv.org/abs/2201.11903, retrieved 2026-06-16.
• Google Research. Language Models Perform Reasoning via Chain of Thought, self-consistency results (2022). https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/, retrieved 2026-06-16.
• Wikipedia. OpenAI o1 (2024). https://en.wikipedia.org/wiki/OpenAI_o1, retrieved 2026-06-16.

Hero image: Christina Morillo via Pexels.