Tips / Tip

Same prompt, different answer: what to check before blaming the AI

By Michael Shreeves · 17 July 2026 · AI & Copilot · 2 min read

"It worked last week and now it doesn't" is the most common prompt complaint going, and it's rarely a mystery. Work down this list in order. The first three are the quickest to check.

  1. The input changed. A longer document, a messier export, a new column. Same prompt, different thing pasted. Compare the input before you touch the prompt.
  2. The chat isn't empty. In a long conversation the model's also reading everything before it: your corrections, your tone, an instruction from forty messages ago. Plenty of tools carry memory or custom instructions between chats too. Open a new chat with just the prompt and compare.
  3. The model or the mode changed. Tools update their models and defaults without much fanfare. If results shifted overnight on identical input, it's a likely one.
  4. The prompt left a gap. It only worked because the model happened to guess right. Ask it to list its assumptions and put the important ones into the prompt.
  5. It's random. These tools don't give identical output twice. If you need the same shape every time, spell the shape out: exact headings, a fixed table, an example to copy.

What holds up in the end is a saved template with an example of the output in it. It's the strongest steer you've got, and it's what stops a prompt drifting.

Keep one input you know the answer to next to the template, along with its good output. When something goes odd, run that one first, twice. Right both times? Your new input is the problem. Different each time? That's randomness, so give it a clearer example. Consistently different from before? The tool's probably changed.

← All tips