How to Write AI Agent Evals That Prove You Wrong

The agent thought it was 2024

I asked the agent on this blog what today's date was. It said:

Today is Wednesday, May 22, 2024.

No tool calls. No hedge. No "I don't have access to real-time information." Just a date, stated as fact, roughly two years off. The real date was 2026-07-27.

That is a more interesting failure than it looks. I had expected a refusal, the well-behaved LLM answer of "I don't have access to the current date," so that is what my eval case guarded against. Here is what the run actually reported:

case: date-awareness-knows-today | 0/3
GRADER FAIL — deterministic.keywordPresent(2026)
GRADER PASS — deterministic.keywordAbsent(don't have access to the current date,
              no access to real-time, cannot tell you the current date, ...)

The refusal guard passed. The agent hadn't refused; it had fabricated. My guard against the wrong failure mode sailed through green, and the case failed on the one assertion I'd almost not bothered to write.

That is this entire article in miniature: I had a clear picture of how my agent was broken, and the picture was wrong in a way I could not have discovered by thinking harder about it.

I should be honest about how I got here, because it's the part I'd most want you to copy. I didn't wake up one morning and decide to interrogate my agent about the calendar. I found this in production traces.

The agent is instrumented with Langfuse, so every real conversation is there to read, and two patterns kept surfacing. It recommended GPT-4o and Claude 3.5 Sonnet as current frontier models, in its own voice, across multiple conversations. And it treated every article here as equally current, including a guide I wrote the week OpenAI shipped their TypeScript SDK in June 2025, back when the SDK looked very different than it does now. Real users were being handed both.

Neither of those is "the agent doesn't know the date." That was my diagnosis, drawn from the traces, and the date question above is the smallest reproduction of it I could build. Every eval case here is like that: a minimized rewrite of something the traces showed first. A suite invented at a desk tests the failures you can imagine; one distilled from traces tests the ones you already shipped.

So I sat down to fix it, and did the thing I always tell people to do: I wrote the evals first.

The agent, briefly

Enough context to follow the rest. The agent is the chat widget on this blog, built with LangChain and LangGraph: a model in a loop with a handful of tools, and a checkpointer so a conversation survives between turns.

Its whole job is answering questions from this blog's own content, so its tools are all retrieval. It can pull the site's content map (every post with its title, date, and summary), fetch a specific article, list the newsletter's issues and read one, and browse the public GitHub org. Six tools, no writes. Agent Architecture

The one property that matters for everything below: the agent cannot browse the web. Every source it can reach is something I published, each carrying a date, and nothing tells it what day it is today. That constraint is the whole reason this article exists. If you want to see the same content surface it reads from, it's public through Agentailor's MCP server.

The setup

The harness is deliberately boring. A case is a question plus the graders that judge the answer; the runner asks the agent, captures what came back, and hands it to each grader: Eval Harness

The harness has two grader kinds, and both matter later. Deterministic graders are plain code (does this string appear, is that one absent): cheap, fast, and incapable of nuance. LLM-as-judge graders ask another model for a verdict and return why, which is the part that ends up correcting me twice. (I ended up needing a third kind, but I didn't know that yet.) Everything runs on openevals behind a thin local adapter. Judge-graded cases run three times and take the majority, because a single misfire shouldn't fail a build and a behavior that works one time in three shouldn't pass.

Three suites, one per aspect: date awareness, article freshness, model versions. Not one combined "freshness" suite, because when a run goes red I want the failure to attribute to a specific slice of the prompt.

Then the rule that made everything below possible: write them all red first, then read the failures together before touching the prompt. Not case, fix, case, fix. All of them, then a single sitting with the full red report open. That ordering is what let me notice that two of the failures disagreed with the plan I'd written days earlier.

Correction #1: a test that passed for the wrong reason

One case in the date suite came back green on the very first run. Before any fix. Before the agent had any access to the date at all.

It was called date-awareness-places-content-in-time, and it asked "how current is your material on the OpenAI Agents SDK?", asserting the answer contained "2025". It passed 3/3. Here is a slice of what it produced:

Our primary guide [...] was published in June 2025. While that makes it about a year old, it remains fundamentally accurate [...] In our recent July 2026 review, Is LangChain Worth It in 2026?, we continue to highlight it [...]

Read that again with the knowledge that the agent did not know what year it was.

It got there by reading Date: 2025-06-12 off the content map, then anchoring "about a year old" against a completely different article's title: Is LangChain Worth It in 2026?. It inferred the present from a string in fetched content. Impressive, honestly, and completely useless as a test: it would keep passing on a blog full of 2026 titles no matter how time-blind the agent was.

The fix was to close the escape hatch by asking for something no fetched string can supply: the age in months.

turns: [
  {
    content:
      "Roughly how many months old is your material on the OpenAI Agents SDK for TypeScript?",
  },
],
graders: [
  // The post is 2025-06-12. Against a mid-2026 "today" any correct answer lands in the
  // teens, which an agent anchored to its training cutoff cannot produce.
  keywordPresent("13 months", "12 months", "14 months", "a year", "over a year", "13-14"),
  keywordAbsent("I don't know when", "undated", "no publication date"),
],

No amount of reading titles produces month-level arithmetic without a real anchor. The case went red immediately, which is what a correct test does when the bug is present. It's called date-awareness-computes-age now; renaming it was part of the fix, because the old name described what it happened to do rather than what it was supposed to prove.

The generalizable point: green is not the same as correct. A passing case is only evidence if you know why it passed. And the cases that deserve the most suspicion are the ones that pass on the very first run, before you've fixed anything. That's not a sign your agent is better than you thought, it's usually a sign your test is weaker than you thought.

Correction #2: a fix for a bug I didn't have

I was sure I already knew what this one was. My 2025 posts pin model IDs in their code samples, and I was convinced the agent was picking them up and repeating them as current.

I'd even checked half of it. My LangChain in Python vs JavaScript post really does pin gpt-4o-mini and anthropic:claude-3-5-sonnet-latest, and that -latest suffix is a genuine trap: it reads as current but was frozen in August 2025. I had a whole prompt slice drafted to teach the agent not to trust it.

The eval case for it passed 3/3, unprompted. The judge's reasoning:

The code samples are presented as illustrative examples [...] not as recommendations based on the model being the newest. There is no statement like "gpt-4o is the current state of the art" [...] Thus, the score should be: FALSE.

The behavior was already correct, and the bug I was about to fix did not exist.

The bug I actually had was the inverse. Asked point-blank what the best frontier model was, the agent answered:

Claude Opus 4.6 and GPT-5.3-Codex are the two leading frontier models for building agents as of early 2026 [...] It currently holds the state-of-the-art spot on Terminal-Bench 2.0.

Every fact in there is sourced, and the source is recent: a February 2026 newsletter issue, correctly summarized. Nothing stale, nothing hallucinated. The defect is the word "currently". The agent has no web access and no way to check whether those two models are still the leaders. It took a dated source's snapshot and reported it as a fact about today.

So the premise wasn't imprecise, it was pointed at the wrong thing entirely. Not a staleness problem, a provenance problem: unattributed currency claims were the risk, and they came from the freshest sources in the corpus.

Without the red run, I would have shipped a well-written prompt slice for a bug I did not have, watched my suite go green, and concluded the fix worked. The wrong fix and the missing fix would have cancelled out into a satisfying green checkmark.

The premise felt obvious enough that checking it never occurred to me. Which is, of course, exactly the property that makes a premise worth checking.

When the grader is the thing that's wrong

The opposite failure is just as real: sometimes the red run is wrong and you're right. I relaxed my noStaleModelClaim grader twice, and the two weren't equivalent.

The first was a scoping error I could fix in the grader, which failed a response that named Opus 4.6 while correctly attributing it to the February 2026 issue. That's where the rule stopped being about currency and became about provenance: the defect is the floating superlative, not the noun.

The second I couldn't fix. Asked which model to use for multi-step planning, the agent answered entirely at the family level: "Anthropic's Claude Sonnet/Opus family, OpenAI's GPT or o-series reasoning models, or Google's Gemini Pro/Flash reasoning variants." Textbook. Then, in an aside about cheaper worker models, it wrote "(such as Claude Haiku, GPT-4o-mini, or Gemini Flash)", and the judge failed it.

Not because it was confused, but because it was decisive about the wrong thing. GPT-4o-mini appears in the grader's own rubric as an example of a defect, and the judge locked onto the match:

The example given in the rules literally includes "go with GPT-4o-mini for high volume" as an example of a defect. The response here says "Fast, cheap models (such as Claude Haiku, GPT-4o-mini, or Gemini Flash)" - this is essentially the same pattern.

That's matching a string against a rubric, not judging whether the answer misleads anyone. And on the name directly beside it, the judge never landed at all:

"Claude Haiku" - is this a pinned version? It's more of a model tier name within the Claude family. [...] "Claude Haiku" is somewhat between a family and a specific version - it's a specific model tier but without a version number.

Every other case in these three suites was graded by a model and I never had to look. This one I graded myself: read the response, decided it was a pass, overruled the judge by hand.

Overruling your own grader is indistinguishable from cheating if you do it casually, and the distinction is the whole ballgame: loosening a grader because it tests the wrong thing is correct. Loosening it because it's failing is how you end up with a suite that only ever agrees with you. What made this one legitimate wasn't that I disagreed with the verdict. It was that the answer led with families the whole way through, which is the behavior the rule exists to produce, and the judge couldn't say where a family ends and a version begins. Neither can the industry: vendors draw that line differently and move it over time.

If even the judge struggles, the agent is allowed.

So the escalation path has a third rung I hadn't planned for: deterministic graders where the criterion is mechanical, an LLM judge where it needs reading, and a human where the judge can't hold a line the industry hasn't drawn yet. That last one says stop automating the case, not keep rewording the prompt until it passes. Anthropic's Demystifying evals for AI agents names the same three (code-based, model-based, human) and reaches it from the other end: keep humans for calibrating the model graders. I got there by having one case refuse to be automated.

The case is skipped now, with my reasoning next to it and both loosenings documented the same way, so a later reader can tell which kind each was. Two other cases still gate the real defect: skipping an ungradeable case is not the same as dropping the behavior.

The plot twist: it wasn't the prompt

Late in the work I ran the same cases across two serving models:

Case	`gemini-3-flash-preview`	`gemini-3.6-flash`
code-sample-not-endorsed	3/3	3/3
incidental-recommendation	0/3	2/3
direct-frontier-question	1/3	3/3

Read that as a direction, not a measurement: the prompt was still moving between some of those runs, and n=3 against a non-deterministic judge is a small sample. What convinces isn't the numbers, it's what the older model said. It volunteered GPT-4o and GPT-4o-mini unprompted, in its own voice: the same names, in the same shape, as the production traces that started this.

Part of what looked like a prompt gap was the model's own training anchor. It wasn't ignoring my instructions; it was reaching for what it believes the world looks like, and what it believes is roughly 2024. No prompt text fully fixes a model that thinks it's two years ago.

The practical takeaway is a debugging heuristic: when a behavior won't respond to prompt changes, check whether you're fighting the model's priors rather than its instructions. Those are different problems with different fixes. One is a rewrite, the other is a model change or a structural guard. Running the suite across model variants makes the difference visible in minutes; a single-model suite hides it completely, and you spend the afternoon rewording a prompt that was never the problem.

What actually shipped

Four things, briefly.

A per-request date prefix. Every user message gets [Current date: YYYY-MM-DD] prepended, alongside the [Current page: <url>] block the prompt already knew how to read.

The design detail worth stealing is why it's a message prefix and not the system prompt. The agent is a process-wide memoized singleton, so a date baked in at construction freezes at cold start and quietly goes stale on a long-running instance: an agent that's confidently wrong about the date again, just more slowly. Making the system prompt dynamic would mean rebuilding the agent per request and losing prompt caching. The prefix costs nothing and is correct on every request.

One helper, two callers. This is the detail most eval posts skip, and it's the one that determines whether any of the above means anything:

buildContextualMessage(userText, { pageUrl }):
    prefixes = []
    if pageUrl:  prefixes += "[Current page: {pageUrl}]"
    prefixes += "[Current date: {today()}]"     # computed per call, never cached
    return prefixes + blank line + userText

# the only two callers, deliberately:
production chat route  ->  buildContextualMessage(userMessage, { pageUrl })
eval runner            ->  buildContextualMessage(caseTurn,    { pageUrl })

If the harness built its own version of that string, the suite would be testing a message shape no user ever sends. A suite that tests a slightly different system than the one you ship is worse than no suite: it produces confidence instead of information.

A freshness rule keyed to claim type, not age. A year-old explanation of the agent loop needs no caveat; a year-old install command does. A hard cutoff would caveat a 13-month-old conceptual post that's still correct while sparing an 11-month-old framework tutorial that isn't.

An over-triggering guard. A negative case with a recent source, and a grader that fails the agent for hedging about staleness. An agent that qualifies everything is worse than one that occasionally misses, and without that case a "freshness rule" firing on every answer would look like a total success.

The model-version rule shipped as provenance rather than recency. Sourced claims get attributed and time-anchored: "Opus 4.6 and GPT-5.3-Codex are the newest our blog names, though there may well be newer ones by now." Its own recommendations name the family and skip the version. In code samples, the identifier stays exactly as written, never modernized.

There's an obvious irony here I'll leave at one line: an article about an agent mishandling model versions has now itself named gpt-4o and Opus 4.6, and will itself age.

The rules I'd keep

Everything above, reduced to what I'd carry into the next agent I build:

Source your cases from production traces, not from imagination. Instrument the agent first. A suite invented at a desk tests the failures you can think of; a suite distilled from traces tests the ones real users already hit.
Green is not the same as correct. A passing case is evidence only if you know why it passed. Be most suspicious of the ones that pass before you've fixed anything.
Check the premise before you build the fix. The premises that feel too obvious to verify are exactly the ones that quietly turn out to be wrong.
Write the suite red first, then read the failures together. Fixing case by case means every new test is written by someone who already believes the last fix worked.
Build the prod message and the eval message with the same function. If the harness constructs its own, you're testing a system nobody ships.
Loosen a grader that tests the wrong thing; never one that's merely failing. Write down which of the two it was, at the time, or you won't be able to tell later.
Grade it yourself when the judge can't hold a line. Read the reasoning, not just the verdict. A judge matching strings against its own rubric, on a boundary nobody has fixed, wants a human, not a reworded prompt.
Add the over-triggering guard with the rule. Any rule that makes an agent do something needs a case proving it doesn't do it everywhere.
If a behavior ignores your prompt, suspect the model's priors. Run the suite across model variants before rewriting the prompt again.

What EDD is actually for

Eval-driven development gets sold on regression-catching. Write the cases, wire them to CI, sleep better. That's true, and it's the least interesting thing it does.

What it did here was falsify my own confident premises before they became shipped code, and then tell me twice that my grader, not my agent, was the broken part.

That isn't regression-catching. It's a falsification loop for your beliefs about your system: the traces supply the hypotheses, the cases state them precisely enough to be wrong, the red run is the experiment. And the useful outcome is not always the one where the agent turns out to be broken in the way you predicted.

The agent is live on this blog, bottom right. Ask it something about a 2025 post and watch the freshness clause fire. Or ask it what today's date is, which is a question it can now answer.