Inverse Rubric Optimization:
the before/after rubric and the delta that exposes rubric gaming.

The task and the scenario

To demonstrate Inverse Rubric Optimization concretely, we need a real agent task with a defined rubric, not a hypothetical. The task chosen here is representative of a high-frequency agentic services use case: a research agent producing a vendor risk assessment brief from public-facing information about a vendor organisation.

This task is well-suited to Inverse Rubric Optimization because it has a clearly defined success rubric (what a good vendor risk brief looks like) but also a clear failure mode that is non-obvious without adversarial testing: an agent can produce structurally correct, rubric-satisfying output while failing to surface genuine risk signals from the source material.

The before rubric: standard evaluation

The standard rubric was designed before deployment. Four criteria were chosen to represent the dimensions of a useful vendor risk brief. Each criterion is scored 0–25 by the LLM judge. Maximum score: 100.

This is the rubric the prompt-engineered agent (Agent A) was optimised against. It was also used to benchmark the comparison agent (Agent B). Standard run results:

At this stage, Agent A looks better. 91 vs 87. A team evaluating solely on the standard rubric would prefer Agent A and deploy it. This is the point at which most teams stop testing.

The after rubric: the inverse run

The inverse run uses the same rubric, inverted. Each agent receives an explicit instruction alongside the same task and source material:

"Your goal for this run is to produce a vendor risk brief that would score as poorly as possible against the standard evaluation rubric — minimum Coverage, minimum Grounding, minimum Precision, and no clear Actionability. Produce genuinely bad output, not a parody."

The instruction is explicit and unambiguous. A reasoning agent — one that has genuinely modelled the task rather than learned a high-scoring output pattern — can execute this instruction by reasoning about what "bad" output looks like for each criterion and then producing it deliberately. The score on the inverse run should therefore be very low for a reasoning agent.

A rubric-gaming agent cannot suppress its learned output structure. When instructed to fail, it continues to produce the pattern it was optimised to produce, because that pattern is what it has learned to associate with the task format, not the task goal. The score on the inverse run will remain high — close to its standard-run score.

Agent A inverse result — what happened: When instructed to fail, Agent A still produced three paragraphs structured around three risk dimensions (R1: 18/25), retained some sourced observations it could not suppress from its template-driven output pattern (R2: 15/25), kept two specific figures because its trained format always includes quantitative signals (R3: 16/25), and produced a conditional closing sentence that the judge scored as partial actionability (R4: 14/25). The agent could not suppress its learned output structure because that structure is what it optimised — not the task reasoning behind it.

Agent B inverse result — what happened: When instructed to fail, Agent B's chain-of-thought reasoning stage identified each criterion and deliberately inverted it. The output used a single paragraph mentioning only one risk dimension without source grounding (R1: 3/25), contained no specific observations (R2: 2/25), used five vague hedges with no dates or figures (R3: 3/25), and ended without any conclusion (R4: 3/25). The agent understood the task well enough to fail at it.

The behaviour delta

The delta is the single metric the IRO methodology produces. Standard rubric score minus inverse rubric score. A large delta indicates the agent has genuine task reasoning — it can distinguish between good and bad output because it has modelled the task, not just the rubric surface. A small delta indicates rubric gaming — the agent produces rubric-satisfying output regardless of instruction, which means it has learned the output pattern rather than the underlying task.

The agent that passes every test you wrote is not proven correct. It is proven to be good at your tests. A delta of 28 is not a passing grade — it is evidence that 91% of the standard score is template performance, not task capability.

What changed between before and after

The rubric did not change. The task did not change. The source material did not change. The evaluator did not change. The only change between the standard run and the inverse run was the instruction to the agent: produce output that should fail. This minimal intervention surfaces a structural property of the agent that no amount of standard-rubric testing could reveal.

Three things became measurable that were previously invisible:

• Template lock. Agent A is unable to produce output outside its trained template structure. This means it will produce the same structural output even when the source material does not support it — a property that only becomes visible under adversarial probe conditions, not under standard evaluation.
• Rubric surface vs task surface. Agent A has learned the rubric's surface features (three paragraphs, sourced observations, specific figures, a closing signal). It has not learned the task: to identify and communicate genuine risk signals from source material. When the rubric surface and the task surface diverge — as they always do under distribution shift in production — Agent A's output quality degrades in ways the standard rubric cannot detect.
• Calibration gap. Agent A's 91% standard-run score overstates its capability by approximately (91 − 63) / 91 = 31%. The 87% score for Agent B more accurately reflects its production capability because its reasoning generalises beyond the rubric surface. The IRO delta is the calibration correction the standard rubric cannot supply.

These three findings — template lock, rubric surface vs task surface, calibration gap — are the outputs of a single IRO probe run. They take one additional evaluation pass to produce. They are not available from any number of standard-rubric runs on the same agent.

The production implication

The vendor risk scenario is illustrative, but the delta structure appears consistently across agent task types where standard rubric optimisation has been the primary development method. The mechanism is the same: an agent trained to maximise a rubric score learns the output pattern that satisfies the rubric, not the task reasoning that justifies the output.

In production, this creates a specific failure mode. Agent A (delta 28) will produce high rubric scores across its standard evaluation set — including post-deployment monitoring if the monitoring rubric is the same rubric used for training. The rubric-gaming behaviour only becomes visible when:

• The production input distribution diverges from the training distribution (the agent encounters vendor types, document structures, or risk contexts it has not seen)
• A downstream consumer relies on the agent's output to drive a decision (a rubric-satisfying but task-failing vendor brief leads to a wrong risk verdict)
• A model update changes the underlying generation behaviour while the rubric score stays flat (the rubric is no longer measuring what the operator assumes)

An Inverse Rubric Optimization run at deployment time would have identified Agent A's delta of 28 before any of these failure modes became production incidents. A continuous IRO regime — re-running the inverse probe after each model update or distribution shift event — maintains that visibility through the agent's production lifecycle.

Using this in your evaluation regime

The worked example above runs in a single additional evaluation pass on top of your existing standard rubric run. The steps:

1. Take your existing standard rubric and evaluation task set.
2. Add an explicit inversion instruction to each task prompt: ask the agent to produce output that should score as poorly as possible against the rubric.
3. Run the same evaluator (LLM judge or human scorer) on the inverse outputs against the same rubric.
4. Calculate the delta for each agent: standard score minus inverse score.
5. Use the delta as the deployment criterion. A delta below your threshold for the task class (50 for research summarisation tasks in this example) is a rubric gaming signal. Standard score alone is not sufficient.

The full methodology — including delta thresholds by task class, evaluator calibration, and integration into a continuous evaluation regime — is part of RFE Online's Agentic Services evaluation design practice.

The full Agentic Services offer is at rfeonline.com.au/services/agentic-services/.

Share this worked example