Is a 28x Cheaper Model Good Enough for Code Review? What 50 Real PRs Show (2026)

The Experiment

A code review company called Entelligence AI did something the category rarely does: it published a comparison with its method attached. The setup was straightforward in the best sense.

Take 50 public pull requests — ten each from Cal.com, Sentry, Discourse, Keycloak, and Grafana, drawn from a dataset called AI-Code-Review-Evals. Run GPT-5.6 Luna ($0.20 per million input tokens, $1.20 output) and GPT-6 Astra ($10 and $50) over the same diffs with identical prompts. Then verify every finding with two independent judges and count only bugs both agreed were real.

The models were given the diff only, with no repository context. That detail matters later.

The Headline Numbers

Metric	GPT-5.6 Luna	GPT-6 Astra
Verified bugs found	69	92
Precision	74%	96%
Total cost, 50 PRs	$0.20	$5.66
Cost per review	$0.0041	$0.113
Cost per verified bug	$0.0030	$0.061
Mean time per review	23s	36s
Output tokens per review	2,104	688

The summary the authors gave is the cleanest framing: Luna found 75% as many verified bugs as Astra for 3.6% of the money. Per review, that is under half a cent against roughly eleven cents — a 28x difference.

Two secondary observations worth noting. The cheaper model was faster, at 23 seconds per review against 36. And it produced three times the output tokens, which is how a cheap model stays cheap: it talks more because it thinks less carefully per word.

The precision numbers are the counterweight. Luna's 74% means 24 of its 93 findings did not survive verification — roughly one false positive in four. Astra's 96% means four of 96 failed. At $0.0041 a review you can afford that noise. In a pipeline that pages a human, you cannot.

Where the Gap Actually Lives

The average hides the useful part. Broken down by bug type, the two models are much closer than the headline suggests — except in one place.

Bug type	Luna	Astra
Data and logic	39	47
Concurrency	10	13
Security	9	19

Security is where the flagship doubles the cheap model. The study's own examples both come from Keycloak, an identity and access management server: a federation recovery code that was never invalidated after use, allowing reuse, and a global view permission that silently overrode a deny rule set on an individual client.

Both bugs share a property that explains the gap. Neither looks wrong on any single line. You have to hold the whole permission model in your head and reason across it to see the flaw. That kind of cross-cutting inference is precisely where a budget model struggles.

The per-repository breakdown tells the same story from another angle:

• Sentry, Discourse, Grafana: Luna and Astra within two bugs of each other
• Cal.com: 21 versus 30 — business logic with complex form validation
• Keycloak: 6 versus 14, and only about half of Luna's Keycloak findings survived verification

If your codebase is ordinary application logic, the cheap model is close. If it is authentication, authorization, or security-critical, it is not.

The Result Worth Acting On

The most useful part of the study is the overlap analysis across 143 verified bugs in the ground truth:

• 44 found by both models
• 48 found only by Astra
• 25 found only by Luna — including 16 data-logic bugs and 4 concurrency bugs
• 26 missed by both

Running both models catches 117 of 143 bugs — 82% — for $5.86, which is about 20 cents more than running the flagship alone. The cheap model is not redundant: roughly one bug in five was visible only to it.

That finding reframes the question. The choice is not which model but what role each plays — and the roles are not "cheap does the work, expensive double-checks it," because if Astra simply reviews Luna's report it will never see the 48 bugs only it can find. That version of the pipeline quietly discards a third of the ground truth.

A Three-Layer Review Pipeline

Structured properly, the division of labour looks like this:

Layer 1 — Cheap model, full scan. Every PR goes through Luna. On three of the five repositories it came within two bugs of the flagship, at under 4% of the cost. Volume work belongs here.

Layer 2 — Flagship, high-risk paths. Authentication, permissions, payments, and anything touching secrets must be reviewed by the flagship model. This is where the 9-versus-19 security gap lives, and it is not a place to optimize cost.

Layer 3 — Judge, false-positive clearing. Merged reports create more findings to triage — Luna's false-positive rate alone is near a quarter. An independent model verifies the merged output and humans spot-check samples.

Two boundary conditions matter. First, this works for offline review, where latency is free because the pipeline does not block development. For agents waiting on review feedback mid-task, the wall-clock cost of running two models is real and you will need to cut somewhere. Second, Layer 3 cannot be skipped. The savings from running a cheap model disappear in human triage time if nobody automates the verification step.

What the Study Cannot Tell You

Entelligence disclosed its limitations, which is why the numbers are worth citing at all. Four of them bound how far the results transfer.

Astra judged a contest it competed in. It served as one of the two verification judges alongside GPT-5.6 Sol. The authors state the conflict openly and added the second judge to suppress bias — but the 96% precision figure stands on that condition.

No repository context. The models saw diffs, not the surrounding code. Context-dependent bugs — exactly the category that includes the Keycloak findings — behave differently when the model can see how a function is called elsewhere.

Training-data leakage cannot be excluded. All 50 PRs predate both models' training cutoffs, so no model could have memorized the post-fix version. A reader raised the concern; the authors answered honestly that since the repositories are public, a model that knows what the correct version looks like has an advantage that cannot be ruled out.

Variance is unmeasured for most runs. One run per PR, with ten re-run for repeatability: of 15 bugs, Astra surfaced 10 on every run while Luna surfaced 7. Part of any single-run score sits inside that variance — and the cheaper model is the shakier of the two.

The broader lesson generalizes past this study: a precision figure without a label protocol is not a number you can act on. Ask three questions of any AI review comparison — who labeled ground truth, did the vendor score its own output, and what was the judges' disagreement rate. A 9% disagreement rate means 9% of the hard calls went the scoring team's way, and those are disproportionately the security-critical ones.

Run It on Your Own Repository

The method transfers, and the cost is essentially the model bill. The study's own recipe:

1. Collect 30–50 historical PRs from your repository that needed follow-up fixes after merging. That is your ground-truth bug library, and it reflects your code, not someone else's.
2. Run both models with identical prompts, asking only about correctness, security, concurrency, resource handling, and error handling. Exclude style and naming suggestions — they inflate counts without adding value.
3. Do not let a competing model judge. Use a third model or human spot checks.
4. Break results down by repository area and bug type. A healthy average can hide a collapse on authentication code, which is the area you most need to see clearly.

Where This Lands

The community reaction split along predictable lines. One engineer argued that sub-cent review costs mean the tool "isn't doing much work." A second replied that a ten-cent difference per PR is meaningless next to the cost of worse review. A third made the sharpest point: if agents push code volume up sharply and much of that code is short-lived, a very cheap first-pass check becomes extremely valuable — because the code was never going to be reviewed carefully anyway.

That last argument contains the real insight. The right model for code review depends on the expected lifetime of the code. Long-lived, security-sensitive code justifies the flagship. Disposable code justifies the cheap model. Most repositories contain both, which is exactly why a routing strategy beats a model choice.

And the arithmetic is getting harder to argue with: a 28x price gap against a 25% capability gap is a scissors that will keep widening. Every stage of an AI pipeline now has a price tag worth recalculating.

Related Articles

• Best AI Coding Agents 2026
• GPT-6 Astra Review
• Cursor Projects Review
• OpenAI Agents API Guide