GitHub Copilot modernize-dotnet vs. Claude Code on a real .NET 8 → 10 migration

Most “tool A vs tool B” posts have a quiet flaw: they compare tools running different models, then report a model difference as if it were a tool difference. So before we ran anything, we removed that variable. Every run in this benchmark used the same model — Claude Sonnet 5, high thinking. What varied was the harness and the process around it: GitHub Copilot’s modernize-dotnet upgrade agent, plain Claude Code, and Claude Code under our twelve-layer guardrail system.

The whole thing is public and reproducible: Code-Majesty-Tech/dotnet-migration-benchmark. One baseline commit, three branches, clone it and check our numbers. Here’s what happened — including the part where the result was closer than we expected and the bare agent was the fastest.

Why the model has to be held constant

Copilot is not a model. It’s a harness that runs your choice of model — GPT-class, o-series, Gemini, or Claude. Claude Code is Anthropic’s harness running Claude models. So “Copilot vs Claude Code” as usually posed is a category error: unless you pin the model, you’re benchmarking whichever model each happened to use.

In 2026 you can pin it. Copilot’s model picker exposes Claude Sonnet, and its modernize agent honored the choice, so all three runs ran the identical model at the identical thinking level. That makes this a clean test of the one thing we actually wanted to measure: does the tooling and the process around the model change the result? If the answer were “no,” the entire premise of guardrail systems — ours included — would be in trouble.

The subject

We didn’t migrate a client codebase (private) or a toy (proves nothing). We built a purpose-made baseline: a small but production-shaped .NET 8 SaaS backend — Organizations, Users, Subscriptions — as a layered solution (Domain / Application / Infrastructure / Api / Tests), FastEndpoints for the HTTP layer, EF Core 8 on SQLite, xUnit. Five projects, ~1,522 lines, one checked-in EF migration, 12 tests green, zero build warnings on .NET 8.

It uses ordinary .NET 8 patterns with no artificially planted breaking changes — the migration friction is whatever naturally falls out of an 8 → 10 jump. The whole point of building it fresh is that it’s public: you can clone the baseline commit and run the same three migrations yourself.

The target: every project to net10.0, packages to their .NET 10 releases, global.json bumped, build clean, all 12 tests still green, no behavior change. Each tool got the identical instruction and ran fully autonomous — no manual code edits, no hand-holding.

The natural friction

Two real obstacles emerged from the 8 → 10 jump, and they turned out to be the whole story:

1. A FastEndpoints breaking change. The library’s only .NET 10-targeting release (8.2.0, up from 5.31.0) removed the instance SendAsync(...) method in favor of a Send property — five compile errors across the five endpoints. Every tool had to detect this and rewrite the call sites to Send.OkAsync(...) / Send.ResponseAsync(...).

2. A transitively-introduced CVE. EF Core 10’s SQLite provider pulls in SQLitePCLRaw.lib.e_sqlite3 2.1.11 — which carries a published high-severity vulnerability, CVE-2025-6965. Nothing in the baseline flagged it; it appears because you upgraded. Whether a tool noticed and fixed it is the sharpest divider in the whole benchmark.

The results

All three reached a green .NET 10 build with all 12 tests passing, fully autonomously. That’s the first honest headline: on a migration this size, every one of these tools works. The differences are in the details of what got handed back.

	A — Copilot modernize-dotnet	B — plain Claude Code	C — Claude Code + guardrails
Model	Sonnet 5, high	Sonnet 5, high	Sonnet 5, high
Interventions	none	none	none
Wall time	6m 13s	5m 31s	6m 31s + ~2m setup
Errors / Tests	0 / 12 ✅	0 / 12 ✅	0 / 12 ✅
FastEndpoints break	fixed	fixed	fixed (reflection-verified)
Warnings delivered	0	6	0
CVE-2025-6965	fixed	deferred	fixed
Hooks fired	—	—	investigation-gate ×1

Run A — Copilot modernize-dotnet

Copilot’s assess → plan → execute flow ran clean in 6 minutes 13 seconds. It bumped all five projects, updated global.json and the EF tool, moved every package to its .NET 10 release, and fixed the FastEndpoints break without prompting. Then it did the thing that impressed us: it proactively added an explicit SQLitePCLRaw.bundle_e_sqlite3 2.1.12 reference to clear the CVE — recognizing that the upgrade had introduced a vulnerability and closing it as part of the job. Zero warnings, twelve tests green. This is a genuinely strong tool, and Microsoft has clearly invested in it.

Run B — plain Claude Code

The bare agent — no CLAUDE.md, no hooks, no MCP, just Sonnet 5 and the migration prompt — was the fastest at 5m 31s and matched Copilot on core correctness: green build, 12 tests passing, FastEndpoints break handled. But it handed back six build warnings, and here’s the interesting part: it noticed the CVE, named the NU1903 advisory, and chose to defer it — “out of scope for a migration, let me know if you’d like it addressed separately.” That’s not a failure of capability. It’s a judgment call: it interpreted “migrate” narrowly and stopped at green. The result was a technically-correct build shipping a known high-severity vulnerability.

Run C — Claude Code + the twelve-layer system

Same model, same prompt, now under the guardrail setup — the CLAUDE.md, the hooks, and the Roslyn MCP server. It matched Copilot’s production-clean bar the bare agent missed: zero warnings, and it fixed CVE-2025-6965 (pinned SQLitePCLRaw.lib.e_sqlite3 to 2.1.12), with the stated reasoning “the baseline had 0 warnings, so I didn’t want to hand back a build with a new vulnerability.” The investigation-gate hook fired once, and rather than guess the new FastEndpoints signatures it loaded the installed assembly and verified them by reflection before editing.

Two honest marks against it. It was the slowest on wall clock (6m 31s), and it carried real setup friction: the MCP server wouldn’t start because our repo pinned an older SDK via global.json, and the launch command had to be changed from dotnet run to the built DLL before the guarded run could begin — about two minutes of debugging that the other two runs never paid. Guardrails are not free.

What actually separated them

Strip away the ties and the benchmark comes down to one axis: what each tool considered “done.” Copilot and the guarded run treated “done” as production-clean — no new warnings, no introduced vulnerability. The bare agent treated “done” as green — compiles, tests pass, stop. All three are defensible definitions; only two of them are the one you want shipping to production.

The mechanism behind the guarded run’s behavior is worth naming, because it’s the reproducible part. The PostToolUse build gate surfaces warnings during the run, not just at the end, and the CLAUDE.md’s definition of done forbids handing back a build dirtier than the baseline — so the same model that deferred the CVE when running bare fixed it when the process wouldn’t let it defer. The investigation discipline is why it verified the FastEndpoints API by reflection instead of guessing. That’s not a smarter model. It’s the same model inside a system that enforces a higher bar.

The caveats, stated plainly

We’re a .NET shop that sells a guardrail system, so treat our benchmark with the skepticism it deserves — and here’s what we’d point out ourselves before you do:

• n = 1. One migration, one run per tool. The differences are real but not proven systematic — a second run could shuffle the warnings or the CVE call. A rigorous study would run each tool ten times. This is a data point, not a law.
• The deltas are small. This was an easy migration and everyone passed. Nobody hallucinated APIs, nobody thrashed, nobody needed rescuing. On a gnarly .NET 6 → 10 migration across a 200-project legacy solution, the gaps would almost certainly widen — but we didn’t test that here, so we won’t claim it.
• The bare agent was fastest. If your definition of done is “green build,” plain Claude Code got there first and cheapest. The guardrails and Copilot both cost more time for a cleaner result. That’s a real tradeoff, not a free lunch.
• Copilot won on more than it lost. It matched the guarded run’s cleanliness with none of the setup friction, inside the IDE, on a Microsoft-blessed workflow. If you live in Visual Studio and pay for Copilot, it is an excellent answer for a migration this shape.

So what does it mean?

The model is increasingly commodity — the same Sonnet 5 drove all three runs, and on core migration ability the three were near-indistinguishable. What differed was the system wrapped around the model: whether it enforces a production-clean definition of done, whether it verifies before it writes, whether a deferred vulnerability is allowed to be deferred. That’s the argument the whole field guide makes, and this benchmark is a small, honest, single-data-point version of it: on the easy cases the tooling barely matters; the tooling earns its keep on the cases where “green” and “done” are different things.

There’s a second difference this benchmark can’t measure in a stopwatch: what you keep. Copilot is a subscription and an IDE workflow — rent it, and when the engagement ends, it leaves with the seat. The guardrails in Run C are scripts and configs that live in your repo — the investigation gate, the build gate, the Roslyn MCP starter are all public and yours to keep. That hand-off quality is the thesis of how we work, and it’s why we build systems you own rather than tools you rent.

Reproduce it yourself

We’d rather you check our numbers than trust them. The whole benchmark is public:

git clone https://github.com/Code-Majesty-Tech/dotnet-migration-benchmark.git
cd dotnet-migration-benchmark
./bench-baseline.sh          # confirms the .NET 8 baseline: 0 warnings, 12 tests green

master is the green .NET 8 starting point. The three bench/* branches each hold one completed migration, so you can diff exactly what each tool changed:

git diff master bench/copilot        # Copilot's migration
git diff master bench/claude-plain   # plain Claude Code's
git diff master bench/claude-guarded # the guarded run's

The differences we reported are right there in the diffs — the FastEndpoints call-site rewrites, the package bumps, and the one that matters most: whether the branch added a pin for SQLitePCLRaw to close CVE-2025-6965, or left it unaddressed. RESULTS.md on master records every number with the same bench-baseline.sh output you can regenerate. To run your own fourth column, branch from master, point your tool of choice at it, and compare. If your results differ from ours, open an issue — a benchmark nobody can contest is just marketing.

When to call us

If your migration looks like this one — a clean, well-tested solution making a one-generation jump — honestly, run Copilot or plain Claude Code and save your money; they’ll get you there. Where the conversation changes is when it isn’t clean: a .NET 6 or Framework 4.8 codebase with no test coverage, where “green build” hides ten deferred decisions and a fresh CVE, and where you need the migration to be evidenced, not just done. That’s a Sprint Zero conversation — we instrument your repo with the guardrail system, migrate under it, and leave the system behind so your team ships clean without us. Or start smaller: an $80 hour walking through your specific migration against this benchmark.

The numbers here are real, the repo is public, and the honest answer is that on an easy migration all three tools are good. The reason we built the twelve layers is everything that happens after “easy.”