I Put Claude in Charge of GitHub Copilot During an Optimizely Upgrade

We recently performed an Optimizely V11 -> V12 migration and, as a consequence, a .NET Framework 4.7.2 -> .NET 8 migration.

The Optimizely upgrade documentation pointed us towards their migration tool. Great! Open the link to GitHub and... it's been deprecated in favour of GitHub Copilot.

That makes sense, given that Microsoft recommend GitHub Copilot for the .NET modernisation work. They trained it, it understands the tooling, and their previous modernisation assistant is now effectively the legacy option.

This seemed like a good opportunity to investigate how we could make future upgrades simpler and easier for us, with the eventual aim of reducing the amount of manual effort involved.

The only issue was that we use Claude rather than Copilot. Sure, we'd tried Copilot previously, but hadn't had much luck with it.

Then came an internal brainstorm: what if we used Claude to direct Copilot?

Think about it:

1. Claude planned the work.

2. Copilot made the changes.

3. Claude reviewed what Copilot had done and decided whether it was allowed to stay.

Reading that back, it sounds slightly ridiculous, but in practice it worked surprisingly well.

As I mentioned earlier, the upgrade itself was a fairly large one. We were moving from Optimizely CMS 11 on .NET Framework 4.7.2 to CMS 12 on .NET 8, so it wasn't really just a CMS upgrade. The framework, project structure, dependency injection setup, routing, authentication and a fair amount of the application behaviour were all changing at the same time.

The audit

Before changing anything, we spent quite a bit of time auditing the solution. There were no automated tests around some important parts of the application, and the audit also turned up several risks that had nothing to do with the migration itself, including APIs that needed reviewing and database contexts that still had automatic migrations enabled.

Based on the audit, our first step was to establish some tests to make sure the upgrade didn't impact any existing functionality. Remember, AI agents love tests!

The tests became particularly important later because once Copilot was making changes at speed, we needed something more useful than "the code looks right" to tell us whether a step had actually worked.

The next thing we needed to work out was what to do with the existing database. The current solution used a code-first approach, but that wasn't something we wanted to carry forward in quite the same way.

Instead, we decided to move towards Entity Framework using a database-first approach through scaffolding. Think EDMX models, but without managing the database tables through a Visual Studio GUI.

Once that was decided, the final part of the preparation was getting Claude to turn the audit into a migration plan and work out a sensible order of attack.

Claude planned, Copilot executed

The process ended up being fairly simple - Claude would decide what the next migration step should be and write the prompt for Copilot. That prompt was then wrapped in a PowerShell command and executed through the Copilot CLI.

Something like:

Once Copilot had finished, Claude reviewed the changes. If it wasn't happy, it reverted them and rewrote the prompt with more detail or added anything it felt Copilot had missed.

What surprised me was how much it started to resemble a fairly normal development workflow i.e. one developer decided what needed doing, another implemented it, then the first reviewed the result and asked for changes; but the unusual bit is that neither of them are human!

There was also one part of the process that made me laugh more than it probably should have.

Copilot would make a change, Claude would reject it and try again with a more detailed prompt. Copilot would still not quite get it right, so Claude would try once more.

After a while, I started noticing that Claude was asking me to approve more changes directly. After a few suspicious ones, I started wondering what was actually going on...

I declined the next change and looked a bit closer. Claude wasn't directing Copilot anymore, it had started making the changes itself!

When I asked Claude what had happened, the answer was essentially: "Copilot wasn't doing what needed to be done, so it was easier to do it myself."

That made me chuckle because it had exactly the feeling of:

Fine. I'll do it myself.

Whilst I found it funny, we quickly put an end to it and updated agent.md to make it clear that Claude shouldn’t make code changes itself. Those changes had to go through Copilot.

Anyone who has ever spent twenty minutes explaining a small change to somebody before eventually opening the file and fixing it themselves will probably recognise the feeling.

Keeping the steps small

One thing we tightened fairly early in the process was how much work Copilot was allowed to do at once.

The migration was split into broad stages such as preparation, conversion, remediation, validation and release, but each of those was then split into much smaller pieces. One step might convert a single project, another might update one group of packages, and another might replace an API that no longer existed.

This made the output much easier to review, but it also made Copilot noticeably more reliable.

It also made it much easier to spot when Copilot wandered outside the task. If the prompt mentioned two files and six changed, Claude could reject the result without having to decide whether those extra changes were somehow helpful.

There were still some things we deliberately kept outside that workflow. Triggering the migration in DXP was one of them, because it started time-limited processes and affected shared environments.

Claude and Copilot could prepare everything around it, but we still wanted a person to decide when that step actually happened.

That was probably one of the more useful things we learned from the process. The AI could move quickly through a lot of the mechanical work, but we still wanted clear boundaries around the bits that had a wider impact.

Keep your eyes peeled for part two! We get into into some of the more awkward migration problems we hit once everything was running on CMS 12 and .NET 8.