The Migration Bugs That Actually Caught Us Out

In the first post, I talked about how we used Claude to direct GitHub Copilot through an Optimizely CMS 11 to CMS 12 migration.

And, this approach worked fairly well for the majority of the upgrade, but once we had the solution building and started using it properly, we began finding the bits that weren't quite as straightforward.

Most of the issues we encountered weren't actually bugs in CMS 12. They were largely down to assumptions the old application had relied on for years, which only became apparent when the move to .NET 8 and CMS 12 introduced subtle changes in functionality, typically relating to the underlying framework.

An empty field that wasn't actually empty

One of the first issues we hit involved `XhtmlString` - we had checks like this in a few places:

And, that looked fine, but it turned out that an empty `XhtmlString` isn't always actually empty - in some cases, the value contained:

So, from an editor's point of view, the field was completely empty, but `string.IsNullOrEmpty` quite rightly disagreed.

In some places that only meant we rendered a bit of empty markup, which wasn't particularly serious. The more awkward version was where the value had been written directly into JavaScript:

That line break then became part of the JavaScript string and caused a syntax error. Naturally, the browser pointed us towards the JavaScript, so that was where we started looking, even though the actual cause was a CMS property that appeared to contain nothing.

Once we'd worked that out, the fix was fairly simple. We used the `XhtmlString` Optimizely API to check whether the property was actually empty rather than treating it as a normal string, and avoided writing the value directly into JavaScript where we could.

The routing cleanup that changed our URLs

Another issue came from API routing.

In the .NET Framework version, the routes were named explicitly, so the URL structure was clear and there wasn't really any ambiguity around what an endpoint should be called.

As part of the move to .NET 8, we removed a lot of that explicit routing and replaced it with attributes instead, which seemed like a fairly sensible cleanup at the time, as it aligns with best practice.

We had controllers named things like:

and moved towards routes like:

The problem with this change, as we all know, is that `[controller]` is based on the controller class name, with `Controller` removed from the end.

So:

becomes:

rather than the existing:

The application built fine, the controllers were registered and the routes existed, but the frontend was still calling the original URLs.

Nothing was technically wrong with the routing change we did - .NET was doing exactly what we'd asked it to do, but by replacing the old explicit routes with attributes we'd accidentally changed the public URLs of the API.

Once we realised what was happening, we reverted that part of the cleanup and made the routes explicit again:

It's more verbose, but in this case that wasn't really a bad thing. Those URLs already formed part of the application's contract, so changing them just because the framework gave us a neater routing option wasn't worth it.

It was also a good reminder that migration work is probably not the best time to quietly tidy up behaviour that already works!

The request model that became null

This one took a bit longer to work out - a lot of caffeine was consumed during this!

We had an API action taking a model using `[FromBody]`, but after the migration the model was arriving as `null`.

Nothing relating to the model had changed. The code and the attributes were still the same. The frontend JSON object that was being posted still looked valid - nothing in the response immediately pointed towards JSON deserialization, which made it look like the request body had simply disappeared somewhere during model binding.

It turned out that we'd stumbled upon a JSON serializer change in .NET framework -> .NET 8.

In the old solution, we were using Newtonsoft.Json as our formatter, but this changed to be System.Text.Json, and Newtonsoft.Json had been forgiving enough to coerce the value for us, whereas System.Text.Json wasn't.

The frontend was sending something along the lines of:

while the C# model expected a string.

It threw during model binding, and because the controller wasn't using `[ApiController]`, the failure ended up in `ModelState` while the action still ran with a null model.

I eventually confirmed it by manually firing the same request from the browser, but sending a string instead:

As soon as I did that, the model bound correctly and the issue made a lot more sense.

The proper fix was on the frontend because the request itself was wrong, but we also added a server-side guard. Returning a clear `400` is much easier to diagnose than allowing the action to carry on with a null model.

The interesting bit here was that .NET 8 hadn't really broken anything. The old solution had just been getting away with sending the wrong type, and Newtonsoft had been allowing it to!

The email that never arrived

We also hit an issue with Optimizely Forms where the form submission itself worked, the user reached the confirmation page, but the notification email never arrived. There wasn't anything particularly useful in the normal application logs either, which made it look like the email had simply vanished.

I ended up looking at first-chance exceptions and checking which assemblies were actually being loaded at runtime, which eventually pointed towards a MimeKit and MailKit version mismatch.

From an MSBuild point of view, everything was fine. The dependency range was valid, restore completed and the project built successfully. The problem only appeared when the email functionality was actually used and the CMS code expected a different assembly version at runtime, and that resulted in a `FileNotFoundException`.

The reason it was so difficult to spot was that the email work was happening in background code, so the exception wasn't appearing anywhere near the original form request.

Once we knew what was happening, the fix itself wasn't particularly exciting. It was just a case of pinning the MailKit package explicitly.

This was one of those cases where finding the problem took far longer than fixing it! Something that we've no doubt all encountered!

Login was actually working

Another issue that sent us in the wrong direction for a while was CMS login.

We had some users who could log in without any issue, while others from the same Azure AD tenant would authenticate successfully and then find themselves straight back at the login page.

Naturally, my first thought was that something had gone wrong with the OIDC setup during the migration, so I started looking at redirect URLs, tenant configuration and the usual authentication-related bits. What frustrated me the most, was that some users could log in perfectly fine - if the OIDC configuration itself was broken, I'd have expected it to affect everyone.

Then I had a light bulb moment! In CMS 11, we'd configured the allowed roles through `web.config`, including the custom Azure AD roles used by the application. When we moved to CMS 12, that configuration moved into an authorization policy and those existing custom roles hadn't come across with it. The login itself was succeeding, but the user was then failing authorization immediately afterwards and being sent back out again, which made it look like the authentication had failed!

The fix was to add those roles into the CMS shell policy explicitly:

But, that wasn't the end of it; the CMS editing UI also has its own virtual role mappings, so we needed to make sure the same custom roles were configured there as well.

After updating both places, everything behaved as expected, and the other users were able to log in.

That's probably a good place to leave this one. Part three will cover some of the extra controls we had to put around Claude and Copilot as the migration went on, including stopping Copilot from rereading completed steps and trying to rerun work we'd already finished.