cns me / blog Cloudy with a Chance of Freefall
← Index | | 7 min read

Explain the workaround before you replace it

AI Generated: The sponsor mentions something about this desk in a sentence. But there’s a lot more to be said.

In 2020, the National Audit Office characterised Semaphore, a system used by the Home Office to collect and analyse passenger data at the border, as being ‘16 years old’. The Digital Services at the Border programme to replace Semaphore began in 2014, with a planned completion date of March 2019. It ran late. A programme reset in July 2019 pushed the completion date back to March 2022 - and made a significant choice: rather than replacing Semaphore, it would stabilise and improve it (to use the NAO’s elegant phrase). The NAO also estimated that the additional cost of sustaining old systems was £145m. It seemed that a system originally intended as a stopgap for a much more ambitious plan had become the very foundation of that plan.

But if you look at Airwave, the radio network used by emergency services, you get a similar story - just on a larger scale. Airwave was due to be switched off in 2020, but in 2019 that date was pushed back to 2022, in 2021 it was pushed back further to 2026, and in 2023 the Home Office was expcting it to be pushed back even further. According to the National Audit Office, continuing to run Airwave through the 2030s could cost at least £250 million per year. The Competition and Markets Authority calculated that there was around £200m a year of over-charging associated with the network. It’s like being charged a fortune by a seaside car park, only the car park in question is a national radio network and you’re not allowed to leave. And, while waiting for its replacement, somebody was getting very well paid to maintain the stop-gap solution. (Remember this the next time you hear that nobody is to blame for a delay.)

The answer is usually that there isn’t enough money, and it’s largely true. The State of digital government review found that legacy accounted for 28% of central government systems in 2024, an increase from 26% the previous year. It also found that it is easier to raise capital for new projects than revenue to keep things going, that funds allocated to legacy remediation are often repurposed, and that 28% of the highest risk legacy systems had no funding for remediation. This all helps explain why we don’t start replacing legacy, but doesn’t explain why replacement projects that have been funded, resourced and initiated are still delivered late or fail to be delivered at all - and why legacy is so often the eventual victor.

What the sponsor thinks they know

In 2002, Rozenblit and Keil asked participants to rate their understanding of common objects such as a zip or flush toilet, on a scale from 1 to 7. Then they asked them to write down, in detail, how the object actually worked, and to rate their understanding again. Somewhere someone is sitting with a pen over a blank page, becoming gradually aware that they have been using a zip all their lives on trust alone. This time around, their confidence dropped - and independent raters agreed that it was more accurate. The participants had been over-confident because they could imagine the object, and they could give the name of its function: there is an uncanny resemblance between naming what something does, and knowing how it does it. It’s the illusion of explanatory depth, and David McRaney has a great introduction to it in How Minds Change.

Article content
AI Generated: You know what a zip is until you are asked to design one.

The important distinction for workarounds is where the illusion of understanding was the strongest: it was strongest for explanatory knowledge (how things work), weaker for smiple facts, and virtually nonexistent for procedures (how to make biscuits) and stories. Consider what people know about an ordinary workaround. The person who would sponsor the replacement of that workaround knows it as a sentence (the reconciliation spreadsheet that runs overnight and matches two systems). This is explanatory knowledge, the sort of thing we tend to overestimate. The person who operates the workaround knows it as a procedure (these columns, these exceptions, make that call when they don’t match). This is the sort of thing we tend to get right. The division of knowledge runs right down the middle of the findings of this study.

This division determines the nature of the business case for replacing the system: it is written by people who ‘own’ the sentence and it is framed with reference to that sentence. The need for extra steps, for exceptions, for a second data source, bceomes apparent later, during discovery or testing. By that time it is too late: the programme has been delayed; the old system has not been decommissioned; and stabilising it seems like a good idea. Perhaps it is. A study of hospital medication systems conducted by Koppel and colleagues found workarounds that were in plain sight, but hidden by faith in technology: they were quietly treated as best practice. There’s never a day when the bodge becomes official: there’s never a meeting where it is decided that it should be.

Sometimes it manifests in the open: in October 2020, Public Health England admitted that they had missed 15,841 positive Covid cases in their daily statistics. The reason was that the files containing the results had become too large for the system that uploaded them. No-one writing a sentence to decribe the process of uploading test results would say, ‘And make sure that the files aren’t too big, otherwise it won’t work.’ Because that’s not what the sentence is about. But there, in the bowels of the mechanism, where the people who approved that mechanism were least likely to look, was a boundary which wasn’t crossed until the numbers got too big.

Neither account is the specification

This doesn’t mean that the operator is the hero of the story. As I tried to show in The Instruction Manual, it is common for procedures to be expressed in such a way that they preserve the what but lose the why. The same is true when the procedure is held in someone’s head: they may be able to tell you all the steps, but not necessarily know why there is a third step, or what rule it was intended to satissfy. The sponsor may have some idea of why the workaround is important, but not know the steps. Neither story on its own is the specification: when placed alongside one another, they are still an imperfect lower bound on the specification (some of the work that the workaround performs is not captured in either story), but they are much better than a sentence.

Article content
AI Generated: If you combine them, you get the form (with a few holes).

Similarly, declaring something to be temporary does very little work. As I wrote in the spring when discussing pet Kubernetes clusters, in a sandbox environment, the temporary nature of resources was their virtue: they expired not because someone had remembered to make them expire, but because the platform made them expire. But this is not the case for most workarounds. The Technology Code of Practice encourages teams to manage the full lifecycle, but does not say anything about decommissioning. Parliament holds itself to a higher standard: the Small Business, Enterprise and Employment Act 2015 places a duty on many regulations to contain an explicit review mechanism. Temporary is not temporary unless you do something about it. ‘Temporary’ is something like a bloke saying he’ll just go to the pub for a quick pint. It’s a wish, not a commitment, and everyone at home understands that. The houses built under the Housing (Temporary Accommodation) Act 1944 may have been intended to be temporary, but some of those houses (in Catford) were apparently listed in 2009.

Article content
AI Generated: ‘Temporary’ is just a word people use until the washing line is erected.

The test

Here’s a task for this month. Find one workaround that you have in your estate, which is due to be decommissioned by a replacement. Get the person who would authorise the business case for that replacement to describe, in detail, what the workaround does in a typical example from last week. It’s a bit like listening to someone describe a lasagne they have only ever seen from the adjacent table: certain of its layers, glowing about its cheese and blissfully ignorant of its béchamel. Then spend time with the person who operates the workaround as they work through that example. Note everything that the description missed: steps, exceptions, data sources, phone calls to people. Don’t ask why, ask how: as Fernbach and colleagues found, asking people to describe how things would work if a policy was in place can moderate their attitudes, whereas asking them to give reasons for the policy has no such effect. You may find that you don’t decommission the workaround after all. The W3C’s design principles tell us that, when something is already pervasive in practice, we should consider whether to adopt it rather than prohibit it.

But that’s not really the measure of its scale. It is merely a lower bound: we don’t know exactly what all the magic of the workaround is. However, it gives a much better indication of how far behind the replacement will be than anything in the risk register of the business case. And it’s easy to check: try running it on your next replacement and find out whether the number comes true. In an earlier article, I asked what would really be different this time when an enterprise attempts transformation once more. One honest answer to that question might be: before someone finances the replacement of this workaround which has survived three strategies, we have explained how it works and recorded how much we didn’t understand.

(Views in this article are my own.)

🦩