The AI arms race

A sizeable part of my job has always been making sure that our servers are secure and up to date. That's not new. There have always been hackers trying to break into our systems, and there have always been new patches to install for the software that runs them.

What's changed is the pace.

Five years ago we updated our server software monthly, and that felt like a sensible, slightly diligent schedule. Now it's much more frequent than that. This year, one of our main software vendors has released a huge number of patches, including multiple high-importance security releases in a single week.

Nobody ships several emergency security patches in one week because things are going well.

What we're actually seeing

The most obvious sign is the traffic we block.

We don't host any WordPress websites, for exactly this reason, so we automatically block any traffic that comes looking for WordPress systems, plugins or tools. That means every one of those blocked requests is unambiguously a bot working through a list, rather than a person trying to visit a site. The number of those blocks has increased dramatically.

Most of it is mindless. Bots endlessly rattling the handles of doors we don't have.

Some of it isn't. A few weeks ago, one of our websites was hit by more than 40,000 different IP addresses over the course of an hour, in the middle of the night.

I don't think that's one organised entity with a plan for us specifically. It feels more random than that. We don't host any black-ops secret spy data, and I don't imagine we're high on the target list for state-sponsored hacking groups. But we treat all our client data as important and confidential, which means it doesn't much matter whether an attack is personal or not.

So is this AI's fault?

Partly, and it's worth being precise about how.

Botnets are not new. Automated vulnerability scanning is not new. AI didn't invent any of the attacks we're seeing. What it has done is collapse the time between a vulnerability existing and it being found, weaponised and used at scale. Bad actors can now probe software for new vulnerabilities with an AI tool and a few keystrokes.

We're not the only ones noticing. Google announced that it fixed 1,072 security bugs in the two Chrome releases that came out in June. For context, that's more than the 1,036 bugs it patched across the previous 23 versions, over two years. Google credits AI tools for the change. Chrome's director of engineering, Doug Turner, put it like this: "By applying models like Gemini, we are preemptively fixing vulnerabilities, outpacing our adversaries and making Chrome safer with every update." Microsoft did something similar in July, patching 570 flaws.

That is what an arms race looks like from the inside. Everyone is suddenly finding a great many more bugs than they used to, because everyone now has better tools for finding them. Including the people you'd rather didn't.

What we've changed

Two things.

First, we're looking for problems in our own systems far more aggressively than we used to. I'm a huge fan of AI. We use it to help write new software, to find and fix bugs in our code, to create images for our websites. So we've turned the same tools on ourselves and built what amounts to our own security audit tooling, to find issues in our software before anyone else does.

Second, we've made our release cycle much faster. We've already shipped five new versions of our content management system this year, with more planned before the end of the year. That's a significant step up from previous years, and every website we manage now runs the latest version of our system.

I want to be clear about what "faster" means here, because it would be easy to misread. We are not rushing security releases. Rushing is an excellent way to introduce more bugs into a system rather than fixing the ones already there. Releases are planned. Alongside them we've built tools that spot and fix bugs and errors before they ship, and new monitoring that continuously checks our websites are healthy and tells us the moment something isn't.

More releases, better instrumentation, shorter gap between a problem existing and a problem being fixed. That's the whole strategy.

Is AI good or bad, then?

I get asked this a lot, as the tech person in the room. The answer is both, in the same way it's both for every technology anyone has ever invented.

Yes, an attacker with an AI tool can go looking for holes in our software faster than ever. But we can build a world-class security audit tool with our AI pal, and find and fix those holes before anyone gets the chance to exploit them. IT infrastructure is fundamentally designed to favour the defender. And the end result of all this is that the quality of the software we build and manage genuinely improves.

So no, I don't think it's a bad thing.

Some days it does feel like an endless battle, though.

Next time: "Don't trust any emails you receive, even from us." We've seen a sharp rise in convincing phishing emails claiming to be from Iteracy, offering SEO reviews, asking for email passwords, or warning that your mailbox is full. Also, as it happens, a consequence of AI.