AI has changed cybersecurity

For about twenty years, the advice on staying safe from online fraud barely changed. Look for the spelling mistake, notice the clumsy phrasing, be wary of the email that doesn’t sound like the person who supposedly sent it. Anyone working in an office in the early 2000s will remember the messages promising a share of a fortune from a prince in Nairobi, written in English so poor that nobody needed training to spot them. They were almost comical, and I’ve come to think that the joke, more than any awareness campaign, is what kept most people safe for so long.

That’s the part AI has changed. The threat is the same, a request designed to get someone to hand over money, access or information, but the raw material behind it has been transformed. AI can write a fluent, error-free email in someone else’s tone of voice in seconds, clone a voice from a few minutes of public audio, and build a login page that’s pixel perfect next to the real one. Every clue we spent two decades teaching people to look for, the typo, the odd phrasing, the slightly wrong logo, has quietly stopped being reliable.

It hasn’t made attackers cleverer, it has made fakes cheap

The headlines tend to focus on deepfakes, and the real story is wider than that. The Arup case is the one most often shared. A finance employee in Hong Kong joined what looked like a routine video call with the UK based CFO and several colleagues, all of them familiar faces and voices. According to Hong Kong police, he had been sceptical of the email that started it, but that scepticism evaporated on the call, and he went on to authorise fifteen transfers totalling around twenty-five million US dollars. Every other person on that call was an AI-generated deepfake built from publicly available footage. His first instinct was right. It was the evidence that changed his mind, and the evidence was fake.

Most businesses will never face anything that dramatic, but they face a smaller version of the same weakness every day, in an email that looks right, a voice on the phone that sounds right, or a document with the right letterhead. We built our systems of trust on the idea that appearance was reasonable evidence of authenticity, and appearance has become cheap to fake.

Your digital twin already exists

The second shift is personalisation. A fraudster used to have to choose between quantity and quality, sending thousands of generic emails or spending real time on a handful of targets. AI removes that trade-off, because it can research and personalise at roughly the speed it used to take to send something generic.

I use myself as the example in the book. Between my TEDx talk, several books, conference recordings and hundreds of LinkedIn posts, there’s more than enough public material to teach a model how I structure an argument, which phrases I reach for and how I’d respond if challenged. Nobody has to hack anything, because I put almost all of it there myself. Most of us have an accidental digital twin assembled from years of entirely reasonable public activity, and a message that correctly names your regional director and mentions last month’s product launch feels as though it comes from inside your world, even though knowing facts about someone isn’t the same as being them.

Cybersecurity has become a judgement problem

This is the shift that matters most for leaders. The old model, which I wrote about in 2018 as the Human Firewall, treated security as a training problem, teaching people what to look for. That works when the warning signs exist, but now they may never appear, so the question changes from “can you spot the fake?” to “what does your organisation rely on as proof, and would it still hold if someone had unlimited time and a decent AI subscription?” You can outsource training, but you can’t outsource judgement. It has to be built into how a team operates day to day, which makes it a leadership responsibility.

I call the upgrade HumanOS, and it isn’t software or an annual course. It’s a different default response to uncertainty, built from a handful of habits. Trust behaviour rather than appearance, because a pattern built over months of working with someone is far harder to fake than a voice or a logo. Create thinking space on the small number of decisions where being wrong is expensive, because urgency is the one thing an attacker can manufacture with almost no effort. And verify independently, through a channel that existed before the request did, because ringing the number at the bottom of a suspicious email isn’t verification, it’s trusting the same source twice. I call that phantom diligence, the motion of checking without the substance.

Technology still matters, and AI helps the defenders too

None of this is an argument against technology. The AI woven through modern security tools is extraordinary at watching millions of events a second and spotting patterns no person could hold in their head, and it learns from far more attacks than any single business would ever see. But tools detect, filter and monitor, while people provide context, judgement and accountability, and somebody has to own the decision.

There are new risks on the defensive side too. AI systems can be confidently wrong in ways older rule-based tools rarely were, and shadow AI, where staff paste confidential information into unsanctioned tools with entirely good intentions, quietly builds the same kind of aggregated footprint an attacker would love to find.

Culture is a security control

The question I ask leadership teams is what happens here when someone admits they’ve made a mistake. The pause before the answer tells me more about an organisation’s security than most technical audits. An incident reported within minutes is almost always containable, while the same one discovered days later rarely is, and the gap between the two has more to do with culture than with any control sitting between the click and the consequence. If someone queries a request that turns out to be genuine and gets a slightly exasperated response, they learn that querying is costly, and they’ll be a little less likely to speak up next time.

Where to start

You don’t need a programme to begin. Pick one moment this week, a payment request, an unusual email, a login that looked slightly off, and talk about it out loud with the people around you, as a genuine conversation about what you noticed and why. If you want somewhere to focus, start with trusting behaviour over appearance, and then look at your culture, because that decides whether anyone around you gets the chance to use the rest.

Human Firewall 2.0: The AI-Era Cybersecurity Upgrade for Business Leaders is structured as ten conversations, the sort I have with people over coffee after a keynote, and they are the conversations that you should be having with your teams right now. The argument running through it is that AI doesn’t reduce the importance of human judgement, it increases it, and cybersecurity is simply the sharpest, earliest example of a pattern that’s about to show up everywhere else in business.

Read more about the book or get a copy here

Frequently asked questions

How has AI changed cyber fraud tactics?

AI has not made attackers cleverer, but it has made convincingly fake content cheap and fast to produce. Attackers can now easily write error-free emails in a specific tone, clone voices from public audio, and build pixel-perfect login pages. Traditional red flags like typos and poor phrasing are no longer reliable indicators of fraud.

What is an accidental digital twin in cybersecurity?

An accidental digital twin is an online footprint created from years of reasonable public activity, such as social media posts, videos, and articles. Fraudsters use AI to analyse this material to mimic someone's communication style, phrasing, and work details. This allows attackers to craft highly targeted messages that feel authentic without needing to hack anything.

What is phantom diligence and why is it dangerous?

Phantom diligence occurs when someone performs the motion of checking a request without true verification, such as calling a phone number listed in a suspicious email. It is dangerous because it relies on the same untrusted source twice. Independent verification must always use a communication channel that existed before the suspicious request was received.

How does organisational culture impact cybersecurity?

Culture acts as a vital security control because it determines how quickly employees report potential mistakes or suspicious activity. Incidents reported within minutes are usually containable, whereas delayed reports are not. If employees fear negative reactions for questioning requests, they become less likely to speak up, increasing the overall risk to the business.

What is the HumanOS approach to modern security?

HumanOS is a leadership framework focusing on habits rather than annual training. It encourages trusting established behaviour over appearance, creating thinking space for high-cost decisions, and verifying suspicious requests independently. It recognises that while technology detects and monitors threats, human judgement and accountability remain essential for making critical security decisions.