AI Phishing Attacks: Why Bad Grammar Stopped Being a Red Flag

Security awareness training has focused on teaching employees to look for typos, awkward phrasing, and messages that felt culturally off for the past decade. That training worked because localization took real effort, which made well-written, native-language attacks rare and usually narrow in scope.

Today, surveys have found that up to 82.6 percent of phishing emails use AI-generated content. The attacker's own writing ability stopped being a variable in the equation. And when linguistic quality no longer correlates with effort, it no longer works as a reliable filter.

Here, we’ll discuss what changed at each stage of a phishing campaign, where AI phishing lands outside the inbox, and what detection looks like when the language signal is gone.

What Makes an AI Phishing Attack Different

An AI phishing attack is a phishing campaign where generative AI produces the message content, the target research, the supporting infrastructure, or all three. We’re not looking for a completely new delivery method here. The distinguishing feature is the removal of the cost and skill barriers that used to limit how personalized and how widespread a campaign could be at the same time.

Four things changed in practice: 

1. Language stopped being a constraint. AI models generate fluent, context-aware messaging in dozens of languages, adapting tone and cultural references without friction. What previously required regional knowledge and preparation now takes a prompt.

1. Iteration replaced translation. Campaigns refine through testing rather than through localization. The output feels locally appropriate without being locally sourced, which is a meaningfully different production process than hiring a native speaker.

1. Personalization became free. Hyper-targeted messages drawn from public profiles and organizational context used to be reserved for high-value targets because research took time. That cost collapsed, so the personalized version is now the default rather than the exception.

1. Volume climbed alongside quality. Those two usually trade off against each other. AI decoupled them.

AI Phishing Statistics Security Teams Should Know

• 82.6 percent of phishing emails use AI-generated content (KnowBe4, Phishing Threat Trends Report, 2025)
• 57.9 percent of phishing attacks originate from compromised accounts (KnowBe4, 2025)
• 4.5x higher click rate on phishing emails written with AI assistance (Microsoft, 2025 Digital Defense Report)
• 95 percent reduction in attacker cost when using LLMs, producing more varied and more frequent campaigns (Heiding, Lermen, Kao, Schneier, and Vishwanath, 2024)
• 87 percent of security professionals say their organization has encountered an AI-driven cyberattack (SoSafe, Cybercrime Trends 2025)

How AI Changed Each Stage of a Phishing Campaign

Targeting

Public profile scraping and organizational chart inference used to be manual reconnaissance. AI handles it at scale, pulling role, reporting line, tenure, recent activity, and writing style from public sources, then shaping the message around what it finds. The result reads like someone who knows the company wrote it, because in a limited sense someone did.

Message Generation

Malicious messaging now mimics internal communication styles closely enough that recipients cannot rely on traditional linguistic cues. In 2025, FINRA warned that fraudsters were sending highly credible emails appearing to come from real employees, complete with authentic names and professional context, to harvest credentials. Nothing in the message reads as wrong, because nothing in it is wrong except the sender.

The same technique scales to spear phishing against named executives, where public speaking appearances, interviews, and social posts supply the tone to imitate.

Infrastructure

AI-generated lookalike websites and spoofed domains spin up quickly with realistic branding, plausible copy, and functional layouts. The credential harvesting page that used to give itself away through broken CSS and placeholder text now renders correctly.

Distribution

Phishing-as-a-service puts kits, hosting, and multilingual templates on subscription. An affiliate buys the campaign infrastructure the way a small business buys a marketing platform, which means the person sending the message and the person who built the capability are frequently not the same person. That separation matters for attribution, and it means volume tells you very little about how many operators are involved.

Beyond the Inbox: Where AI Phishing Lands

Email is where phishing gets measured, which is a reasonable proxy right up until it stops being one.

AI-generated content is being used to build SEO authority for fraudulent pages, which then surface in search results alongside legitimate sources. A customer searching for your support line, your login portal, or your refund policy can land on a page that ranks well, reads well, and belongs to a threat actor. All of this exists but no message was ever sent, so no email gateway saw anything.

The same pattern runs through fake job listings that harvest personal data during a fabricated application process, spoofed social profiles that open conversations before any link appears, and malicious ads placed against branded search terms. Platform verification doesn’t resolve this either. In a single quarter in Europe, 170 verified Facebook accounts ran more than 67,000 fraudulent investment ads, which is a useful reminder that a checkmark is a signal rather than proof.

The practical implication is that the phishing attack surface now includes search results, social posts, job boards, and ad placements alongside the inbox. Email security covers one of those. Domain and phishing protection covers the rest, which is why the two are increasingly bought together rather than treated as separate problems.

Deepfake Phishing and Executive Impersonation

Voice cloning and synthetic video are now sold alongside phishing kits as add-ons for impersonation campaigns. The offerings emphasize realism and scalability, and the technology just needs to hold up long enough to influence one decision.

In early 2024, engineering firm Arup confirmed that a Hong Kong employee transferred approximately $25 million after joining a video conference where fraudsters used deepfake technology to impersonate company executives. Multiple participants appeared on screen, spoke naturally, and interacted in real time. The instructions matched what looked like an internal process. So, nothing in the interaction read as fraudulent.

Two numbers frame how far this has traveled. According to a Gartner survey, 62 percent of organizations have experienced a deepfake attack in the past year. And among people targeted by AI voice cloning specifically, McAfee’s research finds 77 percent reported losing money. More than one in three Americans say they are not confident they could identify a deepfake scam.

Detection here is harder than it is for text. No tool catches every synthetic artifact in real time, and the ones that work best catch a share rather than all of it, which our breakdown of how to detect deepfakes covers alongside the signals that still hold up. That's why process has to carry the weight that technology can't. A verification step that runs on every high-risk request, regardless of how convincing the call felt, doesn't depend on anyone noticing a flaw in the video. Technical deepfake detection is worth deploying where it fits, particularly for recorded media and inbound content at scale. But treat it as one input to a decision rather than the decision itself.

How to Detect AI Phishing When Language No Longer Helps

Detection has to move away from reading the message and toward validating the infrastructure and the request.

Correlate rather than inspect. Link phishing kits, domains, and targeting patterns to known campaigns and actors. A message that looks clean in isolation looks different when the sending domain was registered eleven days ago by a registrant tied to three prior campaigns against your sector.

Stop treating authentication as proof. SPF, DKIM, and DMARC confirm that a sender controls a domain, not that the domain belongs to who it claims. ZeroFox Signals Research found 73 domains impersonating ZeroFox and four peer vendors, every one configured to pass those checks, with no websites attached because the operation was built purely to send authenticated mail.

Automate the high-confidence cases. Where the match against known infrastructure is strong, blocking and takedown should run without a human in the loop. ZeroFox averages 10 minutes to mitigate a phishing URL, with a 95%+ takedown acceptance rate across 80+ Global Disruption Network partners spanning ISPs, registrars, social platforms, and browsers. Reserve analyst time for the ambiguous cases, where context still decides the outcome.

Rebuild awareness training around process, not proofreading. Teaching people to spot typos trains a skill that no longer applies. Teaching them that any payment change, credential request, or urgent executive instruction requires verification through a second, independent channel trains a skill that survives fluent language and synthetic video. Make the verification path fast and socially acceptable to use, since a control that feels like an accusation doesn’t get used.

Monitor for lookalike infrastructure before the first message sends. Domains get registered before they get weaponized, and that gap is the cheapest place to intervene. ZeroFox research on adviser impersonation kits found one that went from registration to a live site in twelve hours, which is a narrow window but a real one if you are watching registration streams rather than waiting for the first report.

Watch your executives specifically. Impersonation scams were projected to surge 148 percent in 2025, and executives carry the most public material for a model to imitate.

ZeroFox Domain and Phishing Protection sources 6B+ domains and URLs annually and analyzes 100M+ daily, using anti-cloaking browsers and AI-powered detection to find phishing infrastructure before it is used. On the response side, ZeroFox completes 1M+ successful takedowns annually, including 528k+ impersonation takedowns.

The full breakdown of how AI changed the signals defenders rely on is in the ebook: The Collapse of Certainty, AI and the Dark Web. Download it here.