Generative AI fraud

Fraud that uses AI tools to create convincing fake faces, voices, documents, or messages.

Instead of forging an ID by hand or writing a scam message from scratch, a fraudster asks software to produce a realistic version in seconds.

Fraud in which generative AI models produce synthetic content, such as images, video, audio, documents, or text, to deceive a person or bypass a verification check. Generative AI refers to models that create new content from examples they have learned from.

Generative AI fraud shows up in several forms:

• Deepfake video and images, which imitate a real person's face or invent a new one.
• Voice cloning, which recreates a person's voice from a short sample.
• Digital forgeries, which are AI-generated or edited images of ID documents and bills.
• AI-written scams, which produce fluent, personalized phishing and romance messages at scale.
• Synthetic identities, which combine invented details with generated photos.
• Agentic fraud, in which AI agents run parts of a scam on their own.

A typical attack chain looks like this:

1. The attacker collects public photos, audio, or details about a target or a persona.
2. Generative tools create the fake content.
3. The attacker delivers it, for example by presenting it to a camera, injecting it into an app, or sending it in a message.
4. The fraud succeeds if no check can tell the content is synthetic.

Generative AI fraud sits next to terms that are easy to mix up:

• Deepfake is one type of generative AI content, aimed at faces and voices.
• Digital forgery is a fabricated document image, often made with generative tools.
• Agentic fraud is fraud run by autonomous AI agents.