Voice cloning

Using AI to recreate a specific person's voice from a short recording, well enough to fool a phone call.

A fraudster might take a few seconds of someone's speech from a video or a voicemail and generate new sentences in that voice.

A deepfake subtype that synthesizes speech in a target's voice from a short sample, often just seconds of audio pulled from a public source like an earnings call or a video, and is the attack that voice biometrics systems must defend against.

A voice cloning attack typically works like this:

1. The attacker collects a sample of the target's voice from public audio.
2. A model learns the voice's characteristics.
3. The attacker generates speech in that voice, either from typed text or by converting their own speech in real time.
4. They use the cloned voice on a call, often with an urgent request to move money.

Defenses include:

• Callback verification, which hangs up and calls back on a number from the organization's own records.
• Shared secrets, such as a codeword or a question only the real person could answer.
• Approval processes, which require a second person to authorize transfers.
• Synthetic voice detection, which analyzes audio for signs of generation.

Voice cloning sits next to terms that are easy to mix up:

• Voice biometrics recognizes a person by voice, and cloning is the attack that tries to fool it.
• Deepfake is the broader category that includes cloned voices.
• Business email compromise is impersonation fraud by email, which voice cloning extends to phone calls.

For more, see Top 5 Cases of AI Deepfake Fraud From 2024 Exposed.