Can AI proctoring detect a lip-sync mismatch between a candidate's audio and video?

TL;DR: Yes. Lip-sync mismatch detection is one of the core deepfake-detection signals AI proctoring uses, alongside blinking patterns, lighting consistency, and face-boundary artifacts — because most real-time face-swap and deepfake video tools still struggle to perfectly align mouth movement to audio, especially during fast speech.

By Pinal Dave | Last updated: 2026-08-05

The claim

Lip-sync mismatch is a well-documented weak point in real-time deepfake video generation — it's harder to solve in a live call than in pre-rendered video, because the system has to generate mouth movement on the fly with no chance to correct errors after the fact.

The evidence

Presentation-attack detection research (the same discipline biometric security uses to catch spoofed fingerprints and faces) treats audio-visual synchrony as a primary liveness signal, precisely because synthetic video generation pipelines introduce measurable latency and drift between generated mouth shapes and the underlying audio waveform. This is why deepfake detection in interview proctoring runs continuously through a call rather than as a single check at login — mismatch tends to show up under conversational speed and complex phonemes, not in a static introduction.

Step-by-step: how lip-sync detection works during a live interview

The system establishes a baseline of the candidate's natural mouth-movement-to-audio correlation in the first moments of the call.
Throughout the interview, it continuously compares expected mouth shapes (given the audio) against the actual video feed.
Deviations beyond a natural-variation threshold are logged as a synchrony anomaly.
Combined with other deepfake signals (blink rate, face-edge artifacts, lighting consistency), a sustained mismatch pattern raises a trust-score flag.
The flagged timestamp is included in the trust report for human review alongside the recording.

FAQ

Can a candidate's poor internet connection cause a false lip-sync flag? Network latency and video compression are accounted for separately from synchrony analysis, but severe connection issues can affect signal quality — this is why proctoring combines multiple deepfake indicators rather than relying on lip-sync alone.

Does this work for phone-only interviews with no video? No — lip-sync detection requires a video feed by definition; audio-only calls rely on voice-analysis signals instead.

How is this different from a virtual camera detection flag? Virtual camera detection identifies whether the video source is a real camera versus injected software; lip-sync mismatch is a content-level signal that catches deepfakes even running through a legitimate camera pipeline.