Deepfake Detection that Generalizes Across Benchmarks

100%

Deepfake Detection that Generalizes Across Benchmarks

Abstract

The generalization of deepfake detectors to unseen manipulation techniques remains a challenge for practical deployment. Although many approaches adapt foundation models by introducing significant architectural complexity, this work demonstrates that robust generalization is achievable through a parameter-efficient adaptation of one of the foundational pre-trained vision encoders. The proposed method, GenD, fine-tunes only the Layer Normalization parameters (zero point zero three percent of the total) and enhances generalization by enforcing a hyperspherical feature manifold using L two normalization and metric learning on it.

We conducted an extensive evaluation on fourteen benchmark datasets spanning from twenty nineteen to twenty twenty-five. The proposed method achieves state-of-the-art performance, outperforming more complex, recent approaches in average cross-dataset AU-ROC. Our analysis yields two primary findings for the field: one) training on paired real-fake data from the same source video is essential for mitigating shortcut learning and improving generalization, and two) detection difficulty on academic datasets has not strictly increased over time, with models trained on older, diverse datasets showing strong generalization capabilities.

This work delivers a computationally efficient and reproducible method, proving that state-of-the-art generalization is attainable by making targeted, minimal changes to a pre-trained foundational image encoder model. The code is at: https://github.com/yermandy/GenD.

One. Introduction

The proliferation of realistic facial deepfakes raises significant concerns regarding misinformation and malicious use, with AI-manipulated videos - those altered by techniques like face swapping or face reenactment - making detection challenging. Unlike fully synthetic content, such forgeries preserve the original context and leave subtle artifacts that are difficult for humans and machines to detect.

A primary issue affecting current detection methods is their limited ability to generalize. A model that has been trained to identify images altered by a particular deepfake generation algorithm often struggles when faced with examples produced by a new generation algorithm.

The generalization gap is the primary issue that we address in this work. Assuming the hypothesis that adapted large-scale, pre-trained foundational vision encoder can serve as a general foundation for deepfake detection, we build the proposed method in three variants, using Contrastive Language-Image Pre-training, Perception Encoder, and DINO models as feature extractors, which are known for their generalizable visual representations.

The proposed method consists of a vision encoder, whose outputs are L two-normalized. We then fine-tune only the parameters of the Layer Normalization blocks while keeping the rest frozen. Additionally, we propose using metric learning in this L two space to enhance generalization.

We benchmarked the generalization capabilities of the proposed model on fourteen deepfake video datasets released between twenty nineteen and twenty twenty-five, listed in Table one. To our knowledge, this represents the broadest evaluation in the deepfake literature. We show that the proposed model outperforms the most recent state-of-the-art methods on the majority of all available benchmarks.

In summary, our key contributions are as follows:

· A novel deepfake detection method called GenD. The method achieves the best average cross-dataset AUROC compared to recently released models.

· The most comprehensive evaluation in the deepfake literature covering datasets released throughout six years of research.

· A demonstration that to achieve the best generalization and prevent shortcut learning, it is essential to construct the training set consisting of real-fake pairs, where the fake video is generated from the real counterpart of the pair.

Two. Related Work

Three. Method

Three point one. Model

Three point two. Data

Four. Experiments

Four point one. Test benchmarks

Four point two. Cross-dataset evaluation

Four point three. Ablation studies

Four point four. Importance of training on paired dataset

Four point five. Evolution of detection difficulty over the years

Four point six. Robustness to image degradations

Five. Limitations and Future Work

Six. Conclusions

Overview

The document introduces a unique approach to deepfake detection, focusing on generalization capabilities across manipulation techniques. By using a parameter-efficient adaptation method on foundational vision encoders, significant performance improvements on benchmark datasets are achieved.

Key Points

1The method called GenD fine-tunes only Layer Normalization parameters for effective detection
2Extensive evaluation on 14 benchmark datasets shows robust generalization capabilities
3Training on paired real-fake data is necessary to mitigate shortcut learning
4The proposed approach outperforms more complex models in average cross-dataset AUROC
5Demonstrates that state-of-the-art performance can be achieved with minimal changes to existing models.

Details

Authors: Andrii Yermakov, Jan Cech, Jiri Matas, Mario Fritz
Category: Technology and Engineering

PDF
Historical Development of the Information Age
This document explores the evolution of information technology from ancient writing systems to modern digital communication, highlighting significant milestones throughout history. It chronicles advancements such as the invention of the printing press, telegraph, and smartphones, illustrating their impact on society.
PDF
C Control Structure Language: Switch Case Structure
This document explains the switch-case statement in C programming, detailing its structure, functionality, and use cases in branching logic. It provides examples and highlights the importance of break statements within the switch-case context.
PDF
Control Structures in C Programming
This document provides an overview of control structures in the C programming language, including decision-making and repetition control structures, and examples of their implementation.
PDF
Web Programming
This document provides an overview of web programming, detailing the evolution of the web from static pages to dynamic, user-interactive websites. It covers key technologies, protocols, and the difference between the Internet and the World Wide Web.
PDF
Introduction to HTML
This document serves as an introductory guide to HTML (HyperText Markup Language), detailing its structure, elements, and usage for creating web pages. It explains basic tags, attributes, and the importance of properly nesting elements.