Deepfake Detection that Generalizes Across Benchmarks

100%

Deepfake Detection that Generalizes Across Benchmarks

Abstract

The generalization of deepfake detectors to unseen manipulation techniques remains a challenge for practical deployment. Although many approaches adapt foundation models by introducing significant architectural complexity, this work demonstrates that robust generalization is achievable through a parameter-efficient adaptation of one of the foundational pre-trained vision encoders. The proposed method, GenD, fine-tunes only the Layer Normalization parameters (zero point zero three percent of the total) and enhances generalization by enforcing a hyperspherical feature manifold using L two normalization and metric learning on it.

We conducted an extensive evaluation on fourteen benchmark datasets spanning from twenty nineteen to twenty twenty-five. The proposed method achieves state-of-the-art performance, outperforming more complex, recent approaches in average cross-dataset AU-ROC. Our analysis yields two primary findings for the field: one) training on paired real-fake data from the same source video is essential for mitigating shortcut learning and improving generalization, and two) detection difficulty on academic datasets has not strictly increased over time, with models trained on older, diverse datasets showing strong generalization capabilities.

This work delivers a computationally efficient and reproducible method, proving that state-of-the-art generalization is attainable by making targeted, minimal changes to a pre-trained foundational image encoder model. The code is at:

One. Introduction

The proliferation of realistic facial deepfakes raises significant concerns regarding misinformation and malicious use, with AI-manipulated videos - those altered by techniques like face swapping or face reenactment - making detection challenging. Unlike fully synthetic content, such forgeries preserve the original context and leave subtle artifacts that are difficult for humans and machines to detect.

A primary issue affecting current detection methods is their limited ability to generalize. A model that has been trained to identify images altered by a particular deepfake generation algorithm often struggles when faced with examples produced by a new generation algorithm.

The generalization gap is the primary issue that we address in this work. Assuming the hypothesis that adapted large-scale, pre-trained foundational vision encoder can serve as a general foundation for deepfake detection, we build the proposed method in three variants, using Contrastive Language-Image Pre-training, Perception Encoder, and DINO models as feature extractors, which are known for their generalizable visual representations.

The proposed method consists of a vision encoder, whose outputs are L two-normalized. We then fine-tune only the parameters of the Layer Normalization blocks while keeping the rest frozen. Additionally, we propose using metric learning in this L two space to enhance generalization.

We benchmarked the generalization capabilities of the proposed model on fourteen deepfake video datasets released between twenty nineteen and twenty twenty-five, listed in Table one. To our knowledge, this represents the broadest evaluation in the deepfake literature. We show that the proposed model outperforms the most recent state-of-the-art methods on the majority of all available benchmarks.

In summary, our key contributions are as follows:

· A novel deepfake detection method called GenD. The method achieves the best average cross-dataset AUROC compared to recently released models.

· The most comprehensive evaluation in the deepfake literature covering datasets released throughout six years of research.

· A demonstration that to achieve the best generalization and prevent shortcut learning, it is essential to construct the training set consisting of real-fake pairs, where the fake video is generated from the real counterpart of the pair.

Two. Related Work

Three. Method

Three point one. Model

Three point two. Data

Four. Experiments

Four point one. Test benchmarks

Four point two. Cross-dataset evaluation

Four point three. Ablation studies

Four point four. Importance of training on paired dataset

Four point five. Evolution of detection difficulty over the years

Four point six. Robustness to image degradations

Five. Limitations and Future Work

Six. Conclusions

Overview

The paper introduces GenD, an efficient deepfake detection technique that outperforms complex models by fine-tuning a small percentage of parameters in foundational vision encoders. It highlights the importance of training with paired real-fake data for improved generalization and presents comprehensive evaluations across 14 datasets.

Key Points

1GenD fine-tunes only Layer Normalization parameters, making it computationally efficient
2The method achieves state-of-the-art performance across various deepfake benchmarks
3Training on paired real-fake data is crucial for effective generalization
4The study provides insights into the historical progression of detection difficulty in academic datasets
5Comprehensive evaluation covering six years of deepfake research demonstrates the model's capabilities.

Details

Authors: Andrii Yermakov, Jan Cech, Jiri Matas, Mario Fritz
Category: Technology and Engineering

PDF
Proficiency of records practitioners using artificial intelligence to enhance records management practices at the Gauteng Department of Education in South Africa
This study examines the proficiency of records practitioners at the Gauteng Department of Education in using artificial intelligence (AI) to improve records management practices. It employs a mixed-methods research approach to evaluate the existing skill levels and proposes a framework for upskilling these practitioners in AI utilization.
PDF
Intelligent Archive Management Based on Deep Learning Technology Driven by Artificial Intelligence
This research article presents a study on an intelligent archive management system (AMS) that employs deep learning (DL) technology to enhance the efficiency and accuracy of processing multimodal data. The proposed model addresses traditional AMS shortcomings such as data classification accuracy and system responsiveness.
PDF
Discharging Records Management Activities Using Artificial Intelligence at the Council for Scientific and Industrial Research, South Africa
This study investigates the application of artificial intelligence (AI) in enhancing records management activities at the Council for Scientific and Industrial Research (CSIR) in South Africa. It provides a framework for utilizing AI to improve efficiency in managing records throughout their life cycle.
PDF
Human-like systematic generalization through a meta-learning neural network
This article explores the capacity of neural networks to achieve human-like systematicity through a meta-learning approach that enhances compositional skills. It presents experimental findings comparing human and machine performance on systematic generalization tasks.
PDF
TBI-603 Network Security & Cyber Law Unit Notes
This document serves as comprehensive unit notes for a course on Network Security and Cyber Law, covering topics like network security practices, types of security attacks, and legal aspects of cybersecurity.