DEEPFAKE DETECTION METHOD BASED ON IDENTITY AND FACE SHAPE FEATURES

Smart overview of the Invention

The patent describes a method for detecting Deepfakes by leveraging identity and face shape features. This approach integrates identity characteristics with three-dimensional (3D) face shape data, utilizing specialized modules to identify inconsistencies in these features. The method employs a face shape consistency self-attention (FSCA) module and an identity guided shape consistency attention (IGSCA) module, which together enhance the detection accuracy by focusing on discrepancies between the reference and target faces.

Technical Field

This innovation falls within the realm of Deepfake detection technology, specifically targeting methods that utilize both identity and facial shape attributes. It addresses the limitations of traditional binary classification methods and aims to improve generalization across various faking techniques, which often degrade the performance of existing detection models.

Background

The rise of Deepfake technology has made it increasingly difficult for individuals to discern real from fake media. While Deepfakes have legitimate uses in entertainment and media, they also pose risks for misuse in fraud and misinformation. Existing detection methods often struggle with generalization and fail to accurately identify Deepfakes created by unfamiliar techniques, prompting the need for more robust solutions like the one proposed.

Methodology

The detection method involves several steps: acquiring video data to form training and test sets, extracting identity and face shape features using specialized encoders, and constructing a network to fuse these features. The process includes calculating a loss function for training and employing cosine similarity to compare features from test videos against reference data. This comparison helps determine the authenticity of the face in question.

Implementation Details

Key steps include using a facial forgery dataset for training and testing, employing convolutional networks for facial keypoint detection, and leveraging PyTorch functions for tensor manipulation. The method constructs an identity encoder with an ArcFace model for identity recognition and a 3D reconstruction encoder for face shape features. These components are integrated within a network that uses FSCA and IGSCA modules to refine detection capabilities.