Digital Multi-Media Forensics

My work includes the photo-realistic video synthesis and editing which has a variety of useful applications (e.g., AR/VR telepresence, movie post-production, medical applications, virtual mirrors, virtual sightseeing). The development of algorithms for photo-realistic creation or editing of image content comes with a certain responsibility, since the generation of photo-realistic imagery can be misused. With the rise of methods that are able to synthesize photo-realistic content, we alread see that the society is confronted with fake imagery that is used for malicious purposes (fake news, cyber mobbing). Thus, the automatic detection of synthetic or manipulated content is of paramount importance (e.g., with a browser plugin that automatically flags manipulated images and videos). Gaining knowledge about the creation process will help to design forgery detection algorithms and vice versa. Given the findings of the current literature, an omnipotent detection algorithm that is able to detect a variety of manipulations is still a challenging problem, since most methods do not generalize to unseen manipulation methods. (Online) Self-supervised and few-shot learning methods show promising results. Especially, learning from real examples how a specific person behaves, looks and talks could lead to detection methods that do not overfit to a specific manipulation method. Note that findings in the forensics community can be used to improve the synthesis, since it provides a measurement whether a manipulation is good (deceiving the detector) or bad (detected as manipulation). In contrast to 'passive' forensic methods that only get access to the image or video, one can also actively add cryptographic signatures or watermarks to images and videos. Digital signatures ideally ensure that the media is created by a specific person and that it is not modified afterwards by a second person. While such digital signatures are standard elements for websites, encrypted emails, etc., it is rarely used for images and videos which makes the research for passive detection methods so important.

BIDT: Echt oder?

What is the impact of DeepFakes on our society? How does it change our view on digital media? Is the technology a danger or a chance?

CVPR 2020: Workshop on Media Forensics

This CVPR workshop covers the advances on all fronts of media forensics: from detection of manipulations, biometric implications, misrepresentation/spoofing, etc.

Learning to Detect Manipulated Facial Images

In this paper, we examine the realism of state-of-the-art facial image manipulation methods, and how difficult it is to detect them - either automatically or by humans. In particular, we create a datasets that is focused on DeepFakes, Face2Face, FaceSwap, and Neural Textures as prominent representatives for facial manipulations.

[Paper]  [Video]  [Bibtex] 

ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection

ForensicTransfer tackles two challenges in multimedia forensics. First, we devise a learning-based forensic detector which adapts well to new domains, i.e., novel manipulation methods. Second we handle scenarios where only a handful of fake examples are available during training.

[Paper]  [Bibtex]