My work includes the photo-realistic video synthesis and editing which has a variety of useful applications (e.g., AR/VR telepresence, movie post-production, medical applications, virtual mirrors, virtual sightseeing). The development of algorithms for photo-realistic creation or editing of image content comes with a certain responsibility, since the generation of photo-realistic imagery can be misused. With the rise of methods that are able to synthesize photo-realistic content, we alread see that the society is confronted with fake imagery that is used for malicious purposes (fake news, cyber mobbing). Thus, the automatic detection of synthetic or manipulated content is of paramount importance (e.g., with a browser plugin that automatically flags manipulated images and videos). Gaining knowledge about the creation process will help to design forgery detection algorithms and vice versa. Given the findings of the current literature, an omnipotent detection algorithm that is able to detect a variety of manipulations is still a challenging problem, since most methods do not generalize to unseen manipulation methods. (Online) Self-supervised and few-shot learning methods show promising results. Especially, learning from real examples how a specific person behaves, looks and talks could lead to detection methods that do not overfit to a specific manipulation method. Note that findings in the forensics community can be used to improve the synthesis, since it provides a measurement whether a manipulation is good (deceiving the detector) or bad (detected as manipulation). In contrast to 'passive' forensic methods that only get access to the image or video, one can also actively add cryptographic signatures or watermarks to images and videos. Digital signatures ideally ensure that the media is created by a specific person and that it is not modified afterwards by a second person. While such digital signatures are standard elements for websites, encrypted emails, etc., it is rarely used for images and videos which makes the research for passive detection methods so important.
EG Junior Fellow
I have been officially elected to the circle of EG Junior Fellows.2021, May 14 — 1 minute read
Neural Capture and Synthesis at the Max Planck Institute for Intelligent Systems
I am happy to announce that I am joining the Max Planck Institute for Intelligent Systems as a research group leader. My Max Planck Research Group 'Neural Capture and Synthesis' will work at the intersection of computer graphics, computer vision and machine learning.2021, Jan 31 — 3 minute read
Cabinet Meeting: Synthetic Media - Danger or Opportunity?
In this session we showed several demonstrations of our current projects, to inform the Cabinet of Germany about the risks and the opportunities of synthetic media.2019, Nov 18 — 1 minute read