Research Community Services

My work includes the photo-realistic video synthesis and editing which has a variety of useful applications (e.g., AR/VR telepresence, movie post-production, medical applications, virtual mirrors, virtual sightseeing). The development of algorithms for photo-realistic creation or editing of image content comes with a certain responsibility, since the generation of photo-realistic imagery can be misused. With the rise of methods that are able to synthesize photo-realistic content, we alread see that the society is confronted with fake imagery that is used for malicious purposes (fake news, cyber mobbing). Thus, the automatic detection of synthetic or manipulated content is of paramount importance (e.g., with a browser plugin that automatically flags manipulated images and videos). Gaining knowledge about the creation process will help to design forgery detection algorithms and vice versa. Given the findings of the current literature, an omnipotent detection algorithm that is able to detect a variety of manipulations is still a challenging problem, since most methods do not generalize to unseen manipulation methods. (Online) Self-supervised and few-shot learning methods show promising results. Especially, learning from real examples how a specific person behaves, looks and talks could lead to detection methods that do not overfit to a specific manipulation method. Note that findings in the forensics community can be used to improve the synthesis, since it provides a measurement whether a manipulation is good (deceiving the detector) or bad (detected as manipulation). In contrast to 'passive' forensic methods that only get access to the image or video, one can also actively add cryptographic signatures or watermarks to images and videos. Digital signatures ideally ensure that the media is created by a specific person and that it is not modified afterwards by a second person. While such digital signatures are standard elements for websites, encrypted emails, etc., it is rarely used for images and videos which makes the research for passive detection methods so important.


3DV 2021: Tutorial on the Advances in Neural Rendering

In this tutorial, we will talk about the advances in neural rendering, especially the underlying 2D and 3D representations that allow for novel viewpoint synthesis, controllability and editability. Specifically, we will discuss neural rendering methods based on 2D GANs, techniques using 3D Neural Radiance Fields or learnable sphere proxies. Besides methods that handle static content, we will talk about dynamic content as well.

1 minute read     [Video] 

SIGGRAPH 2021: Course on the Advances in Neural Rendering

This course covers the advances in neural rendering over the years 2020-2021.

1 minute read     [Video]  [Bibtex] 

CVPR 2020: Tutorial on Neural Rendering

Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. This state-of-the-art report summarizes the recent trends and applications of neural rendering.

1 minute read     [Paper]  [Video]  [Bibtex] 

CVPR 2020: Workshop on Media Forensics

This CVPR workshop covers the advances on all fronts of media forensics: from detection of manipulations, biometric implications, misrepresentation/spoofing, etc.

1 minute read    

Eurographics 2018: State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications

This state-of-the-art report session summarizes recent trends in monocular facial performance capture and discusses its applications, which range from performance-based animation to real-time facial reenactment. We focus on methods where the central task is to recover and track a three dimensional model of the human face using optimization-based reconstruction algorithms.

1 minute read     [Paper]  [Bibtex]