Tutorials & Demos

My work includes the photo-realistic video synthesis and editing which has a variety of useful applications (e.g., AR/VR telepresence, movie post-production, medical applications, virtual mirrors, virtual sightseeing). The development of algorithms for photo-realistic creation or editing of image content comes with a certain responsibility, since the generation of photo-realistic imagery can be misused. With the rise of methods that are able to synthesize photo-realistic content, we alread see that the society is confronted with fake imagery that is used for malicious purposes (fake news, cyber mobbing). Thus, the automatic detection of synthetic or manipulated content is of paramount importance (e.g., with a browser plugin that automatically flags manipulated images and videos). Gaining knowledge about the creation process will help to design forgery detection algorithms and vice versa. Given the findings of the current literature, an omnipotent detection algorithm that is able to detect a variety of manipulations is still a challenging problem, since most methods do not generalize to unseen manipulation methods. (Online) Self-supervised and few-shot learning methods show promising results. Especially, learning from real examples how a specific person behaves, looks and talks could lead to detection methods that do not overfit to a specific manipulation method. Note that findings in the forensics community can be used to improve the synthesis, since it provides a measurement whether a manipulation is good (deceiving the detector) or bad (detected as manipulation). In contrast to 'passive' forensic methods that only get access to the image or video, one can also actively add cryptographic signatures or watermarks to images and videos. Digital signatures ideally ensure that the media is created by a specific person and that it is not modified afterwards by a second person. While such digital signatures are standard elements for websites, encrypted emails, etc., it is rarely used for images and videos which makes the research for passive detection methods so important.

CLIPme if you can!

CLIP is a powerful tool to match images to text descriptions. It can be used for classification of ImageNet classes as shown in the original paper, but it can also be used to assign names of (famous) people to images.

2 minute read    

3DV 2021: Tutorial on the Advances in Neural Rendering

In this tutorial, we will talk about the advances in neural rendering, especially the underlying 2D and 3D representations that allow for novel viewpoint synthesis, controllability and editability. Specifically, we will discuss neural rendering methods based on 2D GANs, techniques using 3D Neural Radiance Fields or learnable sphere proxies. Besides methods that handle static content, we will talk about dynamic content as well.

1 minute read     [Video] 

SIGGRAPH 2021: Course on the Advances in Neural Rendering

This course covers the advances in neural rendering over the years 2020-2021.

1 minute read     [Video]  [Bibtex] 

BIDT: Echt oder?

What is the impact of DeepFakes on our society? How does it change our view on digital media? Is the technology a danger or a chance?

2 minute read    

CVPR 2020: Tutorial on Neural Rendering

Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. This state-of-the-art report summarizes the recent trends and applications of neural rendering.

1 minute read     [Paper]  [Video]  [Bibtex] 

CVPR 2020: Workshop on Media Forensics

This CVPR workshop covers the advances on all fronts of media forensics: from detection of manipulations, biometric implications, misrepresentation/spoofing, etc.

1 minute read    

Eurographics 2018: State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications

This state-of-the-art report session summarizes recent trends in monocular facial performance capture and discusses its applications, which range from performance-based animation to real-time facial reenactment. We focus on methods where the central task is to recover and track a three dimensional model of the human face using optimization-based reconstruction algorithms.

1 minute read     [Paper]  [Bibtex] 

SIGGRAPH Emerging Technologies: Demo of FaceVR

We present a novel method for the interactive markerless reconstruction of human heads using a single commodity RGB‐D sensor. Our entire reconstruction pipeline is implemented on the graphics processing unit and allows to obtain high‐quality reconstructions of the human head using an interactive and intuitive reconstruction paradigm.

1 minute read     [Paper]  [Video]  [Bibtex] 

SIGGRAPH Emerging Technologies: Real-time Face Capture and Reenactment of RGB Videos

We show a demo for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). Our goal is to animate the facial expressions of a target video by a source actor and re-render the manipulated output video in a photo-realistic fashion.

1 minute read     [Paper]  [Video]  [Bibtex]