Neural Capture and Synthesis at the Max Planck Institute for Intelligent Systems

 

The main theme of my work is to capture and to (re-)synthesize the real world using commodity hardware. It includes the modeling of the human body, tracking, as well as the reconstruction and interaction with the environment. The digitization is needed for various applications in AR/VR as well as in movie (post-)production. Teleconferencing and working in VR is of high interest for many companies ranging from social media platforms to car manufacturer. It enables the remote interaction in VR, e.g., the inspection of 3D content like CAD models or scans from real objects. A realistic reproduction of appearances and motions is key for such applications. Capturing natural motions and expressions as well as the photorealistic reproduction of images under novel views are challenging. With the rise of deep learning methods and, especially, neural rendering, we see immense progress to succeed in these challenges. The goal of my work is to develop methods for AI-based image synthesis of humans, the underlying representation of appearance, geometry and motion to allow for explicit and implicit control over the synthesis process. My work on 3D reconstruction, tracking and rendering does not focus exclusively on humans but also on the environment and objects we interact with, thus, enabling applications like 3d telepresence or collaborative working in VR. In both areas, reconstruction and rendering, hybrid approaches that combine novel findings in machine learning with classical computer graphics and computer vision approaches show promising results. Nevertheless, these methods still suffer from limitations like generalizability, controllability and editability which I will tackle in my ongoing and future work.

 

Face Projects  

Scene Reconstruction Projects  

Neural Non-rigid Tracking Projects  

On my website and on my YouTube page you will find a couple of interesting projects I have been working on in the past:




Openings

My group as several open positions for Postdocs and PhD students. If you are interested and highly motivated to work in the above mentioned research fields, please write me an email (justus.thies@tum.de) including the relevant application documents (CV, recommendation letters, transcript of records, etc.) and a short description of your research interests / plans (research statement).

Qualification

  • Applicants for a PhD must hold a Master’s degree in computer science or equivalent
  • Fluent written and spoken English skills are essential
  • Proficient coding skills (Python, C++ (this is critical and will be tested))
  • Experience with deep learning frameworks (TensorFlow / PyTorch)
  • Excitement, self-motivation, and commitment to revolutionize the field
  • Post Doc Applicants Only: academic track record with publications at top-tier venues in computer vision, graphics, or machine learning (CVPR, ECCV/ICCV, Siggraph, Siggraph Asia, NeurIPS, ICML, ICLR)

How to Apply

  • Documents required: CV, research statement, BA and MA transcripts, MA thesis [optional]
  • Ask two recommenders who know your work to directly email recommendation letters
  • Send all documents (incl. rec. letters) directly to Justus Thies (justus.thies@tum.de)
  • Please understand that we cannot review incomplete applications

Additional doctoral programs

The MPI for Intelligent System also offers the following doctoral programs you can apply for:

  • The International Max Planck Research School for Intelligent Systems (IMPRS-IS) brings together the MPI for Intelligent Systems with the University of Stuttgart and the University of Tübingen to form a highly visible and unique graduate school of internationally recognized faculty, working at the leading edge of the field.
  • The Max Planck ETH Center for Learning Systems (CLS) is a doctoral training program. Ph.D. students are jointly supervised by ETH Professors and MPI Directors and Group Leaders, and obtain their doctoral degree from ETH Zurich.