PhD Student at MPI/INRIA - "Robust 3D Hand-Object Interaction"

PhD Student/Doktorand
MPI for Intelligent Systems (MPI-IS) - Perceiving Systems (PS)
INRIA Grenoble Rhône-Alpes - THOTH project-team

Title: “Robust 3D Hand-Object Interaction”

URL: … -doktorand

Position: Doctoral research position (PhD) in the Perceiving Systems (PS) department of MPI-IS ( and THOTH project-team of INRIA (

Project: Hands are important to humans for interacting with the physical world. In this project we use computer vision to jointly reason about interacting hands and manipulated objects. We aim to create a new generation of datasets, benchmarks and tools, using 2D/3D vision, robotics and machine learning techniques, that will automate and revolutionize the capture, analysis, control and our general understanding of hands physically interacting with objects.

For related work please visit:

The challenges of this project include:

  • Articulated tracking of hand-object interactions from multi-view images, 4D scans, motion capture data.
  • Registration of multimodal data: images, videos, meshes, force measurements, etc.
  • Efficient computation of proximal hand-object relations (e.g. distances, contact, penetration).
  • Creating a novel realistic dataset with accurate ground truth for training and evaluation.
  • Introducing novel models for shape and pose estimation, motion generation and retargeting.

Scientific Context: MPI and INRIA share expertise in the fields of computer vision, graphics and machine learning, with publications in major venues such as CVPR, ICCV, ECCV and SIGGRAPH. PS has particular expertise on statistical modeling and motion capture of humans with world-class equipment. THOTH has a particular expertise on machine learning for human pose estimation and action recognition.

Candidate: We seek candidates interested in computer vision, graphics, robotics and machine learning. The successful candidate will have a Master’s degree (or equivalent) in Computer Science (or related) and skills in: written and oral communication in English, Python/C++ programming, working both independently and in a team. Demonstrated experience in any of the following will be appreciated: computer vision/graphics, AR/VR, 3D simulation/game/physics engines, 3D geometry processing, numerical optimization (e.g. ceres), neural networks (e.g. PyTorch, TF), robotics (control, path planning), CUDA, GPU parallelization, etc. Prior publications, internships, industrial experience, participation in open-source projects or competitions are a plus.

Advising & Location: The student will be advised by Michael Black (PS/MPI, and Cordelia Schmid (THOTH/INRIA, and will collaborate closely with Dimitrios Tzionas (PS/MPI, The PhD research will be centered in Tübingen (Germany) with possible visits to Grenoble (France). At PS/MPI a surrounding team of approximately 10 PhDs, 6 senior researchers and 4 engineers are available for interactions and possible collaboration.

The PhD student (m/f/d) will receive a PhD funding contract equivalent in remuneration to pay group E13, 65% of the Collective Wage Agreement for the Public Service. An initial contract will be given for 3 years with possibility of 1 - year extension.

Application: Please apply only through the portal of the IMPRS program (Application | IMPRS-IS International Max Planck Research School for Intelligent Systems). Please mention Michael Black, Cordelia Schmid and Dimitrios Tzionas in your application. The full application deadline, including reference letters, is November 6th 2019 11:59am CET for a start in 2020.

The Max Planck Society is committed to increasing the number of individuals with disabilities in its workforce and therefore encourages applications from such qualified individuals. The Max Planck Society seeks to increase the number of women in those areas where they are underrepresented and therefore explicitly encourages women to apply.