Doctoral Fellow in Information Technology

IDLab is a research group of Ghent University, as well as core research group of imec. IDLab performs fundamental and applied research on data science and internet technology, and is structured in 21 research teams. The research for this position will be conducted within the DECIDE research team.

DECIDE conducts cutting-edge research on Distributed and Embodied Computing in Dynamic Environments. The team works to realize a vision where humans and robots cooperate seamlessly, enabled by advanced planning and control algorithms, real-time and energy-efficient sensing, and distributed perception. Its research is applied across diverse sectors, including agriculture, manufacturing and remote sensing.

Research Topic

As robots transition from single task machines to general purpose agents operating in dynamic and unpredictable environments, the computational demands of the algorithms controlling their behavior will increase exponentially. A promising research direction is the use of Vision-Language-Action (VLA) or world models that unify perception, reasoning and control in one large foundation model. Using an input image of the robot’s surroundings and a textual command, these models are able to output low-level robot actions that can be executed to accomplish the requested task. These models are however computationally expensive. Google Deepmind’s RT-2 model for example, contains a staggering 55B parameters or 220 GB when stored in 32 bit floating point format. Evaluating the model requires 149 TFLOPs per inference pass. As an inference rate of 3 to 10 Hz is needed to operate the robot in real time, this is currently not feasible on edge devices and cloud infrastructure is required. In this project, you will develop the next generation of robot VLA and world models. By combining optimizations such as quantization, pruning and distillation, we aim to build models that can be deployed on real-world embedded hardware.

Role and responsibilities

Your main tasks include:

• You will investigate and extend current state-of-the-art robot control models such as VLA’s and world models with a focus on robot navigation and multi-step task execution.
• You will apply your work to real-world use cases.
• Writing high quality publications, targeting top journals and international conferences.
• You will take on a mentoring role by supervising bachelor and master theses related to the subject of your PhD.

We are looking for a highly creative and motivated PhD student with the following qualifications and skills:

• You have (or will obtain in the next months) a European master’s degree in computer science, Artificial Intelligence, or equivalent, with excellent (‘honors’-level or above) grades.
• You have a strong background in deep learning.
• Previous experience with robotics, world models, reinforcement learning or other ML-based techniques for robot control is considered a plus.
• You have excellent computer science skills (python, git, linux, ROS, etc.) .
• You have hands-on experience with machine learning frameworks such as PyTorch.
• You have strong analytical skills to interpret the obtained research results.
• You are a team player and have strong communication skills.
• Your English is fluent (C1 CEFR level), both speaking and writing. Mastery of Dutch (both speaking and writing) is considered as a plus.

Ghent University is a world of its own. Employing more than 15.000 people, it is actively involved in education and research, management and administration, as well as technical and social service provision on a daily basis. It is one of the largest, most exciting employers in the area and offers great career opportunities. With its 11 faculties and more than 85 departments offering state-of-the-art study programmes grounded in research in a wide range of academic fields, Ghent University is a logical choice for its staff and students. For FEA/INTEC/IDLab we are looking for a m/f/x Doctoral fellow, Ghent University consistently ranks among the best 100 universities in the world. Located in the heart of Europe, Ghent is a beautiful and welcoming city (https://www.lonelyplanet.com/articles/ghent-belgiums-best-kept-secret) with plenty of cultural and leisure activities.