PhD Position in Deep Learning for Acoustic Sensor Fusion on Intelligent Vehicles

In this project you will develop novel AI and Deep Learning techniques for traffic detection using vehicle-mounted microphones, enhancing the perception capabilities of Advanced Driver-Assistance Systems (ADAS) and Automated Vehicles (AVs) beyond traditional camera, radar, and lidar sensors. Unlike these sensors, which require direct line-of-sight, acoustic detection captures traffic sounds like tire noise, sirens, and electric vehicle signals, enabling anticipation of unseen traffic in low-visibility scenarios. This additional sensing modality can improve driving safety, for instance by detecting approaching vehicles around corners or behind obstacles, and by localizing salient sounds such as sirens. Your research will explore new use cases of acoustic perception for advanced automated driving systems, and new methods for fusing sound with other sensor data for more robust environment perception through deep learning.

We offer a fully-funded 4-year PhD position at the Intelligent Vehicles Group in the Cognitive Robotics department of TU Delft. The group frequently publishes in top conferences and journals such as CVPR, ECCV, ICRA, IROS, T-IV, T-ITS, T-RO, T-PAMI. Your primary supervisor will be Dr. Julian Kooij. The project is co-funded and co-supervised by the Audio Innovation Center of NXP Semiconductors in Leuven (NASDAQ: NXPI). For your work you will have access to the compute resources of TU Delft, ranging from personal machines, to shared GPU servers, the Delft AI Cluster that is shared across departments, as well as DelftBlue , which is one of the top 250 supercomputers in the world.

At the Intelligent Vehicles Group you'll find an open and friendly environment, with opportunities for professional development and training to successfully develop your academic skills. We have a collaborative culture regarding research, education, and use of lab resources. We interact on a daily basis and share a drive to strengthen the position of the group as a whole thanks to the growth of each member. In addition, you will have regular progress meetings with experts from NXP's Audio Innovation Centre.

Research challenges

You will improve on state-of-the-art deep learning techniques for multi-sensor environment perception in autonomous driving by integrating acoustics. Possible research directions include the use of audio-visual foundation models, audio-driven sensor fusion for object detection, cross-modal representation learning, and self-supervised learning for this novel perception task. The developed models should provide holistic representations of all surrounding traffic by fusing multi-microphone data with other sensor modalities. Such holistic traffic representations support many driving tasks at once, as required for full self-driving. The addition of acoustics should improve the robustness of the existing sensor suite.

A unique asset is the IV group's Prius demonstrator vehicle with cameras, lidars, radars, and a roof-mounted microphone array. It provides you unique multi-sensor data to investigate and demonstrate how acoustics can improve vehicle perception tasks, for which you collaborate with another PhD candidate in this project. Together with other researchers in the IV group, you help develop and maintain the software for this shared demonstrator vehicle., Doctoral candidates will be offered a 4-year period of employment in principle, but in the form of 2 employment contracts. An initial 1,5 year contract with an official go/no go progress assessment within 15 months. Followed by an additional contract for the remaining 2,5 years assuming everything goes well and performance requirements are met.

Do you have a Master's degree?, * Completed (or about to complete) a MSc degree related to any of: artificial intelligence, machine learning, intelligent vehicles / robotics, acoustics and signal processing, computer vision.

• Demonstratable experience in applying Deep Learning, using PyTorch, TensorFlow, JAX on real-world sensor data.
• Experience with robotic/vehicle perception tasks with computer vision, lidar or radar. Experience in acoustic perception is a plus.
• Knowledge of the Robot Operating System (ROS), and signal processing, is a plus.
• Good theoretic understanding of the fundamentals of Machine Learning.
• Ability to act independently as well as to collaborate effectively with members of a larger interdisciplinary team, take initiative, be result oriented, organized and creative.
• Excellent programming skills (Python, possibly also C/C++).
• Good command of verbal and written English., * A complete record of Bachelor and Master courses (including grades).
• Your Master's Thesis (at least as draft).
• A list of any projects or publications you have worked on with brief descriptions of your contributions (max 2 pages).
• The names and contact addresses of two references.

You can address your application to Julian Kooij.

Doing a PhD at TU Delft requires English proficiency at a certain level to ensure that the candidate is able to communicate and interact well, participate in English-taught Doctoral Education courses, and write scientific articles and a final thesis. For more details please check the Graduate Schools Admission Requirements.

Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities , increasing from €3059 - €3881 gross per month, from the first year to the fourth year based on a fulltime contract (38 hours), plus 8% holiday allowance and an end-of-year bonus of 8.3%.

As a PhD candidate you will be enrolled in the TU Delft Graduate School. The TU Delft Graduate School provides an inspiring research environment with an excellent team of supervisors, academic staff and a mentor. The Doctoral Education Programme is aimed at developing your transferable, discipline-related and research skills.

The TU Delft offers a customisable compensation package, discounts on health insurance, and a monthly work costs contribution. Flexible work schedules can be arranged.

PhD Position in Deep Learning for Acoustic Sensor Fusion on Intelligent Vehicles TU Delft 4.14.1 out of 5 stars Julianalaan 67A, Delft, Delft University of Technology is built on strong foundations. As creators of the world-famous Dutch waterworks and pioneers in biotech, TU Delft is a top international university combining science, engineering and design. It delivers world class results in education, research and innovation to address challenges in the areas of energy, climate, mobility, health and digital society. For generations, our engineers have proven to be entrepreneurial problem-solvers, both in business and in a social context. At TU Delft we embrace diversity as one of our core values and we actively engage to be a university where you feel at home and can flourish. We value different perspectives and qualities. We believe this makes our work more innovative, the TU Delft community more vibrant and the world more just. Together, we imagine, invent and create solutions using technology to have a positive impact on a global scale. That is why we invite you to apply. Your application will receive fair consideration.