Open-Set Object Detection: challenging VLM to understand unknown objects & contexts H/F

Supervised deep learning models have demonstrated significant performances to detect a closed set of known annotated classes seen during training. But, how will they behave when facing up to objects of unknown classes? As their behaviour is uncertain when subject to never-before-seen classes and contexts (e.g. aerial, medical imaging...), we aim to develop robust Open Set Object Detectors (OSOD), able to localise and classify any objects, no matter their classes are known or unknown during training, nor their domain. In specific domains, being able to provide semantic information about the unknown is also paramount and an understudied problem (e.g., characterizing the super-class of an unknown object)., * Study state-of-the-art methods of Open Set Object Detection (OSOD) as well as Visual Language Models (VLM) in the context of Open World containing both known and unknown objects;

• Design an object detector aware of the existence of the unknown, and able to describe the unknown by comparing it to or distinguishing it from what is known via certain characteristics that can be described textually;
• Evaluate the proposed method on recent OSOD benchmarks and compare to the state of the art, Diplôme préparé

Bac+5 - Master of Science

Formation recommandée

Master or Ecole d'ingénieur

Possibilité de poursuite en thèse

Durée du contrat (en mois), * Challenge these methods by applying them to new contexts (e.g. aerial images, medical imaging);

• If relevant, submit your contributions to an international conference or workshop for publication., Students in their 5th year of studies (M2)
• Computer vision skills
• Machine learning skills (deep learning, VLM…)
• Python proficiency in a deep learning framework (especially PyTorch)

Entité de rattachement Le CEA est un acteur majeur de la recherche, au service des citoyens, de l'économie et de l'Etat. Il apporte des solutions concrètes à leurs besoins dans quatre domaines principaux : transition énergétique, transition numérique, technologies pour la médecine du futur, défense et sécurité sur un socle de recherche fondamentale. Le CEA s'engage depuis plus de 75 ans au service de la souveraineté scientifique, technologique et industrielle de la France et de l'Europe pour un présent et un avenir mieux maîtrisés et plus sûrs. Implanté au cœur des territoires équipés de très grandes infrastructures de recherche, le CEA dispose d'un large éventail de partenaires académiques et industriels en France, en Europe et à l'international. Les 20 000 collaboratrices et collaborateurs du CEA partagent trois valeurs fondamentales, * Work in one of the most innovative research organizations in the world, addressing societal challenges to build the world of tomorrow * Discover a rich ecosystem: privileged connections between the industrial and academic sectors * Conduct research autonomously and creatively: encouragement to valorize results (scientific articles, patents, open-source codes...) * Join a young and dynamic team * Benefit from an internal computing infrastructure with more than 300 state-of-the-art GPUs * Receive a stipend between €1300 and €1400 per month * Have the opportunity to continue with a PhD or as a research engineer after the internship * Have the possibility of remote work, receive a 75% reimbursement on public transportation costs, and benefit from the "mobili-jeune" aid to reduce rent costs…