Talk
Registration required!
December 14, 2022
12:00 pm
12:45 pm
(CET)

Building Large-scale, Localized Language Models: From Data Preparation to Production

Save your spot
Powered by
No items found.

About the session

Recent advances in natural language processing demonstrate the capability of large-scale language models (such as GPT-3) to solve a variety of NLP problems with zero shots shifting from supervised fine-tuning to prompt engineering/tuning. However, building large language models raises challenges on data preparation, training, and deployment. In addition, while the process is well-established for a few dominant languages such as English, its execution on localized languages remains limited. We'll give an overview of the end-to-end process for building large-scale language models, discuss the challenges of scaling, and describe some existing solutions for efficient data preparation, distributed training, model optimization, and distributed deployment. We'll use examples on localized languages such as French or Spanish using NVIDIA Nemo Megatron, a framework for training large NLP models optimized for SuperPOD hardware infrastructure.

About the speaker

Miguel Martínez & Meriem Bendris
Miguel Martínez & Meriem Bendris
Senior Deep Learning Data Scientist & Senior Solution Architect - AI at NVIDIA

Watch recording

Registration required!

Save your spot

14 Dec
,
12:00 pm
12:45 pm
(CET)
Save my spotSave my spotSave my spotSave my spot
Code of Conduct
WeAreDevelopers welcomes everyone and is dedicated to defending anybody from harassment, regardless of gender, gender identity, and expression, sexual orientation, disability, physical appearance, body size, race, age or religion.
Read more
Diversity & Inclusion
At the WeAreDevelopers Events we empower underrepresented groups by giving them the stage to share their knowledge and experiences. It is crucial for our international events to bring together the perspectives of people with different backgrounds.
Read more