Breaking Language Barriers: GPU-Accelerated Corpus Curation for LLM Training/Customization

Creating large language models (LLMs) for non-English languages is frequently hindered by the scarcity or imbalance of available datasets. In this session, we introduce an advanced methodology for assembling high-quality, multilingual text corpora—spanning languages such as Spanish and French—by harnessing the computational power of GPU acceleration. We guide you through the corpus construction process, from foundational steps like text selection and cleaning to advanced approaches including semantic deduplication. The discussion will also highlight creative techniques for generating synthetic data, an essential measure for enriching datasets when authentic examples are limited.

Attendees will gain actionable knowledge on corpus curation workflows that can be adapted to a wide range of non-English languages, ultimately fostering the development of more inclusive and representative language technologies.