WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Nov 17, 2023

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

Task-specific neural networks are rapidly becoming obsolete. Master the transformer architectures, latent diffusion models, and security guardrails needed to build the next generation of multimodal AI.

Pause
Mute Enter Fullscreen
#1 about 2 min

Adapting to the generative artificial intelligence era

Transitioning from manual parameter tuning to leveraging generative artificial intelligence drastically accelerates the traditional data science workflow.

#2 about 2 min

Shifting towards multimodal large language models

Moving away from specialized neural network architectures to multimodal large language models establishes a sustainable path toward artificial general intelligence.

#3 about 2 min

Evaluating zero-shot capabilities in large foundation models

Pre-trained transformer architectures achieve high zero-shot performance without requiring targeted dataset retraining.

#4 about 2 min

Enhancing reasoning and visual understanding with multimodality

Adding vision to language models enables complex contextual understanding and helps models reason accurately about visual humor.

#5 about 4 min

Transforming industry workflows with multimodal automation

Implementing multimodal generative tools across industries powers advanced capabilities like personalized education and responsive computer-aided design.

#6 about 3 min

Powering interactive non-playable characters with generative speech

Integrating generative speech models directly into gaming engines produces interactive non-playable characters capable of dynamic conversation.

#7 about 2 min

Generating interactive 3D floor plans using language prompts

Pairing cloud-based simulation engines with text prompts automates the generation of complex structural floor plans.

#8 about 3 min

Defining foundation models and next word prediction

Training foundation models on internet-scale multimodal data empowers transformer architectures to predict subsequent sequences accurately.

#9 about 7 min

Unpacking encoder and decoder mechanisms in transformers

Extracting semantic meaning requires transforming text into numerical embeddings and processing sequential relationships using attention layers.

#10 about 4 min

Differentiating text generation from image diffusion models

Generating artistic visuals necessitates leveraging image diffusion patterns rather than sequential linguistic prediction processes.

#11 about 2 min

Fine-tuning image generation models with specific references

Extracting specific artistic styles without massive compute requires training textual inversion techniques on small curated datasets.

#12 about 2 min

Tracing the evolution of generative image architectures

Studying the progression from early human portrait synthesis highlights the steady advancement toward complex video generation.

#13 about 4 min

Leveraging U-Net architectures for iterative noise prediction

Gradually subtracting predicted random noise from masked inputs directs models to uncover highly realistic visual data.

#14 about 2 min

Expediting training processes using latent space compression

Compressing high-resolution images into smaller dimensional representations via autoencoders drastically reduces computational overhead during latent diffusion.

#15 about 3 min

Directing image generation using contrastive language image pre-training

Mapping distinct modalities into a universally shared embedding space guarantees that generated outputs accurately reflect textual instructions.

#16 about 4 min

Iterating concepts with text-guided image-to-image editing

Modifying conceptual sketches iteratively through text-guided inpainting helps eliminate inherent data biases from generated visual scenes.

#17 about 3 min

Establishing core machine learning fundamentals and robust guardrails

Addressing commercial vulnerability requires establishing robust data guardrails and ensuring absolute transparency regarding source training models.

#18 about 3 min

Addressing future multimodal capabilities and ethical attribution models

Scaling generative applications commercially demands deploying strict mitigation strategies to handle algorithmic hallucinations and proper copyright attribution.

#19 about 3 min

Selecting framework languages and building AI portfolios

Selecting frameworks strategically ensures that developer teams can rapidly prototype neural networks while remaining capable of deploying code to embedded hardware.

#20 about 3 min

Running lightweight large language models on local hardware

Bypassing enterprise server costs involves deploying optimized seven-billion parameter language models locally on consumer-grade graphics processing units.

#21 about 3 min

Harnessing open-source community support and edge device processing

Opting into popular open-source communities secures vital developmental support while facilitating deep learning experiments on embedded endpoints.

#22 about 4 min

Combining distinct processing modalities to elevate network intelligence

Engineering custom generative solutions rather than relying on generic datasets builds competitive applicant portfolios that distinctly showcase advanced multimodal integration skills.

Matching moments

1:01 min

Understanding foundation models and generative AI capabilities

Timo Salm Timo Salm · World Congress 2025

1:29 min

Core mechanics and multimodal capabilities of generative models

Duan Lightfoot Duan Lightfoot · World Congress 2024

59 sec

Creating automated generative media and artificial intelligence products

Nico Axtmann · World Congress 2022

5:05 min

Exploring popular generative AI models and applications

Mary Grygleski Mary Grygleski · LIVE

2:04 min

Exploring breakthroughs in large generalist machine learning models

Toju Duke · World Congress 2022

1:37 min

Exploring common use cases for modern generative AI

Sandra Ahlgrimm Sandra Ahlgrimm +1 · World Congress 2024

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
11 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
18 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
19 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
25 days ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude thumbnail
6:16
26 days ago

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude

Chris Heilmann, Daniel Cranney

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas thumbnail
5:46
about 1 month ago

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas

Chris Heilmann, Daniel Cranney, Marc Plogas

Related videos

AI'll Be Back: Generative AI in Image, Video, and Audio Production thumbnail
29:54
World Congress 2025 • about 1 year ago

AI'll Be Back: Generative AI in Image, Video, and Audio Production

Anonymized By Request, Martin Foertsch, Thomas Endres

Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative thumbnail
29:27
World Congress 2023 • almost 3 years ago

Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative

David Estevez

Make it simple, using generative AI to accelerate learning thumbnail
26:54
World Congress 2024 • about 2 years ago

Make it simple, using generative AI to accelerate learning

Duan Lightfoot

AI & Ethics thumbnail
28:27
World Congress 2025 • about 1 year ago

AI & Ethics

PJ Hagerty

How AI Models Get Smarter thumbnail
35:16
World Congress 2025 • about 1 year ago

How AI Models Get Smarter

Ankit Patel

Inside the Mind of an LLM thumbnail
27:11
World Congress 2025 • about 1 year ago

Inside the Mind of an LLM

Emanuele Fabbiani

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.