WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Nov 17, 2023

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

Task-specific neural networks are rapidly becoming obsolete. Master the transformer architectures, latent diffusion models, and security guardrails needed to build the next generation of multimodal AI.

Pause
Mute Enter Fullscreen
#1 about 2 min

Adapting to the generative artificial intelligence era

Transitioning from manual parameter tuning to leveraging generative artificial intelligence drastically accelerates the traditional data science workflow.

#2 about 2 min

Shifting towards multimodal large language models

Moving away from specialized neural network architectures to multimodal large language models establishes a sustainable path toward artificial general intelligence.

#3 about 2 min

Evaluating zero-shot capabilities in large foundation models

Pre-trained transformer architectures achieve high zero-shot performance without requiring targeted dataset retraining.

#4 about 2 min

Enhancing reasoning and visual understanding with multimodality

Adding vision to language models enables complex contextual understanding and helps models reason accurately about visual humor.

#5 about 4 min

Transforming industry workflows with multimodal automation

Implementing multimodal generative tools across industries powers advanced capabilities like personalized education and responsive computer-aided design.

#6 about 3 min

Powering interactive non-playable characters with generative speech

Integrating generative speech models directly into gaming engines produces interactive non-playable characters capable of dynamic conversation.

#7 about 2 min

Generating interactive 3D floor plans using language prompts

Pairing cloud-based simulation engines with text prompts automates the generation of complex structural floor plans.

#8 about 3 min

Defining foundation models and next word prediction

Training foundation models on internet-scale multimodal data empowers transformer architectures to predict subsequent sequences accurately.

#9 about 7 min

Unpacking encoder and decoder mechanisms in transformers

Extracting semantic meaning requires transforming text into numerical embeddings and processing sequential relationships using attention layers.

#10 about 4 min

Differentiating text generation from image diffusion models

Generating artistic visuals necessitates leveraging image diffusion patterns rather than sequential linguistic prediction processes.

#11 about 2 min

Fine-tuning image generation models with specific references

Extracting specific artistic styles without massive compute requires training textual inversion techniques on small curated datasets.

#12 about 2 min

Tracing the evolution of generative image architectures

Studying the progression from early human portrait synthesis highlights the steady advancement toward complex video generation.

#13 about 4 min

Leveraging U-Net architectures for iterative noise prediction

Gradually subtracting predicted random noise from masked inputs directs models to uncover highly realistic visual data.

#14 about 2 min

Expediting training processes using latent space compression

Compressing high-resolution images into smaller dimensional representations via autoencoders drastically reduces computational overhead during latent diffusion.

#15 about 3 min

Directing image generation using contrastive language image pre-training

Mapping distinct modalities into a universally shared embedding space guarantees that generated outputs accurately reflect textual instructions.

#16 about 4 min

Iterating concepts with text-guided image-to-image editing

Modifying conceptual sketches iteratively through text-guided inpainting helps eliminate inherent data biases from generated visual scenes.

#17 about 3 min

Establishing core machine learning fundamentals and robust guardrails

Addressing commercial vulnerability requires establishing robust data guardrails and ensuring absolute transparency regarding source training models.

#18 about 3 min

Addressing future multimodal capabilities and ethical attribution models

Scaling generative applications commercially demands deploying strict mitigation strategies to handle algorithmic hallucinations and proper copyright attribution.

#19 about 3 min

Selecting framework languages and building AI portfolios

Selecting frameworks strategically ensures that developer teams can rapidly prototype neural networks while remaining capable of deploying code to embedded hardware.

#20 about 3 min

Running lightweight large language models on local hardware

Bypassing enterprise server costs involves deploying optimized seven-billion parameter language models locally on consumer-grade graphics processing units.

#21 about 3 min

Harnessing open-source community support and edge device processing

Opting into popular open-source communities secures vital developmental support while facilitating deep learning experiments on embedded endpoints.

#22 about 4 min

Combining distinct processing modalities to elevate network intelligence

Engineering custom generative solutions rather than relying on generic datasets builds competitive applicant portfolios that distinctly showcase advanced multimodal integration skills.

Matching moments

1:01 min

Understanding foundation models and generative AI capabilities

Timo Salm Timo Salm · WWC 2025

1:29 min

Core mechanics and multimodal capabilities of generative models

Duan Lightfoot Duan Lightfoot · WWC 2024

59 sec

Creating automated generative media and artificial intelligence products

Nico Axtmann · WWC 2022

5:05 min

Exploring popular generative AI models and applications

Mary Grygleski Mary Grygleski · LIVE

2:04 min

Exploring breakthroughs in large generalist machine learning models

Toju Duke · WWC 2022

1:37 min

Exploring common use cases for modern generative AI

Sandra Ahlgrimm Sandra Ahlgrimm +1 · WWC 2024

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Building Stuff with GenAI - The Open Minded Workshop beyond OpenAI

Andreas Erben

CTO for Applied AI and Metaverse at daenet

Andreas Erben
Open session

World Congress 2026 North America

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan
Open session

World Congress 2026 North America

No Single Model to Rule Them All: Building Resilient AI Agents Across Open & Closed LLMs

Emmanuel Acheampong

Senior Manager Developer Relations at Crusoe AI

Emmanuel Acheampong
Open session

World Congress 2026 North America

Understanding LLM Architectures: Inside the Design of Modern Models

Jofia Jose Prakash

Enterprise AI Architect at American Chemical Society

Jofia Jose Prakash
Open session

World Congress 2026 North America

Engineering the Pivot: How Creative Strategy Solves the Hard Problems of AI Accuracy and Scale

Shruti Tiwari

AI/ML product manager, Dell

Shruti Tiwari
Open session

World Congress 2026 North America

The Broken Rung: How AI is Rebuilding Software Development from the Ground Up

Tomislav Tipurić

Chief Technology Officer, Nephos

Tomislav Tipurić

Related videos

AI'll Be Back: Generative AI in Image, Video, and Audio Production thumbnail
29:54
WWC 2025 • 12 months ago

AI'll Be Back: Generative AI in Image, Video, and Audio Production

Anonymized By Request, Martin Foertsch, Thomas Endres

Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative thumbnail
29:27
WWC 2023 • almost 3 years ago

Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative

David Estevez

Make it simple, using generative AI to accelerate learning thumbnail
26:54
WWC 2024 • almost 2 years ago

Make it simple, using generative AI to accelerate learning

Duan Lightfoot

AI & Ethics thumbnail
28:27
WWC 2025 • 12 months ago

AI & Ethics

PJ Hagerty

How AI Models Get Smarter thumbnail
35:16
WWC 2025 • about 1 year ago

How AI Models Get Smarter

Ankit Patel

Inside the Mind of an LLM thumbnail
27:11
WWC 2025 • 12 months ago

Inside the Mind of an LLM

Emanuele Fabbiani

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.