WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Nov 17, 2023

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

Task-specific neural networks are rapidly becoming obsolete. Master the transformer architectures, latent diffusion models, and security guardrails needed to build the next generation of multimodal AI.

Pause
Mute Enter Fullscreen
#1 about 2 min

Adapting to the generative artificial intelligence era

Transitioning from manual parameter tuning to leveraging generative artificial intelligence drastically accelerates the traditional data science workflow.

#2 about 2 min

Shifting towards multimodal large language models

Moving away from specialized neural network architectures to multimodal large language models establishes a sustainable path toward artificial general intelligence.

#3 about 2 min

Evaluating zero-shot capabilities in large foundation models

Pre-trained transformer architectures achieve high zero-shot performance without requiring targeted dataset retraining.

#4 about 2 min

Enhancing reasoning and visual understanding with multimodality

Adding vision to language models enables complex contextual understanding and helps models reason accurately about visual humor.

#5 about 4 min

Transforming industry workflows with multimodal automation

Implementing multimodal generative tools across industries powers advanced capabilities like personalized education and responsive computer-aided design.

#6 about 3 min

Powering interactive non-playable characters with generative speech

Integrating generative speech models directly into gaming engines produces interactive non-playable characters capable of dynamic conversation.

#7 about 2 min

Generating interactive 3D floor plans using language prompts

Pairing cloud-based simulation engines with text prompts automates the generation of complex structural floor plans.

#8 about 3 min

Defining foundation models and next word prediction

Training foundation models on internet-scale multimodal data empowers transformer architectures to predict subsequent sequences accurately.

#9 about 7 min

Unpacking encoder and decoder mechanisms in transformers

Extracting semantic meaning requires transforming text into numerical embeddings and processing sequential relationships using attention layers.

#10 about 4 min

Differentiating text generation from image diffusion models

Generating artistic visuals necessitates leveraging image diffusion patterns rather than sequential linguistic prediction processes.

#11 about 2 min

Fine-tuning image generation models with specific references

Extracting specific artistic styles without massive compute requires training textual inversion techniques on small curated datasets.

#12 about 2 min

Tracing the evolution of generative image architectures

Studying the progression from early human portrait synthesis highlights the steady advancement toward complex video generation.

#13 about 4 min

Leveraging U-Net architectures for iterative noise prediction

Gradually subtracting predicted random noise from masked inputs directs models to uncover highly realistic visual data.

#14 about 2 min

Expediting training processes using latent space compression

Compressing high-resolution images into smaller dimensional representations via autoencoders drastically reduces computational overhead during latent diffusion.

#15 about 3 min

Directing image generation using contrastive language image pre-training

Mapping distinct modalities into a universally shared embedding space guarantees that generated outputs accurately reflect textual instructions.

#16 about 4 min

Iterating concepts with text-guided image-to-image editing

Modifying conceptual sketches iteratively through text-guided inpainting helps eliminate inherent data biases from generated visual scenes.

#17 about 3 min

Establishing core machine learning fundamentals and robust guardrails

Addressing commercial vulnerability requires establishing robust data guardrails and ensuring absolute transparency regarding source training models.

#18 about 3 min

Addressing future multimodal capabilities and ethical attribution models

Scaling generative applications commercially demands deploying strict mitigation strategies to handle algorithmic hallucinations and proper copyright attribution.

#19 about 3 min

Selecting framework languages and building AI portfolios

Selecting frameworks strategically ensures that developer teams can rapidly prototype neural networks while remaining capable of deploying code to embedded hardware.

#20 about 3 min

Running lightweight large language models on local hardware

Bypassing enterprise server costs involves deploying optimized seven-billion parameter language models locally on consumer-grade graphics processing units.

#21 about 3 min

Harnessing open-source community support and edge device processing

Opting into popular open-source communities secures vital developmental support while facilitating deep learning experiments on embedded endpoints.

#22 about 4 min

Combining distinct processing modalities to elevate network intelligence

Engineering custom generative solutions rather than relying on generic datasets builds competitive applicant portfolios that distinctly showcase advanced multimodal integration skills.

Matching moments

1:01 min

Understanding foundation models and generative AI capabilities

Timo Salm Timo Salm · World Congress 2025

1:29 min

Core mechanics and multimodal capabilities of generative models

Duan Lightfoot Duan Lightfoot · World Congress 2024

59 sec

Creating automated generative media and artificial intelligence products

Nico Axtmann · World Congress 2022

5:05 min

Exploring popular generative AI models and applications

Mary Grygleski Mary Grygleski · LIVE

2:04 min

Exploring breakthroughs in large generalist machine learning models

Toju Duke · World Congress 2022

1:37 min

Exploring common use cases for modern generative AI

Sandra Ahlgrimm Sandra Ahlgrimm +1 · World Congress 2024

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Tech Education in Two Languages thumbnail
1:00:18
2 days ago

WeAreDevelopers LIVE - Tech Education in Two Languages

Daniel Cranney, Roxy Rodriguez-Becker

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
9 days ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
23 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
about 1 month ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
about 1 month ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
about 1 month ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Upcoming sessions on this topic

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

From Prompt Engineering to Harness Engineering: Building Observable and Governed GenAI Systems

  • Lakshmi Devi Prakash

    JPMorgan Chase

    VP, Applied AI/ML Lead

Open session

Beware of Strangers Bearing Code: Open Source Trust in the Agent Era

  • Vikram Vaswani

    Consultant

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

Building World-Aware Physical AI with Agent Memory and Context Graphs

  • Vasundhara Shukla

    Neo4j

    Developer Advocate

  • Zaid Zaim

    Neo4j

    Developer Advocate | Microsoft AI MVP

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Related videos

AI'll Be Back: Generative AI in Image, Video, and Audio Production thumbnail
29:54
World Congress 2025 • about 1 year ago

AI'll Be Back: Generative AI in Image, Video, and Audio Production

Anonymized By Request, Martin Foertsch, Thomas Endres

Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative thumbnail
29:27
World Congress 2023 • about 3 years ago

Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative

David Estevez

In the Dawn of the AI: Understanding and implementing AI-generated images thumbnail
30:38
World Congress 2022 • over 4 years ago

In the Dawn of the AI: Understanding and implementing AI-generated images

Timo Zander

Make it simple, using generative AI to accelerate learning thumbnail
26:54
World Congress 2024 • about 2 years ago

Make it simple, using generative AI to accelerate learning

Duan Lightfoot

AI & Ethics thumbnail
28:27
World Congress 2025 • about 1 year ago

AI & Ethics

PJ Hagerty

When worlds collide: How will generative AI change the way we design and build software thumbnail
44:00
World Congress 2023 • about 3 years ago

When worlds collide: How will generative AI change the way we design and build software

Jonas Andrulis, Mike Butcher

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.