WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2024 • Aug 29, 2024

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Ditch the 2D chat interface and build embodied AI assistants. Master the integration of Unity, .NET, and Azure OpenAI to deploy low-latency spatial copilots for Microsoft Mesh.

Pause
Mute Enter Fullscreen
#1 about 2 min

Role of immersive technologies in AI copilots

Combining game engines, spatial computing, and generative AI creates new ways to interact with digital assistants.

#2 about 1 min

Designing 3D avatars for virtual AI assistants

Drawing inspiration from pop culture characters establishes approachable and interactive virtual personas.

#3 about 4 min

Integrating Microsoft Mesh into Teams enterprise collaboration

Embedding virtual reality platforms inside established enterprise software transforms remote meetings and onboarding sessions.

#4 about 1 min

Core technology stack for mixed reality copilots

Relying on specialized game engines, deployment toolkits, and language models forms the foundation of spatial applications.

#5 about 2 min

Configuring and deploying Azure OpenAI cloud services

Setting up secure endpoints and APIs in cloud platforms guarantees reliable communication with large language models.

#6 about 2 min

Integrating OpenAI REST APIs within Unity environments

Parsing incoming JSON API responses inside a C# client efficiently passes model outputs into the spatial environment.

#7 about 2 min

Implementing text-to-speech functionality for virtual reality headsets

Running local voice SDKs on headset hardware accelerates AI response vocalization by circumventing cloud network round-trips.

#8 about 2 min

Optimizing 3D scenes using the Microsoft Mesh toolkit

Validating geometry and using specialized shaders ensure high-performance rendering before publishing scenes to spatial platforms.

#9 about 6 min

Demonstrating holographic artificial intelligence copilot live interactions

Querying spatial agents directly inside an augmented reality headset produces dynamic conversational capabilities for users.

#10 about 6 min

Exploring collaborative virtual reality environments in Microsoft Mesh

Hosting multiple remote users in customized 3D spaces facilitates engaging team interactions alongside embedded AI agents.

#11 about 6 min

Transitioning traditional Teams calls into 3D immersive spaces

Bridging 2D video meetings and 3D virtual settings provides hybrid participants with seamless alternative engagement strategies.

Matching moments

1:48 min

Improving human interaction models with AI avatars

3:16 min

Driving developer productivity with AI in automotive tech

Mike Butcher Mike Butcher +3 · World Congress 2024

1:44 min

Summarizing developer experience and artificial intelligence companions

Robert Hoffmann Robert Hoffmann +1 · LIVE

1:23 min

Developer adoption of integrated AI chat features

Julia Kasper · Coffee With Developers

1:24 min

Building 3D virtual event platforms with AI

6:20 min

Live mixed reality flight execution and control

Zaid Zaim Zaid Zaim · World Congress 2023

More from World Congress 2024

Beyond the Hype: Real-World AI Strategies Panel thumbnail
24:07
11 months ago

Beyond the Hype: Real-World AI Strategies Panel

Mike Butcher, Jürgen Müller, Katrin Lehmann, Tobias Regenfuss

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation thumbnail
21:17
almost 2 years ago

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation

Carl Lapierre

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging thumbnail
29:43
about 2 years ago

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging

Sue Quackenbush

Dapr in Practice thumbnail
24:08
about 2 years ago

Dapr in Practice

marcklefter

Minimising the Carbon Footprint of Workloads thumbnail
27:46
about 2 years ago

Minimising the Carbon Footprint of Workloads

Michael Mueller

Say goodbye to building boring APIs with Azure Data API Builder thumbnail
30:01
about 2 years ago

Say goodbye to building boring APIs with Azure Data API Builder

Sander ten Brinke

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 25, 2026 · 10:20–10:50

Stage 5

Physical AI: 5 Things You Can Build That Aren't Another Chatbot

Vini Senger

Senior Technical Evangelist for Startups

Vini Senger
Open session

World Congress 2026 North America

September 23, 2026 · 10:00–17:00

Stage 11

Building Stuff with GenAI - The Open Minded Workshop beyond OpenAI

Andreas Erben

CTO for Applied AI and Metaverse at daenet

Andreas Erben
Open session

World Congress 2026 North America

September 25, 2026 · 12:30–14:30

Stage 14

Build a Human + AI Agent Team in tools like Slack, Teams and Discord.

Phil Conway

Developer Advocate at Flint AI (SandboxAQ)

Phil Conway
Open session

World Congress 2026 North America

September 23, 2026 · 13:40–14:10

Stage 2

Building an AI-Native Development Workflow

Vanessa Minik

Senior Service Delivery Engineer, GitHub

Vanessa Minik
Open session

World Congress 2026 North America

September 24, 2026 · 16:10–16:40

Stage 6

The Reality of AI Adoption in Enterprises

Priyanka Vergadia

Cloud and AI expert @Microsoft, previously lead Devrel @Google

Priyanka Vergadia
Open session

World Congress 2026 North America

September 24, 2026 · 11:40–12:10

Stage 1

Rise of the Agents: What’s Real, What’s Next, and What Matters

Mitesh Patel

NVIDIA Corporation, Developer Advocate -- Manager

Mitesh Patel

Related videos

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI! thumbnail
38:28
World Congress 2023 • almost 3 years ago

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney thumbnail
1:12:04
over 1 year ago

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More thumbnail
1:09:49
LIVE • over 2 years ago

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More

Simi Olabisi

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure thumbnail
26:35
World Congress 2025 • about 1 year ago

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure

Ricardo

Metaverse, the workplace of the future - Holoportation, the next generation of communication thumbnail
29:43
World Congress 2024 • about 2 years ago

Metaverse, the workplace of the future - Holoportation, the next generation of communication

Nahed Hatahet

Develop AI-powered Applications with OpenAI Embeddings and Azure Search thumbnail
57:52
LIVE • almost 3 years ago

Develop AI-powered Applications with OpenAI Embeddings and Azure Search

Rainer Stropek

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.