WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2024 • Aug 29, 2024

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Ditch the 2D chat interface and build embodied AI assistants. Master the integration of Unity, .NET, and Azure OpenAI to deploy low-latency spatial copilots for Microsoft Mesh.

Pause
Mute Enter Fullscreen
#1 about 2 min

Role of immersive technologies in AI copilots

Combining game engines, spatial computing, and generative AI creates new ways to interact with digital assistants.

#2 about 1 min

Designing 3D avatars for virtual AI assistants

Drawing inspiration from pop culture characters establishes approachable and interactive virtual personas.

#3 about 4 min

Integrating Microsoft Mesh into Teams enterprise collaboration

Embedding virtual reality platforms inside established enterprise software transforms remote meetings and onboarding sessions.

#4 about 1 min

Core technology stack for mixed reality copilots

Relying on specialized game engines, deployment toolkits, and language models forms the foundation of spatial applications.

#5 about 2 min

Configuring and deploying Azure OpenAI cloud services

Setting up secure endpoints and APIs in cloud platforms guarantees reliable communication with large language models.

#6 about 2 min

Integrating OpenAI REST APIs within Unity environments

Parsing incoming JSON API responses inside a C# client efficiently passes model outputs into the spatial environment.

#7 about 2 min

Implementing text-to-speech functionality for virtual reality headsets

Running local voice SDKs on headset hardware accelerates AI response vocalization by circumventing cloud network round-trips.

#8 about 2 min

Optimizing 3D scenes using the Microsoft Mesh toolkit

Validating geometry and using specialized shaders ensure high-performance rendering before publishing scenes to spatial platforms.

#9 about 6 min

Demonstrating holographic artificial intelligence copilot live interactions

Querying spatial agents directly inside an augmented reality headset produces dynamic conversational capabilities for users.

#10 about 6 min

Exploring collaborative virtual reality environments in Microsoft Mesh

Hosting multiple remote users in customized 3D spaces facilitates engaging team interactions alongside embedded AI agents.

#11 about 6 min

Transitioning traditional Teams calls into 3D immersive spaces

Bridging 2D video meetings and 3D virtual settings provides hybrid participants with seamless alternative engagement strategies.

Matching moments

1:48 min

Improving human interaction models with AI avatars

3:16 min

Driving developer productivity with AI in automotive tech

Mike Butcher Mike Butcher +3 · WWC 2024

1:44 min

Summarizing developer experience and artificial intelligence companions

Robert Hoffmann Robert Hoffmann +1 · LIVE

6:20 min

Live mixed reality flight execution and control

Zaid Zaim Zaid Zaim · WWC 2023

3:01 min

Voice recognition integration for application code generation

2:39 min

Transitioning toward immersive spatial computing technologies

Bastian Schütz Bastian Schütz · WWC 2024

More from World Congress 2024

Beyond the Hype: Real-World AI Strategies Panel thumbnail
24:07
10 months ago

Beyond the Hype: Real-World AI Strategies Panel

Mike Butcher, Jürgen Müller, Katrin Lehmann, Tobias Regenfuss

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation thumbnail
21:17
almost 2 years ago

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation

Carl Lapierre

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging thumbnail
29:43
almost 2 years ago

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging

Sue Quackenbush

Dapr in Practice thumbnail
24:08
almost 2 years ago

Dapr in Practice

marcklefter

Minimising the Carbon Footprint of Workloads thumbnail
27:46
almost 2 years ago

Minimising the Carbon Footprint of Workloads

Michael Mueller

Say goodbye to building boring APIs with Azure Data API Builder thumbnail
30:01
almost 2 years ago

Say goodbye to building boring APIs with Azure Data API Builder

Sander ten Brinke

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Building Stuff with GenAI - The Open Minded Workshop beyond OpenAI

Andreas Erben

CTO for Applied AI and Metaverse at daenet

Andreas Erben
Open session

World Congress 2026 North America

From Software Agents to Physical Devices: Inside the Agentic Hardware Stack

Michael Yuan, Vivian Hu

Michael Yuan
Vivian Hu
Open session

World Congress 2026 North America

Beyond Vibe Coding: Using CLI Tools as Your AI Counterpart in Pair Programming

Ron Veen

Java enthusiast and Special agent for Team Rockstars IT

Ron Veen
Open session

World Congress 2026 North America

Beyond the Code: Human-AI Synergies in Product Development

Ajita Kanchivakam Ananth

Staff Technical Program Manager at Google

Ajita Kanchivakam Ananth
Open session

World Congress 2026 North America

GitHub’s Team X-Ray: Your Repository Knows More About Your Team Than Your Team Does

Andrea Griffiths

Senior Developer Advocate

Andrea Griffiths
Open session

World Congress 2026 North America

When Agents Became Users: Rearchitecting Identity and Permissions for AI at Scale

Dor Cohen, Yoav Gal

Dor Cohen
Yoav Gal

Related videos

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI! thumbnail
38:28
WWC 2023 • almost 3 years ago

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney thumbnail
1:12:04
over 1 year ago

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More thumbnail
1:09:49
LIVE • over 2 years ago

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More

Simi Olabisi

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure thumbnail
26:35
WWC 2025 • 12 months ago

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure

Ricardo

Metaverse, the workplace of the future - Holoportation, the next generation of communication thumbnail
29:43
WWC 2024 • almost 2 years ago

Metaverse, the workplace of the future - Holoportation, the next generation of communication

Nahed Hatahet

Develop AI-powered Applications with OpenAI Embeddings and Azure Search thumbnail
57:52
LIVE • over 2 years ago

Develop AI-powered Applications with OpenAI Embeddings and Azure Search

Rainer Stropek

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.