WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2024 • Aug 29, 2024

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Ditch the 2D chat interface and build embodied AI assistants. Master the integration of Unity, .NET, and Azure OpenAI to deploy low-latency spatial copilots for Microsoft Mesh.

Pause
Mute Enter Fullscreen
#1 about 2 min

Role of immersive technologies in AI copilots

Combining game engines, spatial computing, and generative AI creates new ways to interact with digital assistants.

#2 about 1 min

Designing 3D avatars for virtual AI assistants

Drawing inspiration from pop culture characters establishes approachable and interactive virtual personas.

#3 about 4 min

Integrating Microsoft Mesh into Teams enterprise collaboration

Embedding virtual reality platforms inside established enterprise software transforms remote meetings and onboarding sessions.

#4 about 1 min

Core technology stack for mixed reality copilots

Relying on specialized game engines, deployment toolkits, and language models forms the foundation of spatial applications.

#5 about 2 min

Configuring and deploying Azure OpenAI cloud services

Setting up secure endpoints and APIs in cloud platforms guarantees reliable communication with large language models.

#6 about 2 min

Integrating OpenAI REST APIs within Unity environments

Parsing incoming JSON API responses inside a C# client efficiently passes model outputs into the spatial environment.

#7 about 2 min

Implementing text-to-speech functionality for virtual reality headsets

Running local voice SDKs on headset hardware accelerates AI response vocalization by circumventing cloud network round-trips.

#8 about 2 min

Optimizing 3D scenes using the Microsoft Mesh toolkit

Validating geometry and using specialized shaders ensure high-performance rendering before publishing scenes to spatial platforms.

#9 about 6 min

Demonstrating holographic artificial intelligence copilot live interactions

Querying spatial agents directly inside an augmented reality headset produces dynamic conversational capabilities for users.

#10 about 6 min

Exploring collaborative virtual reality environments in Microsoft Mesh

Hosting multiple remote users in customized 3D spaces facilitates engaging team interactions alongside embedded AI agents.

#11 about 6 min

Transitioning traditional Teams calls into 3D immersive spaces

Bridging 2D video meetings and 3D virtual settings provides hybrid participants with seamless alternative engagement strategies.

Matching moments

1:48 min

Improving human interaction models with AI avatars

3:16 min

Driving developer productivity with AI in automotive tech

Mike Butcher Mike Butcher +3 · World Congress 2024

1:44 min

Summarizing developer experience and artificial intelligence companions

Robert Hoffmann Robert Hoffmann +1 · LIVE

1:23 min

Developer adoption of integrated AI chat features

Julia Kasper · Coffee With Developers

1:24 min

Building 3D virtual event platforms with AI

6:20 min

Live mixed reality flight execution and control

Zaid Zaim Zaid Zaim · World Congress 2023

More from World Congress 2024

Beyond the Hype: Real-World AI Strategies Panel thumbnail
24:07
12 months ago

Beyond the Hype: Real-World AI Strategies Panel

Mike Butcher, Jürgen Müller, Katrin Lehmann, Tobias Regenfuss

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation thumbnail
21:17
almost 2 years ago

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation

Carl Lapierre

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging thumbnail
29:43
about 2 years ago

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging

Sue Quackenbush

Dapr in Practice thumbnail
24:08
about 2 years ago

Dapr in Practice

marcklefter

Minimising the Carbon Footprint of Workloads thumbnail
27:46
about 2 years ago

Minimising the Carbon Footprint of Workloads

Michael Mueller

Say goodbye to building boring APIs with Azure Data API Builder thumbnail
30:01
about 2 years ago

Say goodbye to building boring APIs with Azure Data API Builder

Sander ten Brinke

Related videos

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI! thumbnail
38:28
World Congress 2023 • almost 3 years ago

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney thumbnail
1:12:04
over 1 year ago

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More thumbnail
1:09:49
LIVE • over 2 years ago

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More

Simi Olabisi

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure thumbnail
26:35
World Congress 2025 • about 1 year ago

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure

Ricardo

Metaverse, the workplace of the future - Holoportation, the next generation of communication thumbnail
29:43
World Congress 2024 • about 2 years ago

Metaverse, the workplace of the future - Holoportation, the next generation of communication

Nahed Hatahet

Develop AI-powered Applications with OpenAI Embeddings and Azure Search thumbnail
57:52
LIVE • almost 3 years ago

Develop AI-powered Applications with OpenAI Embeddings and Azure Search

Rainer Stropek

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.