Zaid Zaim
Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!
#1about 3 minutes
Introducing the concept of an immersive AI copilot
The talk introduces building an interactive AI character inspired by Disney's WALL-E using immersive technologies and generative AI.
#2about 3 minutes
Using Microsoft Mesh for collaborative XR experiences
Microsoft Mesh integrates into Microsoft Teams to provide a platform for immersive meetings, trainings, and social events.
#3about 1 minute
The core technologies for building an immersive copilot
The project is built using three main components: the Unity game engine, the Mesh Toolkit for deployment, and Azure OpenAI for intelligence.
#4about 5 minutes
Integrating Azure OpenAI service with Unity via REST API
A C# script in Unity calls the Azure OpenAI REST API to handle communication, parse the JSON response, and manage text-to-speech output.
#5about 2 minutes
Using the Mesh Toolkit for cross-platform deployment
The Mesh Toolkit provides shaders, visual scripting, and an optimizer to prepare and deploy the Unity environment to the Mesh platform.
#6about 6 minutes
Live demo of the AI copilot on a HoloLens device
A live demonstration shows interaction with the AI character "Anna" using voice commands on a Microsoft HoloLens.
#7about 6 minutes
Live demo in a shared Microsoft Mesh environment
The AI copilot is demonstrated within a shared, multi-user Microsoft Mesh space, showing collaborative interaction with the character.
#8about 6 minutes
Integrating immersive spaces directly into Microsoft Teams
The final demo showcases how users can join a Microsoft Mesh immersive space directly from a standard Microsoft Teams meeting.
Related jobs
Jobs that call for the skills explored in this talk.
Featured Partners
Related Videos
35:26How Mixed Reality, Azure AI and Drones turned me into a Magician?
Zaid Zaim
38:28Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
Zaid Zaim
31:39Livecoding with AI
Rainer Stropek
23:33Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact
Oliver Will
1:09:49Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More
Simi Olabisi
1:12:04WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney
24:24Best practices: Building Enterprise Applications that leverage GenAI
Damir
26:35Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure
Ricardo
From learning to earning
Jobs that call for the skills explored in this talk.
Senior Backend Engineer – AI Integration (m/w/x)
chatlyn GmbH
Vienna, Austria
Senior
JavaScript
AI-assisted coding tools
Microsoft Copilot Studio & Azure AI Foundry Developer- Remote
FDM Group
Charing Cross, United Kingdom
Remote
Senior
Azure
Python
TypeScript
Continuous Integration
Full-Stack Engineer - AI Agentic Systems
autonomous-teaming
Potsdam, Germany
Remote
Linux
Redis
React
Python
+7
Deep Learning/AI Engineer (Europe - Remote)
Zimperium
Municipality of Madrid, Spain
Remote
Keras
Python
PyTorch
TensorFlow
+2