Zaid Zaim

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

This drone has no formal API, just documentation. So we asked ChatGPT to write the C# code to fly it with a HoloLens.

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
#1about 4 minutes

Combining spatial computing and drones for new interactions

The speaker shares his vision for controlling a Tello drone with a HoloLens to create more engaging and exciting coding experiences.

#2about 3 minutes

Exploring the tools for drone and MR development

An overview of the key technologies used, including the Unity 3D game engine, ChatGPT for code inspiration, and the Mixed Reality Toolkit (MRTK) for HoloLens.

#3about 2 minutes

A closer look at the programmable Tello drone

The Tello drone is introduced as a lightweight, programmable device comparable to a flying Raspberry Pi, controllable via its SDK.

#4about 4 minutes

Understanding the spectrum of immersive technologies

A breakdown of the reality-virtuality continuum defines augmented reality (AR), virtual reality (VR), and how mixed reality (MR) blends the physical and digital worlds.

#5about 5 minutes

How the HoloLens enables hands-free enterprise solutions

The Microsoft HoloLens is presented as an enterprise-focused MR headset enabling natural, hands-free interaction for use cases like remote assistance and healthcare.

#6about 5 minutes

Leveraging ChatGPT to interpret documentation and write code

ChatGPT is used as a tool to understand the drone's documentation and generate initial C# code for UDP communication, including IP addresses and ports.

#7about 5 minutes

A walkthrough of the Unity project and UDP client

A look inside the Unity project and the C# script for the UDP client, which handles connecting to the drone, entering SDK mode, and sending commands.

#8about 8 minutes

Demonstrating drone control with the HoloLens

A live demo shows the Tello drone taking off, moving, and flipping, all controlled by commands from the mixed reality application on the HoloLens.

#9about 3 minutes

Answering questions about Unreal Engine and HoloLens

The speaker addresses audience questions on Unreal Engine as an alternative, the development status of HoloLens, and using mixed reality on mobile devices.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Fusing mixed reality, AI, and drones for magical experiences
03:32 MIN

Fusing mixed reality, AI, and drones for magical experiences

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Programming drone controls with GitHub Copilot
10:45 MIN

Programming drone controls with GitHub Copilot

How Mixed Reality, Azure AI and Drones turned me into a Magician?

The core technologies for building an immersive copilot
00:57 MIN

The core technologies for building an immersive copilot

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Flying a real drone using hand gestures in MR
04:17 MIN

Flying a real drone using hand gestures in MR

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Exploring the core technology stack for the project
01:51 MIN

Exploring the core technology stack for the project

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Introducing the concept of an immersive AI copilot
03:06 MIN

Introducing the concept of an immersive AI copilot

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

A practical walkthrough of the Azure AI Foundry playground
04:24 MIN

A practical walkthrough of the Azure AI Foundry playground

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Live demo in a shared Microsoft Mesh environment
05:51 MIN

Live demo in a shared Microsoft Mesh environment

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Featured Partners

Related Videos

See all videos
How Mixed Reality, Azure AI and Drones turned me into a Magician?
35:26

How Mixed Reality, Azure AI and Drones turned me into a Magician?

Zaid Zaim

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!
31:50

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Livecoding with AI
31:39

Livecoding with AI

Rainer Stropek

ChatGPT: Create a Presentation!
30:24

ChatGPT: Create a Presentation!

Markus Walker

From Syntax to Singularity: AI’s Impact on Developer Roles
58:06

From Syntax to Singularity: AI’s Impact on Developer Roles

Anna Fritsch-Weninger

AI in Africa: How can we bounce back?
26:12

AI in Africa: How can we bounce back?

Ayoub Alouane

Bringing the power of AI to your application.
32:26

Bringing the power of AI to your application.

Krzysztof Cieślak

GitHub Copilot Beyond the Basics - 10 Ways to Elevate Your Coding
29:35

GitHub Copilot Beyond the Basics - 10 Ways to Elevate Your Coding

Liuba Gonta & Yuliya Khadasevich

Related Articles

View all articles
Chris Heilmann
With AIs wide open - WeAreDevelopers at All Things Open 2025
Last week our VP of Developer Relations, Chris Heilmann, flew to Raleigh, North Carolina to present at All Things Open . An excellent event he had spoken at a few times in the past and this being the “Lucky 13” edition, he didn’t hesitate to come and...
With AIs wide open - WeAreDevelopers at All Things Open 2025

From learning to earning

Jobs that call for the skills explored in this talk.