WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2023 • Sep 27, 2023

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

Move beyond a printed hello world and control a physical drone. Master mixed reality by combining Unity 3D, .NET, and ChatGPT to build spatial computing apps.

Pause
Mute Enter Fullscreen
#1 about 5 min

Introduction to coding hardware with mixed reality

Integrating mixed reality headsets with programmable drones provides tangible physical feedback for initial coding experiments.

#2 about 3 min

Development tools for spatial computing and drones

Unity 3D, the MRTK framework, and generative language models serve as the technical foundation for spatial applications.

#3 about 3 min

Telemetry and hardware capabilities of the Tello

The Tello provides telemetry data and autonomous flight capabilities similar to an airborne IoT device.

#4 about 4 min

Understanding boundaries between augmented and virtual realities

Differentiating environmental overlays from fully immersive simulations clarifies the specific requirements of true spatial computing.

#5 about 6 min

Enterprise capabilities and transparent displays of HoloLens

Unobstructed physical vision and hands-free interaction models support remote engineering diagnostics and medical operations.

#6 about 3 min

Sending network commands via the Tello UDP

Unreliable datagram protocols negotiate specific network ports for sending programmatic maneuvers and receiving state telemetry.

#7 about 2 min

Scaffolding initial UDP network connections via LLMs

Generative text interfaces provide multi-threaded baseline functions for binding ports and encoding datagram messages.

#8 about 5 min

Binding spatial input scripts using Unity components

Custom drone handler dependencies translate simulated virtual hands into UDP network transmissions within the game engine.

#9 about 7 min

Live mixed reality flight execution and control

Deploying spatial interfaces connects the physical hardware to holographic views for real-time directional manipulation.

#10 about 2 min

Testing interactive virtual menus in outdoor deployment

Selecting actions from floating holographic panels translates into complex synchronized drone movements across real physical spaces.

#11 about 3 min

Technical trade-offs between game engines and headsets

Alternative mixed reality devices, engine optimizations, and environmental awareness systems differentiate the available spatial computing solutions.

Matching moments

5:48 min

Demonstrating holographic artificial intelligence copilot live interactions

Zaid Zaim Zaid Zaim · WWC 2024

56 sec

Core technology stack for mixed reality copilots

Zaid Zaim Zaid Zaim · WWC 2024

2:39 min

Transitioning toward immersive spatial computing technologies

Bastian Schütz Bastian Schütz · WWC 2024

2:31 min

Outlining the hardware and software architecture of mixed reality

Nahed Hatahet Nahed Hatahet · WWC 2024

4:24 min

Exploring the capabilities of mixed reality headsets

Bastian Schütz Bastian Schütz · WWC 2024

1:48 min

Role of immersive technologies in AI copilots

Zaid Zaim Zaid Zaim · WWC 2024

More from World Congress 2023

The year 3000, a brief history of Web Development thumbnail
25:30
over 2 years ago

The year 3000, a brief history of Web Development

Lorenzo Pieri

Demografie nutzen um jedem Bewerber das passende Angebot zu machen! thumbnail
8:52
over 2 years ago

Demografie nutzen um jedem Bewerber das passende Angebot zu machen!

Nathalie Gierth

Guided Memory Management: Rust's Ownership Model thumbnail
0:18
almost 3 years ago

Guided Memory Management: Rust's Ownership Model

Stefan Baumgartner

Passwordless future: WebAuthn and Passkeys in practice thumbnail
32:32
almost 3 years ago

Passwordless future: WebAuthn and Passkeys in practice

Clemens Hübner

External Secrets Operator: the secrets management toolbox for self-sufficient teams thumbnail
30:07
almost 3 years ago

External Secrets Operator: the secrets management toolbox for self-sufficient teams

Moritz Johner

Building Reliable Serverless Applications with AWS CDK and Testing thumbnail
25:18
almost 3 years ago

Building Reliable Serverless Applications with AWS CDK and Testing

Raphael Manke

Upcoming sessions on this topic

Open session

World Congress 2026 North America

From Software Agents to Physical Devices: Inside the Agentic Hardware Stack

Vivian Hu, Michael Yuan

Vivian Hu
Michael Yuan
Open session

World Congress 2026 North America

Building Stuff with GenAI - The Open Minded Workshop beyond OpenAI

Andreas Erben

CTO for Applied AI and Metaverse at daenet

Andreas Erben
Open session

World Congress 2026 North America

Honey, look! I vibe-coded an OS!

Ian Smith

CTO, LYOS

Ian Smith
Open session

World Congress 2026 North America

Vibe Coding Accessibility

Karl Groves

Focused on actively fixing accessibility

Karl Groves
Open session

World Congress 2026 North America

Beyond Vibe Coding: Using CLI Tools as Your AI Counterpart in Pair Programming

Ron Veen

Java enthusiast and Special agent for Team Rockstars IT

Ron Veen
Open session

World Congress 2026 North America

Look What Java Can Do Now: Live-Coding a GenAI MCP Server with the JAQ Stack

Suren Konathala

Senior Engineering Leader, Marketing & AdTech, AI GTM & Digital Experience Platforms, Java x AI

Suren Konathala

Related videos

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI! thumbnail
31:50
WWC 2024 • almost 2 years ago

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Metaverse, the workplace of the future - Holoportation, the next generation of communication thumbnail
29:43
WWC 2024 • almost 2 years ago

Metaverse, the workplace of the future - Holoportation, the next generation of communication

Nahed Hatahet

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney thumbnail
1:12:04
over 1 year ago

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney

Unleashing a New Reality: The Transformative Power of XR Technologies for Developers thumbnail
23:03
WWC 2024 • almost 2 years ago

Unleashing a New Reality: The Transformative Power of XR Technologies for Developers

Bastian Schütz

Live Coding an Augmented Reality Game with WebXR thumbnail
44:30
WWC 2021 • about 5 years ago

Live Coding an Augmented Reality Game with WebXR

Christian Grail

Exploring the Potential of Decentraland thumbnail
56:19
LIVE • about 3 years ago

Exploring the Potential of Decentraland

Christian Glessner

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.