WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2023 • Sep 27, 2023

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

Move beyond a printed hello world and control a physical drone. Master mixed reality by combining Unity 3D, .NET, and ChatGPT to build spatial computing apps.

Pause
Mute Enter Fullscreen
#1 about 5 min

Introduction to coding hardware with mixed reality

Integrating mixed reality headsets with programmable drones provides tangible physical feedback for initial coding experiments.

#2 about 3 min

Development tools for spatial computing and drones

Unity 3D, the MRTK framework, and generative language models serve as the technical foundation for spatial applications.

#3 about 3 min

Telemetry and hardware capabilities of the Tello

The Tello provides telemetry data and autonomous flight capabilities similar to an airborne IoT device.

#4 about 4 min

Understanding boundaries between augmented and virtual realities

Differentiating environmental overlays from fully immersive simulations clarifies the specific requirements of true spatial computing.

#5 about 6 min

Enterprise capabilities and transparent displays of HoloLens

Unobstructed physical vision and hands-free interaction models support remote engineering diagnostics and medical operations.

#6 about 3 min

Sending network commands via the Tello UDP

Unreliable datagram protocols negotiate specific network ports for sending programmatic maneuvers and receiving state telemetry.

#7 about 2 min

Scaffolding initial UDP network connections via LLMs

Generative text interfaces provide multi-threaded baseline functions for binding ports and encoding datagram messages.

#8 about 5 min

Binding spatial input scripts using Unity components

Custom drone handler dependencies translate simulated virtual hands into UDP network transmissions within the game engine.

#9 about 7 min

Live mixed reality flight execution and control

Deploying spatial interfaces connects the physical hardware to holographic views for real-time directional manipulation.

#10 about 2 min

Testing interactive virtual menus in outdoor deployment

Selecting actions from floating holographic panels translates into complex synchronized drone movements across real physical spaces.

#11 about 3 min

Technical trade-offs between game engines and headsets

Alternative mixed reality devices, engine optimizations, and environmental awareness systems differentiate the available spatial computing solutions.

Matching moments

5:48 min

Demonstrating holographic artificial intelligence copilot live interactions

Zaid Zaim Zaid Zaim · World Congress 2024

56 sec

Core technology stack for mixed reality copilots

Zaid Zaim Zaid Zaim · World Congress 2024

2:39 min

Transitioning toward immersive spatial computing technologies

Bastian Schütz Bastian Schütz · World Congress 2024

2:31 min

Outlining the hardware and software architecture of mixed reality

Nahed Hatahet Nahed Hatahet · World Congress 2024

4:24 min

Exploring the capabilities of mixed reality headsets

Bastian Schütz Bastian Schütz · World Congress 2024

1:48 min

Role of immersive technologies in AI copilots

Zaid Zaim Zaid Zaim · World Congress 2024

More from World Congress 2023

The year 3000, a brief history of Web Development thumbnail
25:30
over 2 years ago

The year 3000, a brief history of Web Development

Lorenzo Pieri

Demografie nutzen um jedem Bewerber das passende Angebot zu machen! thumbnail
8:52
almost 3 years ago

Demografie nutzen um jedem Bewerber das passende Angebot zu machen!

Nathalie Gierth

Guided Memory Management: Rust's Ownership Model thumbnail
0:18
almost 3 years ago

Guided Memory Management: Rust's Ownership Model

Stefan Baumgartner

Passwordless future: WebAuthn and Passkeys in practice thumbnail
32:32
almost 3 years ago

Passwordless future: WebAuthn and Passkeys in practice

Clemens Hübner

External Secrets Operator: the secrets management toolbox for self-sufficient teams thumbnail
30:07
almost 3 years ago

External Secrets Operator: the secrets management toolbox for self-sufficient teams

Moritz Johner

Building Reliable Serverless Applications with AWS CDK and Testing thumbnail
25:18
almost 3 years ago

Building Reliable Serverless Applications with AWS CDK and Testing

Raphael Manke

Related videos

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI! thumbnail
31:50
World Congress 2024 • about 2 years ago

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Metaverse, the workplace of the future - Holoportation, the next generation of communication thumbnail
29:43
World Congress 2024 • about 2 years ago

Metaverse, the workplace of the future - Holoportation, the next generation of communication

Nahed Hatahet

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney thumbnail
1:12:04
over 1 year ago

WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney

Unleashing a New Reality: The Transformative Power of XR Technologies for Developers thumbnail
23:03
World Congress 2024 • about 2 years ago

Unleashing a New Reality: The Transformative Power of XR Technologies for Developers

Bastian Schütz

Live Coding an Augmented Reality Game with WebXR thumbnail
44:30
World Congress 2021 • about 5 years ago

Live Coding an Augmented Reality Game with WebXR

Christian Grail

Exploring the Potential of Decentraland thumbnail
56:19
LIVE • over 3 years ago

Exploring the Potential of Decentraland

Christian Glessner

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.