Zaid Zaim

How Mixed Reality, Azure AI and Drones turned me into a Magician?

You don't need magic to command a physical drone with your hands. Just mixed reality, Azure AI, and a bit of code.

How Mixed Reality, Azure AI and Drones turned me into a Magician?
#1about 4 minutes

Fusing mixed reality, AI, and drones for magical experiences

The talk introduces a project that combines Azure AI, drones, and mixed reality to create immersive, interactive experiences.

#2about 2 minutes

Exploring the core technology stack for the project

An overview of the key technologies used, including the Unity game engine, Meta Quest 3 for mixed reality, and Azure AI Foundry for intelligence.

#3about 1 minute

Getting started with the Azure AI Foundry platform

Azure AI Foundry is presented as a centralized platform for accessing various AI models, developer tools, and services to build generative AI applications.

#4about 4 minutes

A practical walkthrough of the Azure AI Foundry playground

A live demonstration shows how to create a project, deploy a GPT-4o model, and use the chat playground to test the AI before integration.

#5about 4 minutes

Using Unity and the Meta SDK for XR development

The process of building the mixed reality application involves using the Meta SDK in Unity for camera rigs, controller systems, and crucial hand tracking capabilities.

#6about 11 minutes

Programming drone controls with GitHub Copilot

The Tello drone's UDP-based communication protocol is implemented by feeding its technical architecture to GitHub Copilot, which generates the necessary C# code.

#7about 3 minutes

Live demo of the immersive AI assistant Anna

A live demonstration showcases the AI assistant "Anna" in mixed reality, featuring natural language conversation, image recognition, and function calling to manipulate virtual objects.

#8about 4 minutes

Flying a real drone using hand gestures in MR

The final demo shows a real Tello drone being controlled directly through hand movements tracked by the Meta Quest 3 headset, including takeoff and flips.

Related jobs
Jobs that call for the skills explored in this talk.

test

Milly
Vienna, Austria

Intermediate

test

Milly
Vienna, Austria

Intermediate

Matching moments

Introducing the concept of an immersive AI copilot
00:45 MIN

Introducing the concept of an immersive AI copilot

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Combining spatial computing and drones for new interactions
00:55 MIN

Combining spatial computing and drones for new interactions

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Live demo of the AI copilot on a HoloLens device
14:30 MIN

Live demo of the AI copilot on a HoloLens device

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Live demo in a shared Microsoft Mesh environment
20:18 MIN

Live demo in a shared Microsoft Mesh environment

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

The core technologies for building an immersive copilot
06:43 MIN

The core technologies for building an immersive copilot

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Exploring the tools for drone and MR development
04:40 MIN

Exploring the tools for drone and MR development

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Demonstrating drone control with the HoloLens
27:54 MIN

Demonstrating drone control with the HoloLens

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Creating an intelligent cockpit experience with AI and AR
16:05 MIN

Creating an intelligent cockpit experience with AI and AR

Software defines the vehicle: Why customers and developers will love cars even more

Featured Partners

Related Videos

See all videos
Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!31:50

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!38:28

Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!

Zaid Zaim

Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact23:33

Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact

Oliver Will

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure26:35

Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure

Ricardo

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

Unleashing a New Reality: The Transformative Power of XR Technologies for Developers23:03

Unleashing a New Reality: The Transformative Power of XR Technologies for Developers

Bastian Schütz

Best practices: Building Enterprise Applications that leverage GenAI24:24

Best practices: Building Enterprise Applications that leverage GenAI

Damir

From Cultural Heritage to Partying in the MetaVerse33:57

From Cultural Heritage to Partying in the MetaVerse

Zaid Zaim

Related Articles

View all articles
CH
Chris Heilmann
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
Join Scott Hanselman at WWC24 to explore AI's role as a superhero or supervillain. Scott shares his 32 years of experience in software engineering, discusses AI myths, ethical dilemmas, and tech advancements. Engage with his live demos and insights o...
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
EM
Eli McGarvie
13 AI Tools You Have to Try
First, it was NFTs, then it was Web3, and now it’s generative AI… it’s probably time to stop collecting pictures of monkeys and kitties. Chatbots and generative AI are the next big thing. This time we’ve jumped on a trend that has real-world applicat...
13 AI Tools You Have to Try
DC
Daniel Cranney
Stephan Gillich - Bringing AI Everywhere
In the ever-evolving world of technology, AI continues to be the frontier for innovation and transformation. Stephan Gillich, from the AI Center of Excellence at Intel, dove into the subject in a recent session titled "Bringing AI Everywhere," sheddi...
Stephan Gillich - Bringing AI Everywhere

From learning to earning

Jobs that call for the skills explored in this talk.