Playing with go, gobot & pion

Overview

Tello-WebRTC-FPV

Lately there were a couple of articles about pion and it looked very cool, but I didn't know what I could do with it. I then remembered that I had a Tello drone and that it could be managed with go, thanks to the wonderful gobot library.

I connected the two, and this project allows the drone to be remotely controlled from the browser - anyhwere in the world!

Overview

An http server is used to serve some HTML (the interface to control the drone) and transmit the WebRTC ICE offer & answer.

Once the webrtc communication is established, it is used to:

  • stream the video received from the drone
  • have a bi-directional data channel to receive the flight commands (takeoff, turn...) and send the flight data (battery, height...)

Demo (without a drone)

You can run the whole stack without a drone, by setting the MOCK env variable. It sends a pre-recorded video in loop and prints the received flight commands on stdout.

The interface is in german, but the keys should be layout independent (on a french keyboard, type Z instead of W for instance).

MOCK=true go run *.go

screenshot

Real flight

To fly for real, you need two connections: one (via Wifi) to the drone, and one to the internet (ethernet in my case). Once the drone is running, connect to it and then start the program (the program does not check if the drone is actually reachable - you may need to interrupt and relaunch it).

By default, the server will listen on port 3000 (adjust this with the PORT env variable) and anyone who can connect to your computer on this port should see the video and be able to send flight commands (even mutliple persons at the same time!). This (public) port is only used to send the HTML and exchange the WebRTC parameters.

From my own experience, the latency was very good (clearly under 1s).

And if the Tello has more than 50% of battery, it can do some flips (Shift+{A,W,S,D})

Standing on the shoulders of giants

This project would never have been possible without the great folks who created gobot and pion. Thank you so much!!

The only part where I had to do some actual work was regarding H264 streaming.

H264 Streaming

Since I didn't find any way to efficiently transcode the video stream from the drone, I simply forwarded the bytes to the browser. Somehow this worked on Firefox, but not on Chromium. After trying to tweak the parameters of pion and thinking that it was because of some strange codec parameters (which all seem different on the various browsers), I investigated the H264 frames further and found a real fix.

Thanks to https://yumichan.net/video-processing/video-compression/introduction-to-h264-nal-unit/, I was able to interpret the NAL unit frames coming from the Tello. After some trial-and-error, it appears that H264 frame sent via videoTrack.WriteSample(media.Sample{Data: frame, Samples: 1} should be:

  • one or more complete NAL units (which start with 0x00 00 00 01)
  • at least one of those NAL units must contain some frame information (type 1 or 5 in my case).

In the case of the Tello, it always send the following NAL types: 1 1 7 8 5 1 1 7 8 5... My code accumulates and sends: [1] [1] [7 8 5] [1] [1] [7 8 5]...

With this, I was able to have a working video stream on Firefox (the H264 codec must be active in the addons), Chrome and Safari!

Related work

  • TelloGo: program the steps of the drone from the browser

Have fun!

Example of using Pion WebRTC to play H264 + Ogg from disk

This repo demonstrates how you can use Pion WebRTC to play H264 and Ogg from disk. These same APIs can be used to pull from other sources. You can use

Sean DuBois 7 Sep 18, 2021
Demonstration of using Pion WebRTC with a shared socket

pion-webrtc-shared-socket This example demonstrates how Pion WebRTC can use an already listening UDP socket. On startup we listen on UDP Socket 8000.

Sean DuBois 6 Apr 4, 2022
This project is the eloboration of pion/webrtc.

This project is the eloboration of pion/webrtc. The idea is to make the (pion/webrtc) sfu-ws example be able to handle multiple rooms

null 0 Nov 29, 2021
A tool helps connect to your AriPods when sound playing in your Mac

Auto connect to airpods this tool helps connect to your AriPods when sound playing in your Mac. dependencenes SwitchAudioSource $ brew install switcha

Fndroid 2 Dec 9, 2021
Gobot - Golang framework for robotics, drones, and the Internet of Things (IoT)

Gobot (https://gobot.io/) is a framework using the Go programming language (https://golang.org/) for robotics, physical computing, and the Internet of Things.

The Hybrid Group 7.9k Aug 7, 2022
Gobot - A Telegeam bot example with go

Telegeam bot example Included saving users in postgres database Start Set your t

null 0 Jan 19, 2022
Demonstration of a browser connecting to Pion WebRTC without a signaling server.

Offline Browser Sync WebRTC without a signaling server! This repo demonstrates how you can connect two WebRTC proccesses without signaling. No configu

Pion 138 Jul 12, 2022
A yet to be voice call application in terminal. with the power of go and webRTC (pion).

Kenny I'm just trying to make a cli operated voice call chat application using go with help of webRTC and PortAudio. It might stay a Work In Progress

Mohammad Fatemi 37 Apr 30, 2022
A yet to be voice call application in terminal. with the power of go and webRTC (pion).

A yet to be voice call application in terminal. with the power of go and webRTC (pion).

Mohammad Fatemi 37 Apr 30, 2022
Example of using Pion WebRTC to play H264 + Ogg from disk

This repo demonstrates how you can use Pion WebRTC to play H264 and Ogg from disk. These same APIs can be used to pull from other sources. You can use

Sean DuBois 7 Sep 18, 2021
Demonstration of using Pion WebRTC with a shared socket

pion-webrtc-shared-socket This example demonstrates how Pion WebRTC can use an already listening UDP socket. On startup we listen on UDP Socket 8000.

Sean DuBois 6 Apr 4, 2022
This is a simple test application that sends fake video data from one pion instance to another

Pion test app for BWE This is a simple test application that sends fake video data from one pion instance to another. It is a modified version of the

Mathis Engelbart 2 Jun 8, 2022
This project is the eloboration of pion/webrtc.

This project is the eloboration of pion/webrtc. The idea is to make the (pion/webrtc) sfu-ws example be able to handle multiple rooms

null 0 Nov 29, 2021
A library for playing with colors in go (golang).

go-colorful A library for playing with colors in Go. Supports Go 1.13 onwards. Why? I love games. I make games. I love detail and I get lost in detail

Lucas Beyer 972 Jul 29, 2022
Cgo bindings to PulseAudio's Simple API, for easily playing or capturing raw audio.

pulse-simple Cgo bindings to PulseAudio's Simple API, for easily playing or capturing raw audio. The full Simple API is supported, including channel m

Tommy 20 Jun 17, 2022
A tool helps connect to your AriPods when sound playing in your Mac

Auto connect to airpods this tool helps connect to your AriPods when sound playing in your Mac. dependencenes SwitchAudioSource $ brew install switcha

Fndroid 2 Dec 9, 2021
A tui for playing media from a caddy fileserver

kwatch a little tui interface to play media from a caddy fileserver. options: -a: server address -u: server http username -p: server http password -o:

Rylee 5 Jan 10, 2022
66 is two player game played with playing cards and from now on you can play it from browser with your friends.

altmis-alti 66 is two player game played with playing cards and this project provides multiplayer game space from browser. How to run? Clone the proje

Cem Asma 1 Feb 1, 2022
Go-colorful: A library for playing with colors in golang

go-colorful A library for playing with colors in Go. Supports Go 1.13 onwards. Why? I love games. I make games. I love detail and I get lost in detail

Lucas Beyer 973 Aug 8, 2022
Just a dummy Kubernetes Operator, playing with another dummy service

My first operator Just playing/learning to create a K8S operator in go. I will create a dummy operator that creates pods to open a shell inside It is

Jose Gato Luis 0 Dec 16, 2021
A telegram bot to help remember tips about cards in your hand while playing Hanabi

?? Hanabi Assistant Bot ?? Hanabi Assistant is a telegram bot to help remember tips about cards in your hand while playing Hanabi. Before Run: Generat

Renan Gustavo 0 Oct 12, 2021
REST API to create and interact with decks of playing cards

Cards API REST API to create and interact with decks of playing cards. Tech Stac

null 1 Jul 14, 2022
Lightweight SQL database written in Go for prototyping and playing with text (CSV, JSON) data

gopicosql Lightweight SQL database written in Go for prototyping and playing wit

null 2 Jul 27, 2022
minimal now playing / music log

np === minimalistic 'now playing' music log. POST to add a song/album, GET to retreive log. usage ------ server: $ go build -o np main.go $ ./np -

null 0 Feb 22, 2022
Git-now-playing - Git commits are the new AIM status messages

git-now-playing git-now-playing is an attempt to bring some of the panache of th

Paddy 1 Apr 4, 2022
Spoti2wall: Set the Spotify album cover you are currently playing as your wallpaper

?? spoti2wall Set the Spotify album cover you are currently playing as your wall

Park Min Hyeok 5 Jul 16, 2022