Eyes of Midgard: Vision of the drone
In the time of the old Gods, Odin, the Norse god of wisdom, sent his two ravens, Huginn and Muninn, to oversee Midgard, the realm of humans. Huginn, whose name means "thought," would gather information and knowledge, reporting back to Odin with his findings. Muninn, meaning "memory," would remember everything he witnessed, ensuring that no detail went unnoticed.
Thought and memory are two sides of the same coin that is called knowledge. Without memory, our thoughts would be fleeting and ephemeral, unable to contribute to our understanding of the world. Maybe that is the reason why the god-wanderer favorized Muninn. Anyway, thought and memory work in tandem, constantly influencing and shaping each other, ultimately leading to the acquisition and expansion of knowledge.
Shaking the feathers
OpenCV is a library of programming functions mainly for real-time computer vision. The library is cross-platform and licensed as free and open-source software under Apache License 2. I will use windows OS for this project. My project uses some part of this library, so let's install it.
First of all, visit this site: https://opencv.org/releases/ for the project I have used OpenCV – 4.7.0
Extract the opencv
Try to find system environment variables (use Control panel or the windows search bar)
On the System Properties window, click on "Environment Variables."
Choose "Path" and click on "Edit."
Add a new environment variable (yourPathtoOpenCvopencvbuildx64vc16bin)
Clone/download my project. Now you will experience some errors like this, we will explain to VS where to look for opencv
Choose the Properties
Set Additional Include Directories (yourPathtoOpenCvopencvbuildinclude
Set Additional Library Directories (yourPathtoOpenCvopencvbuildx64vc16lib
Set Additional Dependencies. Choose opencv_world470.lib from( yourPathtoOpenCvopencvbuildx64vc16lib)
1,2,3 Ready to fly
After you clone my project, you will have a few things in front of you.
In C++, the file extension ".hpp" typically refers to a C++ header file. Header files in C++ are used to declare the interface of a module or a set of related functions, classes, or variables. They typically include function prototypes, class declarations, constant definitions, and other declarations that need to be shared across multiple source files. By including /tello.hpp we can access all the functions that we need for our flight
In Resource Files you can find FaceDetection.xml which is Open Source Computer Vision Library made by Intel
In C++, the file extension ".cpp" typically refers to a source files that contain the actual implementation of functions, classes, and other constructs declared in header files. They include the details of how the code works. Inside the main.cpp file there is 4 methods and it will be explained how to use them
Let's explain the main:
Line 90: 5 seconds delay is introduced so that I can prepare my mobile phone and start recording.
Line 91: This line is creating an instance of a class named Tello. Tello can be found in tello.hpp and its instance will help us to record video and to fly.
Lines 93-98: we are checking if connection to our drone is ok (you should turn on the drone and connect to its wifi). If you are connected successfully, then we are enabling video stream.
Lines 100-111: In summary, this line initializes a VideoCapture object named capture to capture video frames from a UDP stream with specific parameters. The captured frames can then be processed using the OpenCV library. Additionally, there are two parameters appended with question mark ?: overrun_nonfatal=1 and fifo_size=580000. These parameters are specific to the video stream and may be used to control aspects such as buffer overrun handling (overrun_nonfatal) and the size of the FIFO (First In, First Out) buffer (fifo_size). Without those parameters you will get buffer errors, so make some space for your video :)
capture.set(cv::CAP_PROP_FPS, 30);defines the speed at which the video frames will be captured.
capture.set(cv::CAP_PROP_BUFFERSIZE, 10); The buffer size affects how many frames are buffered by the VideoCapture object.
Next thing is setting the width and height of the video and it will be used on here
cv::VideoWriter videoWriter("output_video.avi", cv::VideoWriter::fourcc('M', 'J', 'P', 'G'), 30, cv::Size(width, height));
on the line 111 we define name of the video that will be saved, codec of the video (Motion-JPEG codec), frames per second and we use alredy defined width and height. Strange part is that Tello original videos are in mp4 format by default, but unfortunately, opencv supports only avi format. Anyway, I was not bothered too much with that constatation.
Line113: We define time of the flight
Lines 115-121: Respecting the SRP principle I decided to separate function that will be associated with flight and the second one that will be in charge of recording the video.
Lines 125-139: When the video is recorded, you will need to write down the path of the output_video.avi (freshly saved video of the flight). The path will be the argument for analyzeStoredVideo function.
Huginn's journey
In the function moveTello, the Tello object tello is passed by reference (Tello& tello). Passing by reference allows the function to modify the original Tello object directly, rather than working with a copy of the object.
While VideoCapture and VideoWriter are passed by reference we have a const value of our flight tha will not be cahned and it is passed in this way for respecting the best practices of C++. Inside this function is a while loop that will be escaped when the elapsed time is >= than durationInSeconds. It is very straightforward, but only thing that was counterintuitive is calling the capture.grab() and capture.retrieve() methods. For some reason, during the flight it is better to call those methods instead of writing simply to the stream with capture >> frame;
Muninn's journey
In summary, this function records video frames from a cv::VideoCapture object to a cv::VideoWriter for a specified duration. It uses a loop to continuously capture and write frames, checking for errors and the elapsed time to determine when to stop recording.
The analyzeStoredVideo function has one parameter, the path to the video. It creates a VideoCapture object named capture and attempts to open the video file specified by videoPath. Additionally, it creates a CascadeClassifier object named faceCascade and loads a pre-trained XML file for face detection.
The loop in the function performs several tasks:
It enters into a loop that captures frames from the video stream until the end of the video.
For each frame, it uses the loaded CascadeClassifier to detect faces using detectMultiScale.
Detected faces are highlighted with rectangles drawn on the frame using cv::rectangle.
The processed frame is displayed using cv::imshow.
This is a small hommage to a god-wanderer on Stacksaga
A few thoughts for the end
Kudos to author who shared his projects:
For flying part I used his .hpp - https://github.com/HerrNamenlos123/tello
Using full namespaces like: using namespace cv; using namespace std; is not a good practice but for this project I think it is just fine.
The TLW004 model of drone is a great starter kit. While the video quality may not be the best, mounting a small camera with better resolution could be a more cost-effective solution than purchasing other drone models.
Hero of the story
It is a small yet powerful companion that can be navigated via an application. Besides that, it can be programmed, which can be very useful for small DIY projects. I will use C++ and some open-source libraries.
If you want to skip the coding part and see final result, click here
You can download my project on here: https://github.com/StackSagaCom/HuginnAndMuninn-fly-with-tello
You liked the content?
Subscribe
A thank you is always appreciated!