INTRODUCTION

Applications and Technologies for Unmanned Aircfrat Systems (PAYLOAD)

OUR GOAL

The idea is to connect and install a camera on the Raspberry and mount the Raspberry on the drone platform. In this way, images can be taken during the flight and the drone can take decisions during the flight based on these images.

In the next project our goal will cover the following topics:

1.-Setting up the RPi camera module.
2.-Accessing the RPi camera using Python and OpenCV.

3.-Sending live video from the RPi to the computer.

The steps will be:

• Connecting the camera
• Enabling the camera
• Test out the camera module
• Installing OpenCV
• Capturing a single image from the camera.
• Step and problem found during the process.

We will try to explain the process step by step and comment on the problems encountered and how we have solved them.

Some PC have had to reinstall the GUI graphic interface for Ubuntu.

Setup WiFi: cmd.exe and ipconfig.

Connecting the camera.

We just follow the steps in the PDF file, make sure to attach properly the flexible ribbon cable.

Squeeze black tab!!!!

Open Ubuntu and terminal, and install the libraries:

Enabling the camera

File for editing

sudo nano /boot/firmware/config.txt

Important to add !!!!

start_x=1

Reboot the system

Updating OS:

sudo apt update

sudo apt upgrade -y

Install libraries:

sudo apt install libraspberrypi-bin libraspberypi0………..etc

Open terminal an execute

• Test out the camera module:

We need to check that the RPi camera is working properly.

“raspistill -o output.jpg” is the command line tool for capturing photographs with the RPi camera.

Troubleshooting

#pip3 is not installed in my Ubuntu

search in google: python3-pip

www.odoo.com/es_ES/forum/ayuda-1/how-to-install-pip-in-python-3-on-ubuntu-18-04-167715

sudo apt install python3-pip

Install the packages to acces the RPi camera using Python and OpenCv

Installing OpenCv

sudo apt install libgl11-mesa-glx

sudo apt install python3-pip python3-dev

Install the library for images:

sudo pip3 install -U opencv-contrib-python

Check the installation:

python -c “import cv2”

Now, we try the camera:

unplug and plug the camera again and write the code:

import cv2

#open the camera for video capturing

cap = cv2. VideoCapture (0)…..etc

Troubleshooting

The camera is not working, we tried to check the code with my camera web and it works, we tried to reconnect the cables, the flexible ribbon,etc but it doesnt work.

We gonna try with the other camera provided by UPC, Intel RealSense D435.

Troubleshooting

We have tested the codes to see that the camera and the libraries are working, and have been executed correctly. In the final part, on having to send live video to the station, we have followed the steps of obtaining the IP and configuring the code. Also, we have found some conversion from OpenCv to the “station” to modify the video. I.E, pose the image in grayscale. Note that we have tried to transform the image to the raspberryPi and send it directly in grayscale, but Netgear didnt accept the name of dimensions (2 expected 3), which must be because Opencv does not take care of the channel in the black and white images. Affecting a new dummy dimension with numpy doesn’t work either, which seems to be quite limited in this aspect of NetGear.

NOTE!!! If I try the Intelreal camera, try both USB 3, which has more bandwidth.

Install the packages script/path-realsense.ubuntu.lts as administrator

Troubleshooting

The packages do not appear for the version of Ubuntu 21, only till the 18th version.

We realize that the packages are not relevant, they are simply the software associated with the camera (the SDK). The problem was that the VideoCapture index was incorrect, since there are several devices that can fetch the camera. If I have index -1, OpenCv selects a camera automatically, which was the problem. When changing to an index 2, it works!!!!

Script GROUND STATION

#import required libraries

from vidgear.gears import NetGear
import cv2

#define tweak flags

options = {“flag” : 0, “copy” : False, “track” : False}

#Define Netgear Client at given IP address and define the parameters,(… .77) home IP address.

client = NetGear(
address = “192.168.1.77”,
port = “5454”,
protocol = “tcp”,
pattern = 0,
receive_mode = True,
logging = True,
**options)

#loop over

while True:
frame = client.recv()

if frame is None:
    break
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cv2.imshow("Output Frame", frame)

key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
    break

#close output window

cv2.destroyAllWindows()

#safely close client
client.close()

Script Dron time face detection and video using RPi camera

dron_simulator_cameraface.py is the name of the file

#import required libraries

from vidgear .gears import NetGear

impor cv2…………………….etc the same code as provided in the PDF file.

from vidgear.gears import NetGear
import cv2
import numpy as np

stream = cv2.VideoCapture(4)
#define tweak flags

options = {"flag" : 0, "copy" : False, "track" : False}

#define netgear server at given IP address and define parameters

server = NetGear(
	address = "192.168.1.77",
	port = "5454",
	protocol = "tcp",
	pattern = 0,
	logging = True,
	**options
)

#load default  face cascade classifier
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')


while True:
	try:
		(grabbed, frame)  = stream.read()
		if not grabbed:
			break
			
		# Process the frame
		gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
		faces = face_cascade.detectMultiScale(gray, 1.3, 5)
		for (x,y,w,h) in faces:
			frame = cv2.rectangle(frame, (x,y), (x+w, y+h), (255,0,0), 2)
			
		#send frame
		server.send(frame)
		
	except KeyboardInterrupt:
		break
#safely close video
stream.release()

#safely close server
server.close()

RESULTS

Capturing a single image from the camera.