Autonomous Multiwii Quadrocopter with Raspberry Pi

Post Reply
Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

Hallo everybody,

I want to introduce a project we started 10/2012 in Oldenburg (north of Germany). The project goal is to build an autonomous flying quadrocopter.

This is the (very simple) project page: http://www.keten.de

Basic details:
Quad with 45 cm
FC: MWC Crius SE with Gyros, ACC, Baro- and Magnetometer.
Bluetooth connection to GUI
Software: Multiwii 2.1
6 Channel 35 MHz RC (for secrurity and german law...)
Raspberry Pi connected via serial with FC
WiFi USB Stick to Raspbery Pi
Stereo cam via USB to Raspberry Pi

Status:
The Quad flies with RC
Raspberry Pi gets video and streams it over WiFi

Next Steps:
Make a program for Raspberry Pi to control the FC over serial
Control the quad via WiFi
Use the stereo video to calculate 3D information and use that to control the quad
Sonar for ground detection

If you are interested in the project, we are happy when you leave a message or write an e-mail.

Please have a look at the project page for up to date information. Thank You!

User avatar
Hamburger
Posts: 2578
Joined: Tue Mar 01, 2011 2:14 pm
Location: air
Contact:

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Hamburger »

keep us updated, please. Moving the higher level functions to a separate computer looks to be a convincing approach.
Good luck.

User avatar
dramida
Posts: 473
Joined: Mon Feb 28, 2011 12:58 pm
Location: Bucharest
Contact:

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by dramida »

I also propose a Raspberry PI implementation in multirotor drones , see the topic above :)
viewtopic.php?f=7&t=2903


Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

@ Hamburger
Yes of course we'll keep you updated. :) For the higher functions we planed to use ROS, the robot operating system. This can calculate 3D images/video from the two cameras. We are not sure if the 750 MHz from the Radpberry are fast enough for this in realtime or we have to stream the videos down and calculate it on the ground.

By the Way: Thank you for all the work you did! :) Its helps us a lot.
And I was born in Hamburg, Germany...So I'm a Hamburger too. ;-)

@dramida
This is very interesting, but I think this is not allowed in Germany. We have very hard laws for sending electromagmetic waves. Only some frequencies are free for private people and the sending power ist very limited.

@mahowik
Thank you for the links too. I will read it tonight. :)

PatrikE
Posts: 1976
Joined: Tue Apr 12, 2011 6:35 pm
Location: Sweden
Contact:

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by PatrikE »

Kinect instead of stereocameras...

Eric
Posts: 38
Joined: Thu Nov 08, 2012 7:08 am

Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Eric »

Take a look at TPLINK WR703N. It's a WiFi 802.11n router with a serial port, you can install OpenWRT on it. It's small and cheap.

By the way, I do not think PI is capable calc real time 3D.

Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

@PatrikE
Kinect don't work outdoor at day. The Kinect IR Laser only have 1mW and the IR light from the sun is too heavy.
An other reason we use stereocams instead off sonar, laser or radar is, I'm interested in human perception.


Eric wrote:Take a look at TPLINK WR703N. It's a WiFi 802.11n router with a serial port, you can install OpenWRT on it. It's small and cheap.

By the way, I do not think PI is capable calc real time 3D.


I don't get the point. Why we should use a WiFi router? We have a WLan stick in the Rasperry.

I think too, that the power of the Pi is not enough. It depends on the framerate an the resolution. Maybe 160x120 px with 5 fps is enough for us. We'll test it.

balto
Posts: 2
Joined: Sun Oct 27, 2013 6:59 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by balto »

Was anyone able to connect MultiWii to ROS?

Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

@ Balto: Not yet.


Project Update:

The power of the Raspberry Pi is far away from calculation images in realtime. We have problems only to make videos with a USB web cam and save it to the onboard SDcard.

Now we work with the ODROID U2. It's a 4-core 1,7 GHz ARM processor and smaler then the Raspberry Pi. The same processor is in the samsung galaxy S3.

More infos comming soon...

User avatar
NikTheGreek
Posts: 348
Joined: Thu Dec 08, 2011 4:17 pm
Location: Greece
Contact:

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by NikTheGreek »

Stephan Oldenburg wrote:
We have problems only to make videos with a USB web cam and save it to the onboard SDcard.



Have you tried the Pi camera module instead of USB cam ?
I guess that will work much better...
Could you please tell us the latency you receiving during the Video transmission through the Wifi ?

Keep up the good work :)
and please ...keep us informed ;)

User avatar
treym
Posts: 258
Joined: Sat Jul 21, 2012 12:28 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by treym »

If we do a recap ... you attached a raspberry pi and a usb camera onto a multuiwii quad to stream video over wifi ?

Stephan Oldenburg wrote:The power of the Raspberry Pi is far away from calculation images in realtime. We have problems only to make videos with a USB web cam and save it to the onboard SDcard.



one hub only : you cant barely use the same hub for video and wifi ....have a look at the schematic
http://en.wikipedia.org/wiki/File:Raspb ... on_v01.svg


about processing power for video processing with the pi :

http://www.youtube.com/watch?v=3BJFxnap0AI

please folow this link : http://tothinkornottothink.com/post/593 ... csi-camera

as a side note .. you should better invest in either :

- a naze flight controller , with 3.3v for easy connection with your pi
- a beagleBone (either black or white) that run multiwii natively ( http://code.google.com/p/beagle-fly/downloads/list )

also .. a link worth of importantce : http://paparazzi.enac.fr/wiki/Installation/RaspberryPi

Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

NikTheGreek wrote:
Stephan Oldenburg wrote: We have problems only to make videos with a USB web cam and save it to the onboard SDcard.

Have you tried the Pi camera module instead of USB cam ?
I guess that will work much better...


No we didn't try that, because the goal is to use a stereo-cam (two cams). The Pi can only handle one camera module.


NikTheGreek wrote:Could you please tell us the latency you receiving during the Video transmission through the Wifi ?


No we can't. We want to do the image processing hopefully on the copter. If this will not work, we want to stream the images down to a notebook and process it there. Than we may be have latency problems.

Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

treym wrote:If we do a recap ... you attached a raspberry pi and a usb camera onto a multuiwii quad to stream video over wifi ?
Stephan Oldenburg wrote:The power of the Raspberry Pi is far away from calculation images in realtime. We have problems only to make videos with a USB web cam and save it to the onboard SDcard.

one hub only : you cant barely use the same hub for video and wifi ....have a look at the schematic
http://en.wikipedia.org/wiki/File:Raspb ... on_v01.svg



Good schematic. But....: It depends on the resolution and the framerate. 320x240 px with 5 fps should go in the usb hub and out at the same time with no problem.

treym wrote:about processing power for video processing with the pi :
http://www.youtube.com/watch?v=3BJFxnap0AI

please folow this link : http://tothinkornottothink.com/post/593 ... csi-camera


Thanks for those links. The object tracking is impressive, but i think that this robot has no real 3D information with one camera. I stops when the ball is getting "bigger". This will not help to avoid collision with "unknown objects".
But this is what we want to do.

treym wrote:as a side note .. you should better invest in either :
- a naze flight controller , with 3.3v for easy connection with your pi
- a beagleBone (either black or white) that run multiwii natively ( http://code.google.com/p/beagle-fly/downloads/list )
also .. a link worth of importantce : http://paparazzi.enac.fr/wiki/Installation/RaspberryPi


Very interesting project!

Did You work with paparazzi?
At the moment witch naze flight controler is up-to-date? Can You give us a tip?

Thank You!

User avatar
treym
Posts: 258
Joined: Sat Jul 21, 2012 12:28 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by treym »

i do not have any paparazzi autopilot , but you may find plenty of info on the wiki -> http://paparazzi.enac.fr/wiki/Main_Page


here you can find the latest rev5 > FunFly Controller "Acro Naze 32"

you have i2c , uart .. and spi by using the unpopulated epprom pads

http://www.abusemark.com/downloads/naze32_rev3.pdf

User avatar
farzadsw
Posts: 10
Joined: Tue Mar 19, 2013 8:20 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by farzadsw »

Interesting topic !

I am working on (almost) same project.
I have a 33cm (shaft to shaft) quadrotor running multiwii 2.2 code (custom board). also I have built a new board based on naze32 and it works OK, but still using the old 8bit board (it is sufficient).

So far, I have taken some preliminary steps toward autonomous navigation of a quadrotor at indoor environments:

1. altitude control using SRF08 sonar sensor (with minor changes in multiwii 2.2 code)

2. modified v2.2 code to be able to set SOME of rc channels using serial commands (Roll and Pitch channels only) by switching AUX1. It lets me switch between manual and autonomous mods easily :)

3. Running OpenCV+Linux on MK808 board. MK808 has a dual core 1.6Ghz ARM Cortex-A9 CPU which is far more powerful than Raspberry Pi's CPU (and of course weaker than Odroid u2). This board is very small, low weight and have a nice built in Wifi at very low price!

4. Developing OpticalFlow software to measure relative velocity with respect to ground. using Mk808, I am able to run OpticalFlow (LK method) program at 75 FPS ( 160x120 pixels ) and CPU load is between 30-50% ! the bottleneck is board's USB bandwidth.

5. Controlling quadrotor position based on opticalflow measurements and send appropriate commands to multiwii board. A video of early tests are here (sorry, video quality is awful) :
http://www.aparat.com/v/V2M4j

Currently I am working on the position controller to be more stable.

After that I Will use a front Camera for higher level navigation.

Hope it helps.

EDIT:
New video link:
https://vimeo.com/83369166
Last edited by farzadsw on Sat Jan 04, 2014 8:28 am, edited 2 times in total.

Stephan Oldenburg
Posts: 25
Joined: Thu Jan 03, 2013 11:04 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by Stephan Oldenburg »

Thank you very much for that post. Indeed it's very helpful! The idea to control speed or position-hold over optical flow is very nice.
The video is impressive! But the "aparat.com" site took very long to load it...?!?

And the information, that you can transfer 75 fps with 160x120 px over USB is very useful !!!
We want to use two cams (stereo) so we can use round about 2x 35 fps with that resolution. And yes the USB Port is a bigger problem, than the power of the CPU.

If we can use the stereo data to measure distances and the optical flow for speed too with 35 fps, the copter should fly very safe and fast only with optical information. Thats exactly the way we want to go!
Your cam is looking to the ground, right? If we use optical flow too, than we will use optical flow with a front cam. (Because normaly people looking to the front (Only computer scientists looking to the ground... ;-)

Is the data from the cam in raw-format or compresses the cam the videos before the transfer over USB?
Witch cam do you use?

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by copterrichie »

If weight is not an issue, I would suggest using two Raspberry PI with Raspberry PI Cameras and network the two PIs. Reason is, it requires a lot of CPU power to process the USB images where as with the PI camera, the processing is handle by the Graphic processor(GPU).

User avatar
farzadsw
Posts: 10
Joined: Tue Mar 19, 2013 8:20 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by farzadsw »

Stephan Oldenburg wrote:Thank you very much for that post. Indeed it's very helpful! The idea to control speed or position-hold over optical flow is very nice.
The video is impressive! But the "aparat.com" site took very long to load it...?!?

And the information, that you can transfer 75 fps with 160x120 px over USB is very useful !!!
We want to use two cams (stereo) so we can use round about 2x 35 fps with that resolution. And yes the USB Port is a bigger problem, than the power of the CPU.

If we can use the stereo data to measure distances and the optical flow for speed too with 35 fps, the copter should fly very safe and fast only with optical information. Thats exactly the way we want to go!
Your cam is looking to the ground, right? If we use optical flow too, than we will use optical flow with a front cam. (Because normaly people looking to the front (Only computer scientists looking to the ground... ;-)

Is the data from the cam in raw-format or compresses the cam the videos before the transfer over USB?
Witch cam do you use?


I'm Glad you found it useful.
Speed of my Internet connection is very limited, so have to upload it on "aparat" :roll: , but I will upload it on youtube ASAP.

I use a PS3 eye camera (looking ground) which have great performance for the price. It can transfer pictures up to 120 FPS at 320x240 frame. unfortunately minimum resolution is 320x240 so I resize the frame to 160x120 in program before using them. Therefore USB bandwidth limit of mk808 is 75 FPS in 320x240 resolution.
It is something related to MK808 mainboard because I can use the camera at 120FPS on my laptop with no problem. So maybe you can get even more FPS on ODROID U2.
In that video, frame rate was 30 FPS so It can be more Stable at higher frame rates. 8-)

Notice:
The OpenCV uses compressed video format (RGB) to capture frame from camera. It is very slow and almost kills the processor (for nothing!) :o . To solve this problem I used an opensource project, named "OCVCapture" which uses V4l2 library to capture video frames in raw format (UVC I guess). Now capturing video takes much less CPU load(~10%). :D

I think calculating Optical-Flow on a downward camera gives you more accurate velocity data, compared to front facing camera. For higher level navigation (such as obstacle avoidance) a mono-front-facing camera would be enough. Calculating OpticalFlow divergence can gives you relative distance to obstacles (apparently flying insects use the same method!).
I plan to use 2 camera, one facing ground and another front facing. but it would be nice to compare it with your method (one stereo camera). ;)

If weight is not an issue, I would suggest using two Raspberry PI with Raspberry PI Cameras and network the two PIs. Reason is, it requires a lot of CPU power to process the USB images where as with the PI camera, the processing is handle by the Graphic processor(GPU).

I agree, but CSI cameras generally are more expensive and have limited Frame rates. Using v4l library, capturing video frames takes much lower CPU load but still it is more than what CSI cameras take.
I prefer to use a more powerful board with an usb camera.

copterrichie
Posts: 2261
Joined: Sat Feb 19, 2011 8:30 pm

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by copterrichie »

It maybe worth mentioning here, there is a Official V4L2 driver supporting the Pi Camera in beta.

http://www.raspberrypi.org/phpBB3/viewt ... 43&t=62364

User avatar
treym
Posts: 258
Joined: Sat Jul 21, 2012 12:28 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by treym »

farzadsw wrote:I use a PS3 eye camera (looking ground) which have great performance for the price. It can transfer pictures up to 120 FPS at 320x240 frame. unfortunately minimum resolution is 320x240 so I resize the frame to 160x120 in program before using them. Therefore USB bandwidth limit of mk808 is 75 FPS in 320x240 resolution.
It is something related to MK808 mainboard because I can use the camera at 120FPS on my laptop with no problem. So maybe you can get even more FPS on ODROID U2.
In that video, frame rate was 30 FPS so It can be more Stable at higher frame rates. 8-)


those ps3 eye are very heavy .. be carefull with how you mount them .. any jello or vibration in your video will cause wrong calculation and ghost points :

here, i am standing 2meter in front of the copter ,

screenshot-1331549541.png


for stereo i would stick to a kinect

User avatar
farzadsw
Posts: 10
Joined: Tue Mar 19, 2013 8:20 am

Re: Autonomous Multiwii Quadrocopter with Raspberry Pi

Post by farzadsw »

treym wrote:those ps3 eye are very heavy .. be carefull with how you mount them .. any jello or vibration in your video will cause wrong calculation and ghost points :

here, i am standing 2meter in front of the copter ,

screenshot-1331549541.png

for stereo i would stick to a kinect

thanks for the tips.
I have opened the ps3 eye's case and shortened its USB cable and now it is about 40 grams (camera PCB + 10 cm USB cable).
also for mounting it, I have just sticked it under quadrotor frame using a double sided tape :D .
But I agree that mounting front facing camera (mono or stereo) needs more attention.

Post Reply