Making wheel under the cameras - Beagle has 4 wheels

All the work with stereo vision and kinect hackery is of not much use without obstacles to sense and avoid. Hence the beagleboard is now mounted on 4 DC motors driven by the DFRobot Arduino Romeo board. The robot base is quite sturdy aluminum and has plenty of pre-drilled holes, but to mount the beagle I had to acquire my first rotary tool set.



ASUS is coming out with the PC version (Xtion PRO) of Kinect soon, which will be smaller than the original and will not have the RGB camera, it will also be purely USB powered. Perfect opportunity to use it along with the stereo vision cameras to increase the depth of field and build a platform which can use vision indoor and outdoors and in complete darkness, by combining the sensors.

Next comes getting data from the camera out wirelessly with sufficient bandwidth. I have the Beagle working with 3G modems, but this was going to be a first with a Ralink 2870 (which I got el-cheapo at DealExtreme) . With Koen's advice I decided to use the rt3070sta, the modinfo makes is obvious that this is going to work with the card:


root@beagleboard:~# modinfo rt3070sta
filename:       /lib/modules/2.6.32/kernel/drivers/net/wireless/rt3070sta.ko
version:        2.3.0.4
license:        GPL
description:    RT2870 Wireless Lan Linux Driver
author:         Paul Lin


It does not like pumping dhcp at boot time so I have to go the long way and pull it up with ifup later on. The usual gstreamer tricks work fine for sending the camera streams around:


In the beagle - gst-launch v4l2src device=/dev/video1 ! queue ! videorate ! video/x-raw-yuv,framerate=15/1 ! videoscale ! video/x-raw-yuv,width=320,height=240 !  ffmpegcolorspace ! jpegenc ! udpsink host=x.x.x.x port=xxxx

In the laptop - usr/bin/gst-launch udpsrc port=xxxx ! jpegdec ! autovideosink

Next comes pan-tilt and actual object avoidance and object following. Pass the dog a ball.

BeagleBoard Stereo Camera on Flickr

A while back I made my BeagleBoard dual-USB stereo camera. Gstreamer is set to capture single frames in this mode and no video synchronization etc. is performed. Gst-Opencv is now installed on the beagle and I might try chaining in some colour detection, but I will wait this opencv-dsp comes along. Getting the images out of the board over scp/sftp was proving to be a pain, so I decided to install a flickr upload script in Python. Apparently the same trick can be done using cUrl with this REST endpoint, but it did not work out too well for me. The USB wireless modem from yesterday works as a perfect network link to push out the images captured by webcams.

Now I can produce unlimited amounts of stereo samples on Flickr for people to compute disparity maps. I will also put on an IR filter on one of the cameras and try capturing some 4 channel images at a distance where the perspective is not such an issue.

The beagle has 2 eyes - OpenCV Stereo on BeagleBoard

I have the ambitious aim of allowing my Quadcopter to see in stereo. So recent tests are being conducted of a suitability of on board stereo processing via OpenCV. So far the unoptimized version of OpenCV makes this really slow and impossible to do in realtime. I just got the last day of author registration in for IGARSS as well, I hope it was on time.

Some USB devices arrived from deal extreme - USB GPS running at  1Hz / 38400 baud and a Ralink Wireless-N dongle which does not have prebuilt drivers in Linux or atleast not one that autoloads with the USB device. I will investigate the wireless later but for now it frees up a slot to try out stereo - so here is the rig.

The gstreamer capture and opencv stereo_match both work as expected and it takes about 300ms to grab frames and 12300 milliseconds to form the disparity map with default settings. Much can be improved in the opencv front to reduce the time required for processing. As a comparison my laptop using the same set of tools - Gstreamer for windows with the ksvideosrc as frame source and OpenCV 2.1. The frame capture speed was 250ms and the disparity map was produced in 300ms.

So a realtime stereo vision based quadcopter may not be possible, but data capture and indoor map building with post processing is definitely feasible. I got a quote for the Hokuyo laser scanner, though significantly more expensive than the stereo rig ( few hundred thousand yen) it has larger field of view and more reliable point cloud production independent of features on the target surface.