Saturday, May 8, 2010

The beagle has 2 eyes - OpenCV Stereo on BeagleBoard

I have the ambitious aim of allowing my Quadcopter to see in stereo. So recent tests are being conducted of a suitability of on board stereo processing via OpenCV. So far the unoptimized version of OpenCV makes this really slow and impossible to do in realtime. I just got the last day of author registration in for IGARSS as well, I hope it was on time.

Some USB devices arrived from deal extreme - USB GPS running at  1Hz / 38400 baud and a Ralink Wireless-N dongle which does not have prebuilt drivers in Linux or atleast not one that autoloads with the USB device. I will investigate the wireless later but for now it frees up a slot to try out stereo - so here is the rig.

The gstreamer capture and opencv stereo_match both work as expected and it takes about 300ms to grab frames and 12300 milliseconds to form the disparity map with default settings. Much can be improved in the opencv front to reduce the time required for processing. As a comparison my laptop using the same set of tools - Gstreamer for windows with the ksvideosrc as frame source and OpenCV 2.1. The frame capture speed was 250ms and the disparity map was produced in 300ms.

So a realtime stereo vision based quadcopter may not be possible, but data capture and indoor map building with post processing is definitely feasible. I got a quote for the Hokuyo laser scanner, though significantly more expensive than the stereo rig ( few hundred thousand yen) it has larger field of view and more reliable point cloud production independent of features on the target surface.


telex said...

Just a guess, but OpenCV should run really well on NEON.
Where does stereo_match spend most of its time ?

Ive used OpenCV in the past on a really low-powered ARMv5 device and got actually some decent results, but my processing was really basic, with carefully selected resolution.

Mert Turanlı said...

I think the OpenCV is optimized for Intel processors, are you sure that the compiler will generate code for corresponding SSE functions to NEON SIMD ?
I suppose some extra effort will be needed for converting the opencv functions into NEON SIMD code.

General question,

I think you are planning to do obstacle avoidance using stereo disparity map. My question is what kind of cameras are you using? I am planning to to a stereo project on Beagle by using analog cameras and a TVP5150 video decoder. Are you using usb cameras, are their drivers available?

Mert Turanlı said...

I think the OpenCV is optimized for Intel processors, are you sure that the compiler will generate code for corresponding SSE functions to NEON SIMD ? I suppose that extra effort will be needed to convert the opencv codes into NEON compatible code.

General question;

What is the cameras used in this project, can you give model? I think that you are planning to do obstacle avoidance using stereo disparity map.
I am also planning to do a stereo project on Beagle, by using two small analog cameras from Sparkfun and two video decoder (TVP5150) chips, so the cameras can be triggered simultaneously.

Can you synchronize the frames from the USB cameras?

Tisham Dhar said...

The USB cameras are currently no syncronized, they can be by injecting time or other info via gstreamer into an audio track. The system reference clock can be used to syncronize frames.

I am not planning obstacle avoidance. I believe a laser scanner based solution will fit that bill much better.

telex said...

hudvin said...

I also work on stereo vision for beagle board. What cameras do you use?

Tisham Dhar said...

Some generic cheap USB camera's off ebay.

Unknown said...

Downscale the images to 50% and you will reduce the time to a 25%.

Walter Britton said...

1st of all: Great work. I was researching stereo for flying my quad through obstacles and found you had already looked at this problem.

2nd, have you tried lower resolution? Have you looked at the new Beagleboard-xM?

Tisham Dhar said...

Hi I am still using the original beagle. It can deal with the Opencv decently if the Neon/DSP on it are properly utilized. Lately I have been busy interfacing to the kinect. A stripped down version of the kinect will take off the stereo calibration and processing overheads. Depth data from it will possibly be more accurate as well. On the downside it can't be used outdoors.

Suraj Swami said...


I was deciding on which would be the best webcam for such a project.

Tisham Dhar said...

Best webcam is definitely not the highest resolution, since the limited RAM and USB bandwidth will not let you make full use of it. Choose one with good dynamic range and configurability, e.g turning auto-contrast on/off, also choose one with low power consumption. UVC drivers usually just work for most webcams so drivers are not a problem unless you get something really esoteric.