Week of fun Hardware - Octocopter, Extruder, Kinect Poetry and MIDI 2 Life

Last week was an amazing overload of gadgets (with some flying to Canberra and hanging out at a bar with E-Tax release manager thrown in). First highlight was the Octocopter at UTAS - it is indeed a massive beast with 8kg border line payload, paltry 5 minute endurance, but it gets the kit off the ground. I also saw some of the amazing work Arko has been doing with Bundler, PMVS and Meshlab.

 The night before I went to Canberra, there was a Sound2Lights show in the Long Gallery with lots of cool gadgets. Including the aforementioned Rep-rap extruder printing out music (Joseph) , a Kinect you had to do a weird dance to get to put words in place (and possibly rhyme and form poetry - Aaron) and of course Nick's game of life with music controlling organisms rendered in Unity 3D.

 

Thursday early in the morning I flew off to Canberra to help deploy Australia wide terrain, but that is another story.

Kinect + BeagleBoard-xM (now need GLES)

My shiny new BeagleBoard-xM arrived a few days ago. I have been too snowed in with OSGi to spare it some time. The test image on the supplied SD card boots fine, but I have grand plans for a slamming robot vehicle which will require more goodies (probably eventually ros).

It has become even easier to get started with kinect and beagleboard thanks to the inclusion of libfreenect master in angstrom. It can be almost plug and play.

1. Install non-ramdisk angstrom. From windows the easiest way to get started is to download and SD image Koen graciously provides and burn it on top of the test image supplied with the board. As expected a few hiccups with the image writer but nothing a downgrade would not fix.
2. Install libfreenect directly (opkg install libfreenect) or grab git/cmake etc. to build from source. Even when building libfreenect from source, having libfreenect-dev installed is handy since it fetches all the other headers needed.
3. Make an OpenCV based demo, the standard demos don't work so well because the Beagle video driver lacks GLX. Need some more packages opencv, opencv-dev, compilers and whatever else. The building required a few symlinks for compilers and opencv libraries since I was compiling on the beagle itself rather than cross-compiling from oe.

My TinCanTools xM trainer board with its Arduino will be arriving soon. The idea is to get a couple of servos and probably even a mobile robotics platform to start building an indoor track and shoot platform, with some slamming and co-operative map building thrown in. If anyone is interested in building a GLES client for the Kinect on the BeagleBoard let me know.

Beagle has 3 eyes - Kinect + Beagleboard

People have hooked up the Kinect to a few embedded platforms - Intel Atom and Gumstix Overo have come up. It was time for the other darling of open development, the BeagleBoard to talk to the Kinect. I have done stereo on the beagleboard before. The kinect makes things a lot easier.



First the usual step of getting libfreenect going, I ended up using unstable since it includes the OpenCV bindings. The sync API is much more stable than the callbacks and has fewer frame drops. Since I don't have a spare keyboard/mouse set for the Beagle and Synergy refused to co-operate, I did a demo frame viewer run with OpenCV rendering the frames. I had to upgrade opencv, ffmpeg, libusb, cmake and bunch of things on Angstrom, then replace CMake opencv detection with pkg-config to get everything working.


Meanwhile there have been some major changes in the OpenKinect world with PrimeSense releasing the OpenNI framework. This releases the Microsoft stigma from the Kinect and encourages more open source development of applications. You can grab the OpenNI code from Github and build on your platform a from the manufacturer kinect driver. No stigma about a hacked Kinect (or shall we say PrimeSensor) any more either, it is officially open.

Openkinect with IR and CSIRO Cop Meeting

The IR sensor on the Kinect can now pull images illuminated by the laser. The camera needs to be initialized with a separate command to fall into IR mode, search is also on for higher resolution streams than 640x480. I have made some headway towards a v4l2 wrapper for the Kinect streams using AVLD. This involves capturing data using libfreenect and writing it to the pseudo-device created by avld.

This week I am also at the CSIRO Cop meeting at ANU in Canberra. There will be 1.5 days of chats and some informal follow-ons. This mostly a programming and data management community with a pile of Matlab, Unix scripters, web-service developers and database admins. The first talk was about eResearch and corporate infrastructure to support it. The second talk was from the legal section covering open source and open data, terms like encumberances, freedom to operate (FTO) and due diligence. This talk covered licences and caused a fair bit of discussion.

The third talk was about OffSiders, which feels like a new object based scripting language. It will require some experimenting to work out what it is good for.  The persistence is non-atomic and exists as a transaction as supported by the filesystem. It has a pure computer science glow and will definitely trigger some "what is like ?" buttons.

Joys of cleaning the house - Nuking non-compliant code

The openkinect project is nascent and as such the win32 platform support is still stabilizing. Due to the naughty behaviour of libusb on windows several attempts have been made so far to create common API across platforms without a simple maintainable and working version emerging.



I decided to learn about kinect, coding on github and merging things in general by taking Zephod's code and putting it in a win32 folder under the library source. I got most of the demo code compiling and running with this approach. I took a master piece of my Ikea chair in depth on Vista 64, apparently a rather difficult OS to get to work on. By then my git code tree had become cluttered with lots of merges, I was working off 3 remotes (my own fork, Zephod and mainline). With some nudging from this video, I deleted all copies of my work into oblivion. I can't control Google cache and such but it felt liberating.

Python Bindings for Accidental API - and vivi.c

I started a couple of Kinect related projects. Firstly, writing linux kernel driver based on v4l2 and cloning vivi.c to act as a stub. The responses indicate that a gspca based approach might be better, but I had a lot of fun hand hacking the vivi.c code and adding 2 video devices from 1 driver. I will have to wait till someone puts some gspca stub code out.
Secondly, I completed a first working version (as in can get some image buffers) of the win32 python binding from Zephod's code. I followed the swig how-to's and had to resort to typemapping to extract the image and depth buffers, but it all worked out in the end. Not quite as well as I had expected on the 1st attempt but I can replicate some of the early work on Linux in windows now. The colour's in the RGB image are flipped, I will have to change the data order. A lot of frames are dropped so the response is not as smooth as it could be.

Getting Data from Kinect to OpenCV

A fair bit of work has already been done in simple template based object identification with the Kinect and even Skeleton tracking. I finally got my hands on one today from JB Hi-Fi, and paid a few bucks extra for it. Harvey Norman finally got them in stock at the end of the day. Terrible logistics or favouritism to employees, or something in-between. Anyway I can now get a 2nd one from the pre-order. Apparently a single USB 2.0 bus is insufficient to handle the stream from 2 Kinects. I did some simple experiments with OpenCV, just adding some hue mapping and swapping the RGB channels, tomorrow we are having the 12Hr Hackathon at Adelaide Hackerspace, might get to make a lantern controller then.

Preparing to hack the Kinect - OpenKinect on MinGW32

My Kinect is on pre-order, I can only get my hands on it tomorrow. That hasn't stopped me from hacking up the OpenKinect code on my Virtualbox Ubuntu and MinGW. There are multiple API's for it by now in C, Python, Java and C#. I choose to use the C version for easy portability and common Windows/Linux version, as well as the luxury of using CMake.

Here are the steps to get the glview kinect code compiled with MinGW32:

1. Grab the OpenKinect code from Git, using MsysGit or similar.
2. Grab the windows compatible libusb, I got the precompiled version to save time.
3. You may or may not have GLUT, if you don't grab the libraries here and use reimp to make them MinGW32 compatible.
4. Add a win32 specific linker line into CMake which reads: target_link_libraries(glview freenect OpenGL32 glut32)
5. Configure using CMake, point to where libusb is, compile and you are ready to possibly enjoy OpenKinect on windows. I make no guarantees on whether it will actually work yet. If you have Windows and MinGW try it and let me know.