On map Augmented Reality with Vuforia + Unity

I have been doing some more development with Android, using a different (and easier) renderer this time than the still nascent OpenSceneGraph port, with a bit of Augmented Reality thrown in via Vuforia.Which basically functions as an OpenCV port with image orientation recognition thrown in optimized for Qualcomm's Snapdragon SoC. The feature detection and texture packing is annoyingly performed at the Qualcomm site, I guess they want to keep track of images over which features are detected and prevent inappropriate use by script kiddies. From the looks of FastCV, the features are either MSER or Harris corners.

I already have Android dev kits going so the fun bit was in importing our nice models into Unity and sending the package over to the device. Textured models seem to import best into Unity via Blender, after a bit of copying around of textures and forcing association with the right materials.

Setting up the scene in Unity is fairly straight forward, the Vuforia SDK delegated to Unity for rendering and simply attaches a handler to the Camera. The visible object is automatically centred in the scene so the camera to target geometry is irrelevant in this case. However lighting is not, so a bit of tweaking in lights is necessary for a nice model render. Switch all the materials to diffuse/mobile to load the appropriate shaders in GLES2. Add an LODGroup node if the model is getting too big, though I was able to render some 100,000 triangle models.

When all is set the tracking can begin. I lined up the model to our orthoimagery, the screenshot does not do it justice. It is really cool watching a 3D object stick out of your screen. Everybody at work is very used to viewing things in stereo, but multi-perspective 3D still has a wow-factor.

Building OpenSceneGraph with Android NDK

I have been rapidly picking up Android development for the last 2 days with Native coding in C++ thrown in. Cross compiling for embedded systems is nothing new for me having played with Beagleboards in various incarnations, the Hawkboard and Pandaboard. Neither is JNI, I did a fair bit of JNI work while at CSIRO trying to make NetCDF faster.

So to build OpenSceneGraph for Android (I am using my Nexus S running 4.2), do the obvious:

• Get OSG source
• Get ADK
• Get NDK
• Get ADT if you are Eclipse fan like me
• Get Sequoyah, CDT and whatever else Eclipse wants to build native code.

Build OSG for Android with whatever hack you can imagine. I ended up getting CMake to generate with NMake build files, but built directly using ndk_build afterwards since the MSYS shell could not access DOS commands ndk-build seems to call. Install automation did not work either so I copied OSG headers over by hand into the build directory. It took a while to build, the NDK compilers are rather slow, in the future I will run with -j 12 to take advantage of all my cores.

The next bit was rather easy since the OSG Android sample is already configured for Eclipse. I had to fix up settings with MSYS for bash. Most tutorials refer to cygwin for Linuxy utilities, but I loathe cygwin and I already a have a bunch of MSYS installs floating around through OSGeo4W and Git-bash. The include files and compiler checks are rather strict in Eclipse and any errors in the IDE will prevent uploading the build, be naughty and just delete the errors due to ADT bugs. This is just a proof of concept after all. I had to change a few little things like false to JNI_FALSE to keep the compiler happy. After some toing and froing with gnustl_static, the whole thing built and ran fine.

I wanted to have jpeg textures embedded in the .ive and .osgb files displayed as well as load models with PagedLOD's over http. So I needed to link the little trivial application to curl and jpeg in their Android native incarnations. The hard work has been done already and you can get thirdparty bundles. The first attempt with 8192x8192 textures expectly blew up on the little platform, but atleast libjpeg seemed to be working, I squished textures down to something sane. Libcurl works to download a file hosted on our webserver just fine, but you can't wait for an eternity on this plaform, unless the basefile is a few kb and links to the rest via LOD's the GL context will be lost before the download finishes. Overall it was a fun experiment and I am glad i managed to load a Jesus statue to celebrate this festive season - though it represents death rather than birth.