Merry Christmas and Happy New Year

So the holiday season is finally here, but I have burnt up all my leave and my spare cash going to Kenya and buying a car. So time to sit tight and haunt people's houses for free meals which they will gladly provide in this time of the year when generosity and goodwill rule their thoughts, otherwise the ghost of Christmas will haunt them every year instead of just me. Not that I can't cook, with sites like Cooking for Engineers to help me, it is not hard. Tonight's special was Prawn Jalfrezi, with prawns you have to be careful not to overcook them or they become tough like hard boiled eggs.

I am still studying for my Australian Citizenship Test. Paul if you are so eager you can do this one here, it has answers unlike mine where I am too mean to provide answers. Only the question about which is the political party currently in power needs the answer to be revised since Kevin Andrews is no longer holding down that particular plum job. All hail the separation of power now that the Federal Court has reinstated the Haneef visa. I hope they have a question about how arms of the government tangle with each other.

Also test drove RAT TerraSAR-X Driver and INSAR capability, it was a breeze producing unwrapped phase. Just need to quantify the baseline and extension to spaceborne platform position information to allow accurate DEM generation.

CNES(Pere Noel) releases ORFEO Toolbox 2.0.0

Pere Noel from CNES has presented his eager Remote Sensing children with OTB 2.0.0 this Christmas. Orfeo Toolbox is part of the Orfeo Accompaniment Program for the upcoming Pleiades and Cosmo-Skymed sensor constellations. This toolbox provides an Open-Source framework for processing and exploiting SAR and Optical imagery.

From the OTB 2.0.0 Release announcement:

A major release like this one happens once a year. The OTB Development
Team has worked very hard in order to make available a robust library
with plenty of very interesting things. Thanks to Emmanuel, Thomas,
Julien, Romain and Cyrille for the good work.

Thanks also to ITK, OSSIM, GDAL, 6S, libSVM and BOOST developers for the
good software upon which OTB is built.

Be careful: with OTB-2.0 you can build powerful applications easily and
for free. You can for instance produce a pan-sharpenned, ortho-rectified
Quickbird image in a few lines of code. This can hurt sensitive people.

Other key features include:

- Radiometric calibration using the 6S radiative transfer code. - Image registration with parametric geometric transformations. - Demo applications for change detection, road extraction, visualization, etc. - All existing features of version 1.6: segmentation, classification, feature extraction, spatial reasoning, change detection, etc.


OTB tends to lean towards being monolithic at times with internal source tree copies of ITK, Boost and Ossim, but it keeps people away from having to separately download and install dependencies. The only external things to install are CMake - it makes multi-platform builds a breeze, a pre-built binary copy of GDAL from FWTools and FLTK, a simple GUI toolkit which trades speed for aesthetics. Personally I would prefer a nicer GUI Toolkit such as Qt but FLTK works just fine, and the whole approach is library oriented so it will be easy to plug-in your own GUI.

After a few false starts I built OTB with MSVC++ Express and NMake files from CMake. Here are the basic screenshots.

Basic Viewer Manager:



Diaporama (ENVI Style viewer with transitions):



Interactive Change Detection:

New Car for the New Year

So I finally I got myself a new car. It is a very well maintained Toyota Camry, looks like an oldish Mercedes and drives very quiet, accelerates smoothly and has a lot of power. Quite a bit bigger than my last car , I hope to have this one for a bit longer. Just that it is dark colour so I will have to drive it with the lights on.

Using Sqlite with QGIS-Grass Toolbox

Now that I have slogged together a lot of database drivers for Grass and GDAL it is time to put them to good use. In this article I will demonstrate how to create a Grass vector map with attributes stored in an Sqlite embedded database. I am using my Brazil mapset since the data is cities and roads in Brazil.

First open a suitable mapset in Grass from QGIS (follow the QGIS-Grass tutorial) and connect the mapset to an sqlite database by choosing sqlite driver from the dropdown box as shown below. Edit the database location so the it is created in a directory that exists, I suggest removing the already suggested dbf folder(meant for the dbf driver) and replacing it with sqlite.db. In the future this should be auto-suggested by QGIS.
Then open a suitable Shapefile or other OGR supported vector format for import into Grass. Your screen will look something like below.
Now import this dataset into Grass using the Import Vector from OGR command as shown below. You can also use the Grass shell to ensure that your attribute table gets created in the Sqlite database. After this step your vector is ready for use in Grass and all the attributes can be manipulated using SQL in the Sqlite database to perform joins with other tables or any other operation you can dream up.
The final map with various classifications produced from the Brazil dataset after importing it into Grass is shown here.

Using MingW-Fu to link to MSVC dll's and lib's

Lately my MingW-Fu has improved vastly thanks to the attempts at building Grass, GDAL and Qgis with all the possible dependencies and drivers. Here are some lessons I have learnt :

• Don't trust libtool - Libtool in MingW is rather hobbled and needs to be fixed for long link lines using a the sed command described here.
• Use -no-undefined - MingW linker does not like linking to libraries with some undefined symbols, use that linker flag when compiling dependencies.
• Create shared libraries not static ones for dependencies.
• Learn you way around the reverse engineering tools for porting from MSVC built static and dynamic libraries, specifically reimp, pexports and dlltool. Sometimes function call definitions need to be renamed using sed or added with @4 or @32 at the end (not sure what purpose this serves), I picked it up while building MySQL libraries for linking.
• Edit any autoconf or pkgconfig scripts to use MSYS style include and libs location(Those nasty slashes will get you otherwise).
  
• Any of these tricks may fail if you have specific name mangling, assembly code and other fancy things in the MSVC libraries. I have still not managed to make a linkable version of libecwj2.dll (ErMapper - Leica ECW/JPEG2000 library), mainly because it contains some of the above features.
  

New things - Computers and Scholarships

This week is turning out to be quite good. I got a new machine, Quad-core Intel with Nvidia 8800 graphics card and double 22inch monitors. I spent a couple of days setting it up. Then today I got a reply from the University of Adelaide saying that I have been given the APA scholarship to study for a PhD in SAR over 3-years. You have a few bad months then things turn around. I am planning to buy a new car over the weekend , then I will have to zealously guard the 1 remaining demerit point I have.


The Gold Qgis release build with grass compiled with ODBC, Sqlite, MySQL and PostgreSQL. I could not get GDAL linked with the same MingW being second class citizen in the GDAL world and libtool being annoying and limited as always. The current one is available here. Hope it works for you. May be one day we will have a Platinum release.

ImageLinker , the Ossim image processing GUI is now nearly fully ported to Qt4 and builds and runs fine if you do not try to optimize too much and strip symbols.

Releasing Qgis 0.9.1

Qgis releases seem to be coming along fine. Now even my photo is in the official developers list. What an honour !!

The windows installer is available here for testing till an official release is announced. It includes latest grass release candidate, but misses Xerces and hence GML support. Living on the edge has cost me, GDAL Xerces detection to longer picks up my Xerces and Expat. I hope this will be resolved by the time we make a full release. I will try rolling back GDAL until it works but this is the latest of the latest code based release.

Who wants to be an Australian ?

So I am finally at the stage of preparing for the Australian Citizenship Test, the resource book is quite detailed but living here and reading wikipaedia is even more instructive. There are some sample tests that allow you to practice, but require signing up. Since on sample test page they say the Minister for Immigration is looking for professional question setters, I took it up to make my own flash-card like test practice application. It is simple HTML-JavaScript-PHP with the questions in a text file and no answers, once I have built it into a decent set I am planning to add it facebook as an application so that Australians can check their Australianity.

Back in the Saddle - Qgis 0.9.1 with MSVC

I am back in Adelaide in space , but not it time. My body clock is all messed up and keeping me awake at night. At least it is some other time of the day somewhere else so I can be productive. Qgis seems to making great progress towards a full MSVC build, that may not sound so good to Open Source purists, but on Windows MSVC offers superior compile time and debug capabilities than MingW does at the moment.

Tim has put together a short-cut bundle of binary dependencies so that you do not have to build everything (but building stuff is my favourite passtime anyway). The build works fine without Python-SIP, which will be easy to add and GRASS which will be slightly messier to add.

Adelaide via Bangkok and Sydney

Finally back in Adelaide after flying around and waiting around in airports for 3 days. I had a 12 hour stop over in Bangkok, which I spent snooping on wireless networks and recruiting for my Metal Gear on PSP using AP Scan. Some shopping ensued as well, but being cash strapped does not bode well for that activity. The security check queue got huge and amorphous, it was funny watching people gulping down bottles of mineral water to avoid having the fluids check. Someone should really drink a lot of water and take diuretics on-board.

The flight from Bangkok was a bit delayed - 20mins, that nearly made me miss my connection to Adelaide and my luggage did not make it. So I spent an anxious evening in the city waiting for it. Then I spent 30mins in the taxi queue, they have made it really hard for taxi's to get into Adelaide Airport. Eventually I managed to go to Sosta for dinner, been meaning to go to an Argentinian Restaurant for a long time. The roasts were fabulous and so much that we had to take some home.

Camping in SweetWaters

Tonight I am staying at the SweetWaters Tented Camps ( these are first grade tents with running water and furniture) in the Ol Pejeta Conservancy on the Laikipia Plateau, nestled between Mt. Kenya and the Abderdares.

The conservancy houses orphaned/rescued chimpanzees from Congo, Cameroon and other parts of central and westerm Africa. It was founded by Jane Goodall. There are also breeding populations of Black and White Rhinos, various other rare herbivores are also conserved. I got to pet and feed sugarcane to a tamed black rhino called "Morani". Here you can just sit outside in the dark with a small ditch separating you from the animals at the water hole, it is a great experience indeed.

Watching Elephants from TreeTops

After all the excitement of last week I am relaxing this weekend in the Aberdares, watching elephants and buffaloes from the windows of the wooden lodge at TreeTops. We also went for a game drive but visibility hampered by the dense thicket in the park. Stay in the hotel , up all night spotting game and chatting with people is more fun.

I did a fair bit of chatting last Friday as well doing an interview with Bertha from Nairobi Star about growing up and education in Kenya and life afterwards. I demonstrated the the time based refugee tracking in WorldWind at UNHCR-Somalia which happens to be in the same building as Nairobi Star.

NASA and its value to the people

Since its inception NASA has tried to justify its raison d'etre, in the beginning it was the space race with the Soviets, but with the warming up of the cold war that incentive has evaporated. These days the public want real deliverables from NASA and the perception is that it is over-funded and under-performing. This is where NASA needs to communicate to the populace how it will directly benefit them much like the national education (NASA has much to educate people about) and health agencies (Space observations can track pollution levels) . This article brings out the discrepancy between people's perception of NASA funding and the reality.

"In a survey ... NASA’s allocation, on average, was estimated to be approximately 24% of the national budget (the NASA allocation in 2007 was approximately 0.58% of the budget.)"

Misconceptions indeed ....

Visiting FAO-SWALIM, UNHCR-Somalia and Wananchi Online

I set myself up for a busy day on Tuesday the 20th. Running around Nairobi all day and visiting various contacts Maurizio had dug up. I also had to get to Loita house by 6:00 pm for a talk to SkunkWorks, a Nairobi gathering of techies.

First visit to FAO-SWALIM was fruitful, Craig was eager to embrace the OpenSource ideas and spend their limited budgets on setting up infrastructure in Kenya and capability development rather than on expensive software license purchases. They also have a lot of satellite and thematic data to share and are taking steps towards making it publicly available.

Next I went to UNHCR office for Somalia, John Marinos was trying to publish the data peace keeping, protection forces and aid agencies gather on displaced populations to Google Earth. A small bug in GE due to the asymmetry of KML import and output was frustrating his efforts. This bug is basically discarding the thematic palette information for KML icons when multiple KMZ's are being aggregated. So I had to script the aggregation myself and send him a master KML which can be kept updated with monthly data, ultimately I hope the database will be able to automatically generate proper KML's for distribution. These KML's had folder based time-tags, so I went ahead and extended the WorldWind KML Parser to respect these. Now the TimeController can be use to play through series and fully appreciate the grim reality of life in Somalia. I will try to add a time slider as soon as possible.

The final meeting was at Loita House - Wananchi Online. A selection of SkunkWorks members turned up, my talk on satellite sensors - SAR and Optical went without a hitch. People really perked up when I started demonstrating WorldWind. Lots of questions arose on performance in low bandwidth, age of data, data costs, methods for acquiring street data and building heights: there seems to a lot of need in Kenya for base mapping data and few very expensive sources.

Open Source SAR Analysis - Story of RAT

I have a few more minutes at the airport, the flight is delayed. This is a good time as any to announce that now I am an official RAT - Radar Toolkit developer with commit access to the berlios svn. My contributions so far are only in the "IO" segment, ALOS-PALSAR CEOS, TerraSAR-X COSAR and XML metadata loaders. I am hoping to contribute some dual-polarimetry analysis tools as my skills develop.

SAR is a very specialized sensor without wide spread application, as such good SAR programs are few and far between. Most so called commercial packages are half-hearted commercialization attempts from university research groups. RAT is a robust open-source alternative chiefly developed by the enthusiastic Andreas Reigber at the Technical University of Berlin. IDL is the language of choice for these folk, it is similar to Matlab, high-level and and mainly procedural. Obejct and events support are sort of patched on. Other attempts at similar toolkits have been by Atlanta Scientific-Vexcel-MDA using Python-C base and an OpenEV viewer and by ESA in TCL (PolsarPRO) but neither of these in my opinion offers the easy extensibility and ease of use of RAT, provided you somehow procure an IDL licence. I similar parallel toolkit in Matlab will also come in handy, but the development of this has been sporadic, I really need to dig up the bits I wrote during my undergraduate work and start something going.

One of the best SAR features in RAT so far in my exploration is the LLMMSE Speckle filter, it produces a clear image from the standard speckled SAR image.

Final day in Mombasa - Amenities Failure

For the last day in Mombasa, Kenya struck back with vengeance. The power and water failed for the day. We were glad to be getting out as the five-star service tore apart at the seams. The only cooling was the pool. I did however manage to go to Kilifi and see the swahili huts and a beautiful pristine super white sound beach with clear blue water. The best part was it was totally unpopulated, except for a few sand-pipers and crabs.

Now waiting for the flight back to Nairobi and roaming the net, GPRS is such a boon.

Weekend in Mombasa - Life on the Beach

Here continues the greatest monologue in my life. I am spending a 3-day weekend in Mombasa , in the Sarova Hotel with a white sand beach just outside the window. Spending time with the family is a delicate balancing act, when you get around it life on the beach is a lot of fun. When you are not losing the rooms keys attached to giant fish in the pool.

Having drinks in the beach bar, lazing about in the pool, taking morning dips in the sea-weed encrusted water. I took a ride on a camel and we went looking for sea-life among the corals, the tropical fish and echinoderms ( starfish, sea-urchins etc.) are beautiful and find a great habitat among the coral. You have to dive or at least snorkel to fully appreciate them. The family insisted on recording every activity with forensic precision. Mombasa feels like some old asian town, with its mostly swahili and arabic population, lots of somali have come in recently as well. We ended up in a packed narrow-lane in the old town looking for a "Kanzu"-traditional swahili long dress. I felt like I was back in the cloisters of chandni-chowk in Calcutta.

I have one more day here, the plan is to see Fort Jesus (old Portuguese Fort), Mamba Village (Crocodile Farm) and Haller Park. Lets see what happens plans never turn out as planned.

Mobile Net in Kenya - Vodafone VMC with Safaricom SIM

Restless while being disconnected from the net I took the first opportunity to get my Vodafone Mobile Connect Card from Australia working here with Safaricom. Safaricom is the local representative of Vodafone, the operate the network while Vodafone does all the back-end, billing etc. I am not exactly sure of the relationship, which seems to be fraught with confusion and corruption allegations. As long I can get the net I will bear any back-hand deals, they are for the greater good.

Since Jerry works for Safaricom now I called him up and got the relevant APN and PAP/CHAP authentication details to make my Huawei hardware work properly. He came into the party a bit late, but I definitely owe him a drink.

Pyramids better wait - Story ni Ndefu

So I am finally in Kenya, having fun here but the pyramids have to wait a few more years in addition to the thousands they have for me to get to them. Unless some ultra-religious group comes to power in Egypt and decides to knock them down like the Bamiyan Buddhas. The visa from Kenya did not materialize.

Meanwhile I have been shuttling around Nairobi-Thika-Juja trying to stay in touch with people and see the sights. Today we ended up in Lake Naivasha, riding in a small boat, spotting birds and hippos.

The other major event was a giant get together of all my friends from Mang'u , after some 7 years. People have moved ahead and become lawyers, doctors, entrepreneurs, pilots and one is even standing in the Kenyan elections. We went on expanding and taking over Kengele's until there were nearly 50 people. I nearly burned a hole in my wallet buying everyone a round of drinks. Catching up was fun and there were excited groups swapping mobile numbers. I need to do this more often.

On my way

Leaving Australia on another grand adventure to Nairobi then hopefully on to Cairo. Wait for me pyramids.

Backporting Interactive Compass - Reflector and Obfuscation

Recently a Brazil targeted version of WorldWind was released, called GeoPortal-3D. Chad talks about it in his blog. The main additional features are Brazil specific placenames and imagery, the other useful new feature that quickly meets the eye is the interactive compass. This behaves in a manner very similar to the Google Earth navigation widget, just does not fade out to a normal compass when not in use, I guess we can merge one of WorldWind's existing compasses to achieve the same result. Anyway since WorldWind already has a few navigators another one which is familiar to people will do no harm and offer more choice.

Thus began the effort with Reflector and FileGenerator to create the same plugin in the WorldWind core tree. After a few fixes due to code upgrade and path changes the plugin works without a hitch. Useful new addition to moving around the globe. Other than this there are a few more plugins added in GeoPortal with some database features which would be useful to backport.

Back-porting from binaries raises the specter of intellectual rights protection and code security, I guess this is where assembly signing can come in for authenticity verification and this nasty bit of obfuscation can be used to deter the not so eager disassembler.

Qgis Release - with Grass 6.3RC1

Finally went out for a night in town to celebrate the hard work and final release of Qgis 0.9.0 . The night ended with a predictable crash brought on by too many shots of vodka-midori-cointreau, called "What's That".

Before that I had to spend a few nights relinking Qgis and GDAL with Grass 6.3 Release Tag. Thanks to this a lot of advanced GIS functions are now available in Qgis. The relink brought on a dependency hell which I had to resolve using Dependency Walker. It is a great tool for windows release builder, helps to make sure the user gets all the components bundled in the installer. Just makes the installer very big, but hey it works. If you want better package management use Linux or BSD.

New World of SAR - A round-up of SAR Systems

Recently I wrote an article with the same topic for a magazine that shall not be named, and received claims from editor that it was too biased. Since I can't make the deadline for a new article and the editors left it too late to pass on their opinion, I am going to use this soap box to stand and narrate my perspective on the current state-of-art in SAR Systems, commercial and research based, the rest (presumably a lot of them military) are not common knowledge.

• Air-Borne Systems

• Commercial - Very few commercial players exist in this field
  
1. Intermap IFSAR - Operational X-Band single pass Interferometric System with proven track record and very large archive of proven quality data. The P-Band system feature foliage penetration but is repeat-pass based.
2. Fugro-EarthData GeoSAR - Newly operational for X and P band single pass interferometry. Available data archive is limited and data validation is not wide-spread. Theoretically should produce good quality DEM's using P-Band but this may conflict with the X-Band results, needing reconciliation. The system is ex-NASA. The accuracy in the system is achieved by redundancy/repeat flights. A good set of samples can be obtained at the NOAA site.
   
3. Orbisat InSAR - A Brazilian system with InSAR capability in X and P bands.

• Research - A fair few research systems exist operated by space agencies and educational institutions, the data from these systems has limited availability is based on pre-planned campaigns. A suitable summary is on the POLSARPRO site. In addition to these a lot of institution have the aspiration to operate a SAR system, it is a highly prestigious club to belong to.
  
1. AIRSAR(NASA/JPL) - The elder statesman of air-borne systems, last known campaign was in 2004. The last time I accessed the AirSAR site was offline.
   
2. EMISAR(DCRS) - Technical University of Denmark dual-band(L/C) fully polarimetric system.
3. ESAR(DLR) - Quad-Band(X/C/L/P) fully polarimetric system with very high quality data used for Insar, Polsar and Polinsar research. This system served as a template for the TerraSAR-X sensor.
4. Pi-SAR(NASDA-CRL) - JAXA Airborne L-Band system, the inspiration behind JERS and ALOS-PALSAR.
5. RAMESES/SETHI(ONERA) - Someone in France must be obsessed with Egyptian history and pharaohs, or may be it is related to the sand penetration experiments with these systems.
6. SAR-Convair(CCRS) - Polarimetic X/C-Band system used as a test-bed for Radardat 1 and 2 sensors by the Canadians. Mainly used for research in ice-monitoring, because that's all Canadians are interested in.
• Space-Borne Systems -Recent years have seen the launch of numerous SAR sensors, both civilian and military. I have started to lose track of the military ones, here are a few: SARLupe-1 and 2(Germany) , YaoGan(Chinese), Cosmo-Skymed 1 and 2(Italy) and many more. The systems of interest to me are those that have readily accessible data, are currently operational or will be near future(which can mean anytime in the next 5 years given the nature of the space industry - you can really feel the relativistic time dilation, we must be near a black hole).
• Currently In-Orbit Systems - These are either old die-hard systems, long past their scheduled expiry date or recently launched top-of-the-line sensors.
1. RADARSAT-1 - The long lived Canadian SAR system operating in C-Band HH.
2. ENVISAT-ASAR - SAR sensor on the multi-sensor Envisat bus. The data from this sensor is accessible for research from a rolling archive over the last 15days. The sensor can operate in alternate polarization mode.
   
3. ALOS-PALSAR - The first fully polarimetric L-Band space borne sensor. The data from this sensor is heavily consumed by the Kyoto and Carbon project for global forest monitoring. It collects on a fixed schedule over all land-mass. The data is highly affordable and of good quality.
4. TerraSAR-X - Newly launched poster child of the SAR world, first commercial SAR sensor to provide upto 1m resolution. Alternate polarization mode is operational, full-polarimetry and along track interferometry are some of the research modes available.
• Planned/To-be-launched-soon systems - These are the bad boys, getting to school late or the toddlers which show great promise. Not yet in orbit but will be nice to have data from them.
1. RADARSAT-2 - The naughtiest of the bad boys, showing a lot of promise but not sitting for the exam. Long planned, but delayed in launch, it will be the first fully polarimetric C-band spaceborne system.
2. Sentinel-1 - Follow on to the aging ENVISAT system mentioned above, with upgrades with new technology in C-band. Unlike its predecessor, it will be a smaller and dedicated SAR bus, other optical sensors will have to find their own rides on Sentinel 2 and 3. It is due for launch in 2011.
   
3. TerraSAR-L(Cartwheel) and Tandem-X - The novel concept in SAR systems is a constellation, this will allow single pass along-track and cross-track interferometry.
4. MAPSAR - An L-band joint program between INPE(Brazil) and DLR, due some time the next decade.
5. RADARSAT Constellation - Another program due next decade or after that(if RS-2 is anything to go by) designed to provide daily global coverage using SAR.
   

That is my summary for today. Those that have attended more conferences than I have may have encountered a lot more exotic ideas and sensors. I hope this will attract comments on those sensors not mentioned here and their capabilities.

TerraSAR-X format support in GDAL

SAR data is so out of this world that they always have to invent new data formats to put it in and present it to the end-user. Terrasar-X is no exception, with a new satellite comes a new format. Thankfully DLR has ditched the ever so complicated and CEOS format , which is as not standard as it could be, with simple georeferenced tif files with XML metadata.

This approach is very similar to the data format for RADARSAT-2 , which is quite elegant and richly supported in GDAL (betraying the Canadian roots of the principal author). Now everything would be fine if RADARSAT-2 weren't more that 5years delayed and still scheduled to be launched in this year December.

For the complex data in all modes DLR invented a new format - COSAR. Specifications have been out for a while but no format support was worth building till TerraSAR-X won the race for next generation space-borne SAR sensor in X-Band. Now ironically, another Canadian has added TerraSAR-X format support to GDAL , using the specifications only, which works perfectly for stripmap cosar data. Time to move onto supporting the orbital information and slant-to-ground range conversion.

Wrestling Xerces on MINGW - Qgis GML support

With all the fuss about using XML based standards, GML is becoming increasingly popular as the medium of vector data representation in the GIS community. For the rest there is always KML /KMZ and for the old-timers Shapefiles. The zip and gzip compression algorithms compress text very well and compressed XML based formats are less bandwidth hungry compared to binary formats. GeoServer can produce all these formats for distribution from various datasources, GML is the primary format. GML2-GZIP fares best in terms of size and speed of creation.
After struggling for weeks to link GDAL with Xerces-C on MINGW I ran across this little nugget, a compiler flag that simply and cleanly allowed static linking to parse GML. Xerces-C is not the most elegant, lean and fast XML library, but it is used in GDAL and conversion to another library, possibly Expat which is also a GDAL dependency will require developer resources not currently available. So finally after all the fiddling around I have GML loading in QGIS and hopefully better luck with WFS servers providing it.

Saving World Wind .NET Sessions - The long xml road

XML can be a blessing and a bane. It is the rose that is full of prickly thorns. Original World Wind XML configuration engine used Altova for marshalling and unmarshalling objects. Since then the parsing got moved to MSXML using XPath to remove the Altova dependency, but the configuration saving is not tied cleanly to .NET object serialization system leading to the need the a long-hand major layer parameters-to-file scheme I am working on. May be this can be extended to plugins once there is a common repository for them.

Hopefully this will be standardized to something allowing saving-restoring and network transmission of World Wind sessions, allowing people across the globe to view the same "world" in their respective World Wind installs.

Quaternions, Embedding Python and just passing time

Quaternions are an old and elegant mathematical concept. Superseded in higher dimensions by tensors and other associated constructs but still holding sway in graphics and gaming and mechanics. I have had exposure these fun representations of rotation through both graphics in WorldWind and lately, satellite mechanics, through implementation of optical models in Ossim. There is talk of implementing matching and stereo analysis solution for terrain generation using the optical models within the ossim framework, let's see where it leads.

The other thing taking up my time is getting a Qgis release out. The one with Python bindings is proving to be a real nuisance, making installers in Windows is such a pain mainly due to lack of coherent location of all the dependencies and mess in the registry for locating them even while they are there. Hence the windows package inherently gets larger and larger as I struggle to throw all but the kitchen sink in to ensure all the stuff needed is there and the user does not fall into dll-hell. So far I have packaged Qt,Gdal,Grass,Msys-Mingw,Gpsbabel,PyQt4 and who knows what else into it. I might also have to include an optional embedded python, just in case Python is not cleanly detected on the system. So far Gary Sherman has managed to coax it to work on Windows Vista and Aaron Racicot on Windows XP.

Here are revised steps:

1. Install Python 2.5
   
2. Install PyQt4
3. Install Qgis with Python bindings - will replace some PyQt4 binaries with compatible ones.

The whole beast is now some 62MB-20MB is PyQt4 I had to include. I am looking at adding SIP as well, to get rid of step 2. If need be a simple python distro to rid of 1 as well. Since this will bloat things up there will be a Python-less installer too, as a compromise. I simply wish windows had a better system for binary reuse ( well outside of .NET and the evils of the registry).

When the installer is not building in NSIS I have been raking up gold and monster corpses in Dungeon Runners, lucky it is a fast hack and slash. Cheap ($5 p.m), totally unaddictive (unlike WoW) and free from gold farmers since you can't drop the gold.

Finishing off Polygons - Area Measurement in World Wind

Back on the theme of polygons , another common problem is measuring area of 2d and 3D polygons, polygons on an ellipsoid and polygonal meshes (think realistic farm area measurement for a large farm with hills and valleys).

World Wind now has simple sphere based area calculations suitable for roughly measuring countries, states and something of similar scale. I have finally hooked up the calculations to the measuretool display. The goal is still to allow access to the triangulated terrain mesh and calculate areas with known accuracy for personal properties.

The simple area measurement produces passable results for Metropolitan France ( area 543,965 - 551,695 km² according to wikipaedia article) as shown:

Qgis 0.9 Release and Religion in Space - The re(li)gion outside Earth's Atmosphere

Finally managed to put together a working Qgis installer with GRASS and PyQt4. The major issue was binary incompatibilities between the MSVC builds of PyQt4 released by Riverbank and the MSYS-MINGW builds of QGIS. With the MSVC build of QGIS going ahead these issues will probably disappear by the next release. Aaron Racicot has been nice enough to provide constant feedback and help me fill the holes in the installer.

Coming to the other niggling topic - that of practicality of Earthly religions in the heavenly frontier (or just space). Most of the religious dogma is built with earth-bound mortals in mind. At the ISS in polar orbit with 15-16 sunrises a day fasts and prayer times become rather irregular, as does prayer direction. There have been a few articles about this issue with a Malaysian astronaut heading to the ISS. The official "how-to-behave-guide" bases things on an initial condition/launch site perspective rather than dynamic time. I was disappointed since I was dreaming up a gyroscope stabilized Kabba facing prayer mat for orbital. Anyway it is good to see dogma and rites being shredded by zero-gravity and fast-moving space-time and the core of the belief system being exposed. This is in the wake of reading some Richard Dawkins evangelism, he is just as bad and narrow minded in his way as the rest of the religious folk.

New solutions to old problems - Point in Polygon

Some algorithms in computer graphics and topology never get a satisfactory solution that is both fast and logically simple to implement. One is line segment intersection , the other is the classic point-in-polygon (PIP) , fundamental to object selection / picking. I was attempting to make an implementation for World Wind for generic PolygonFeature picking.

Polygons rendered can be quite complex with holes and are curved to the earth's surface. 2D-topological solutions may not be adequate in all cases and DirectX based object selection may not be accurate. There are multiple strategies for PIP solution, here is the summary of a few:
1: Ray-Trace Test
2: Winding Number Test
3: Successive approximation using grids and bounding boxes
4: Triangle test

Not all strategies work for complex polygons(re-entrant,concave, with holes and what have you), and need to be carefully chosen for intended use and efficiency.

The strategy I am proposing for World Wind will be bounding box checks then rigorous checks using winding rule for holes and outer ring, that is if I don't get lazy and choose to use NTS instead, or just tag any quads being rendered using Direct-X ( filled polygons at altitude) and use DirectX picking.

Visiting the Hurt - RIP Little Green Frog

I have been trying to distract myself by trying to get a working Qgis release ready and writing my PhD proposal. Other than that I am on a tight budget and visiting the sick at the hospital. I may go broke paying reparations.

I visited the tow yard to pick up stuff from the car and the full extent of the damage sunk in. Here are some disturbing images, not meant for the faint hearted. If anyone has any contract jobs , I am happy to take them on. I am badly in need of a new car and some extra cash.

Car Crash - The Aftermath

1 word of advice, DO NOT crash your car, especially with a passenger. It has been a terrible week so far. Crash, trouble with Visa for Holidays, uncertainties in PhD application.

I have been wading through knee deep in forms for personal injury and damages to 3rd party's car. Paying towing fees and fines. Hopefully the insurance will honour the claims and I will not be in debt for the next few years. Also hopefully no demerit points will be taken and I will keep my provisional licence.

The visa may just get delayed enough for me to be unable to get it before I go on holidays, canceling half of it and hitting me with the hotel bill where I will not be staying. It is just not my week.

The Crash

No it's not the server. Rain and my bad driving skills have set me back once more. I turned right in front of a car going 80kph. Ended up being spun out of the road into a ditch , I walked away unscathed but the car is a write off and a friend did not fare so well - broken wrist and hip. I spent the weekend at the hospital.

To add to the spookiness it was the birthday of the driver in the other car and a magpie was eying me strangely while I got into the car, it would not fly away , it stood right at my feet looking up. Talk about signs ...

So happy together - GYM side-by-side

Finally overcame all my prejudice and added the Google Geographic Search to the World Wind PlaceFinder via the REST API. It is all quite nice and simple to interface and highlights the theme of World Wind as a geographic data browser, to stand shoulder to shoulder with all the html-browsers like Firefox, Opera, Safari and oh-so-notorious Internet Explorer. In the same spirit it should offer equal footing for all the major players to have a presence, we don't think twice about having MSDN, Yahoo Movies and Google Search open in tabs next to each other. An independent geographic browser should offer similar visibility and democracy to those willing to publish data in an understandable format. FlashEarth and OpenLayers demonstrate this bringing together with the imagery as well as the search API, but I guess World Wind will never be able to publicly do the same due to legal wrangles.

A geographic browser should allow server side(image servers/feature search/routing/traffic) applications as well as client side extensions(scripts/plugins/xml layers/movie making) and World Wind to most extent fits the bill. The .NET version is the Firefox of the geobrowsers, with all features thrown in but left open to change(or is it SeaMonkey), while the Java version is Gecko, a barebones multi-platform geographic data rendering engine that you can drop in where you wish.

Now coming to the head-off between the 3 geographic search API's , having them side by side like that is really neat for a comparison. In my personal opinion Google allows the best free-form entry of addresses , producing valid and very accurate results. Here are a few things I tried:

• Incomplete entry - Try calif instead of California , Google reads your mind and delivers.
• Misspelling - Google does a spell check and suggested correction
• Different address format - 182 Murray Street, Tanunda S.A. 5352 ends up in USA in VE but in the place I wanted in Google (possibly because they are tracking my IP, time to get paranoid later).
• Post Office accuracy - Google consistently located post offices when supplied just town names, where as Virtual Earth picked a random spot in the town.

Not that I am biased , just do your own tests and see which placefinder gets you what you want, another excuse to try new World Wind where you can find them all side by side.

Google-Digital Globe Repositioning

Digital Globe is now in a strong position with a successful launch of a functional new high-resolution satellite. Google has got some unwarranted free publicity out of it as well. This has led to some market positioning and jostling between the two entities , with Digital Globe trying to push their new acquisition GlobeExplorer as a direct imagery access service. Everything seems to be resolved now, but Digital Globe seems keen to get back to their tried and tested money from imagery sales business model, this time peddling access to global mosaic with a few extra ribbons such as online processing instead of individual QuickBird Scenes.

The heavily watermarked public WMS works cleanly with World Wind for those pining for QuickBird data, albeit a bit slowly ( we are thinking of putting a tile-cache in front of it). Here is what it has to offer over Sydney.

Engineering Awards Night

Another occasion to suit up and flash the red-tie - Engineers Australia Annual Awards Dinner. It was quite fun with acrobats from Cirque du Soleil and comedian waiters. I renewed my acquaintance with a few friends I had met last year at the dinner ( I must be a hermit , I meet people once per year).

Then the rest of the night was spent dancing Swing and variations at Boho. Turned out to be quite fun after all.

Merits of doing a PhD

I am considering doing a PhD on L-Band Polinsar . There is quite a bit of groundwork now in this field with the repeat pass systems from DLR E-SAR and ALOS-PALSAR. I have worked with data from both of these sensors in the past and doing a PhD will be natural continuation.

On the other hand there are disadvantages of doing a PhD too early in the career without an explicit academic bent, or even worse a PhD constrained by commercial interests( not that anything in life is ever unencumbered by it). If anyone wants to offer me any free advice to help me make up my mind feel free to do so.

WorldView-1 Launch and what to do with it

The big news today in satellite imaging circles is the launch of WorldView-1. The launch videos are on the Boeing site. The not so cognizant seem surprised that WorldView-1 is a panchromatic only sensor. Panchromatic captures greater energy due to the broad filter and uses it to produce higher resolution. WorldView-1 probably had a MultiSpectral sensor as well which is rumored to have suffered some sort of accident during the build and was not replaced due to time and budget constraints.

The imagery will go on priority basis to NGA , people-friendly colour can easily be obtained via fusion with the high quality MultiSpectral from Quickbird via virtually automated chain in minutes if need be. The only problem will be temporal decorrelation producing artifacts due to:

• Moving objects (cars,boats,people - they will be clearer at 50cm)
• Shadows due to different sun angles
• Cloud position differences

Hollywood was still popular in the black and white days as long as the actors did their job and there was a reasonable plot. I have little doubt about the popularity of WorldView-1, the quality of the colour products will depend on the orbital match with Quickbird and that will become clearer once the TLE's are published and both of the satellites are being tracked.

Sharpen-Blur and Brightness-Contrast via Pixel Shader 2.0

Using pixel shader 2.0 more complex operations can be performed on World Wind texture tiles.

The spectral contrast stretch operations involve converting from RGB to HSI space and adjusting the intensity with a multiplier and offset, then converting back to RGB. This involves a quite long and complex pixel shader. The results are shown here.

Spatial enhancements such as blur,sharpen or any other convolution oriented operations involve look up using texture coordinates and are dependent on texture size. I have implemented blur and sharpen, hopefully arbitrary 3x3 convolutions soon with parameterization of the matrix. Notice how the jpeg artifacts are affected by sharpen and blur.

Quick and Fast Image Processing in World Wind - via Pixel Shaders

After a brief foray into GLSL shaders and suffering a beat back from the not so good GL support on my laptop graphics card I went back to HLSL and .NET World Wind hoping to implement some simple image processing via pixel shaders and expose the shader parameters via the QuadtileSet properties GUI, much like in NVIDIA FxComposer.

It must be noted that with little hard drive space limits and high bandwidth it is possible to have full-resolution imagery in World Wind as zipped dds, png or even tiff tiles. These can then be processed via a pixel shader to perform a range of enhancements. Spatial operations such as sharpen, blur or unsharp-mask will require reverse texture coordinate lookup and Pixel Shader Model 2.0, but a lot of high speed in your graphics card enhancement of imagery is possible.

I tested out an old shader from Stephan demonstrating multitexturing and started on one for dynamic some brightness-contrast enhancements on a per imagery layer basis.

Weekend in Barossa - Happy B'Day Fleur

Just back from the weekend of camping in the Barossa Valley Wine Region, organized by Fleur for her birthday. Happy birthday Fleur!! I definitely recommend her as a travel organizer/tour guide.

Saturday I had to wake up early and drive for an hour and a half through the scenic Adelaide and Barossa hills to get to the campsite in Tanunda. This is the longest non-stop drive I have ever done in my car and that was quite exciting. I sped about like crazy with the car loaded up with camping gear. It took time setting up all the tents and buying food for dinner.
The rest of the day was spent winery hopping and the night was spent around the campfire talking non-sense to random people. When I finally went to sleep, I just barely felt that my feet were very cold.

The next day we went for a spot of real wine tasting, mostly because I was driving and could not drink any just smell and swirl it around in my mouth. We found some really good wines at Rockford. It is a small winery with lots of old barrels lying around and a cobwebbed ceiling, but they make very fine wine. Then off to Grand Burge with beautiful sceneries matching the great wine. Some of the aged wine was apparently just found in the cellars from times they did not have permanent records, it was at least 25 years old and tasted delicious. I am sure that will not stand up to a breathalyser test, but luckily I did not have any. Finally we passed the Craneford winery which is owned by Fleur's family for some personal attention, and some jockeyed for mates rates. Now I am back home but the camping gear is still filling up the car, the wine is out and is safely in the dark corners or the fridge as appropriate.

May be I will get some real coding done thanks to the break. Drifting off into vertex and fragment shaders from the land of wine and olives.

World Wind Java and Shader Trickery

Just to keep up my end I have been doing regular release builds of Qgis lately. Then I had a look at some of the prettier shader applications in World Wind Java. Now I have new goals. I pulled out Render Monkey and the big thick GLSL Tome.

I have the following on my list:

1. Layer opacity control via fragment shader
2. Gradual layer transitions as user zooms in, basically eye distance based opacity
3. Raster enhancement operations - such as RGB to Grayscale, sharpen, contrast stretch etc.
4. The old favourites sun shading and day-night, this already seems to be there is a simple way in render monkey.
5. Global fluid water layer, hopefully a simple spherical mesh with a simple vertex shader or varying bump map.

Meanwhile I am waiting for the 0.3.0 release to get a stable start, a lot seems to have been already achieved. At some point I need to start on some decent KML support, I already have a schema based parser from castor, pity so little KML in the wild actually follows the schema.

Qgis 0.9 Preview Installer up

I put together an NSIS based installer(long removed) for Qgis 0.9 Preview 1 with some primitive Python detection support for testing, it will complain while loading the Qgis Python bindings if PyQt4-Sip are not located. It should work with the Binary package for PyQt4 available here. I will try to add auto-install of PyQt4 soon.

Post after Long Silence

It has been a while since I last posted, why ?

Well I changed browsers, Firefox was taking up too much memory making it hard to perform those giant ossim links, so I switched to Opera and Blogger has pathetic support for Opera.

Anyway in this offtime I managed to attend a charity dinner, fall flat on the dance floor, get on the Qgis release team and contrary to popular misconceptions got Qt 4.3 to work with MSVC++ Express Whidbey(patches) and compiled OssimPlanetQt on this platform. It works just fine.

The Qgis 0.9 Ganymede ( my name of choice is in !) Release is undergoing tests and you can pick up a zip archive of binaries here. There are some issues with the PyQt bindings , please leave feed back on your experience here or on the Qgis mailing list. I am working on automatic detection and install of Python2.5+PyQt in the NSIS Script, any snippet suggestions for doing this are also welcome, especially from the PyQt folk so that I know where I am standing.

Generation-Y Employees

I have never been really keen on generation nomenclature. I am never sure whether I am in generation X or Y. But I am concerned when surveys show a certain bias towards people of my generation. This came to my attention through my Engineering Association newsletter. So I dug up the original article from smartcompany.com.au.

The gist is that younger employees have a high technical skill level, but lack corporate slick. Their bosses increasingly rely on them to provide technical solutions and solve the problems that were impossible 1year ago but are probable now( due to pace of technology movement). The bosses then complain about the high expectations and low loyalty of the younger employees.

Over all the article is not all negative and puts forward a few useful suggestions for making the most of Gen-Y employees.

''The key issue for employers, then, is not whether to hire a member of Gen-Y, but how to make the most of them.''

BigTiff support coming to LibTiff 4.0

Thanks to the input from Joris and others various enhancements from the BigTiff proposal are now part of LibTiff. Hopefully it will soon get the blessings from Adobe, who own the Tiff Standard and become part of the official Tiff spec and encourage wider adoption.

Adobe seems to be aware of the issue with the 4Gb Tiff barrier and looking into already implemented enhancements such as by Aperio. John Nack points to it in his Adobe Blog. Hopefully Photoshop will soon take this up and implement a tiling scheme for images bigger than 30kx30k pixels.

Meanwhile I am busy shuffling BigTiff into my processing stack. Testing it with Gdal(Support works fines) and Ossim(Support not yet stable). If anyone wants to help in the testing with other applications grab some sample images here.

Qgis 0.9 to be released soon

Tim is warming up to release Qgis 0.9 before FOSS4G 2007. I am happy to report that it builds easily and cleanly in windows thanks to CMake with all the goodies and the much anticipated Python support via PyQt-Sip.

The Python API makes quick development and debugging of Qgis plugins as breeze. Quite a few have already been made to show the possibilities. To support the release ( and to plug my name choice for the release) I made up a new splash screen. I will welcome any criticism of my design's shortcomings.

Sweet and Sour Pork ... return to World Wind coding

I have been taking a break from coding World Wind. With the Java version building up and a lot of things on my plate, I was not sure where to concentrate. Knowing the code base well enough I can quickly solve problems and add features in the C# version. So I touched up a small section of KML Importer today for Image Overlays, now hopefully all sorts of image overlays work(local,over http and embedded in kmz). The rotation tag still needs working and I would like to add handles for editing the overlays as well from the Image Overlay plugin. The other nice upgrade would be support for multiple KML files.

Anyway to celebrate this small achievement I tried out the recipe for sweet and sour pork. The results were delicious and rewarding.

M-Party

Went to Marine's farewell M'Party. The dress up was hilarious , including those not willing to dress up and turning up as 'Me' . Overall there were some pretty good costumes and I attracted plenty of attention strolling home in my magician gear across the crowded Saturday night streets, pity I had to give away my and wand and kanga-rabbit.

The party was taken over by Martians of both sexes and Mummies till the MIB came to rescue.
But they let their hair down and messed up, even the 3 Maids could not get the alien dust off.