Are you artier than a fifth grader ?

Once upon I time while I still used to watch TV (instead of the world go by, which is much more interesting), Rove used to host a show called "are you smarter than a fifth grader". It seems that it is still on. The show mostly tested textual and numerical skills or pure recall (very left brained). I have recently started experimenting with drawing techniques and I hear "My 5th Grader could have done that" quite often. It is so nice to see young talent, and I do indeed have some pieces from the open-sketchbook project proving the beauty of children's artwork.

So I am putting forward here a right-brained show in the same vein. Put an established artist and a 5th grader in the same stimuli field and let them each produce a piece. Then do a double-blind test with a jury of their peers (both age groups need to be represented since art is highly subjective), on which one is better. It may get hard to keep the stimuli field interesting for the adult and the child at the same time. I did one of my coolest subliminals while watching Final Destination 5.

I believe the education system currently established is very asymmetric and a right brained show may help highlight this or prove me wrong ( none are so blind as those who will not see).  Just to keep it clear on my belief in children's abilities, the bits of TV (Time Tunnel) I do watch is recommended by kids I meet while walking to Salsa.

ESA SOCIS - Coding in the Space Space

The summer of code pattern seems to have become rather popular, pity it is a mis-nomer for us Southern hemisphere folk, that's what you get for the earth being top heavy in landmass ( whoever decided they are on top anyway).

This year the European Space Agency (ESA) is running an SOC event, the catch line being a slight variant of the Copenhagen Interpretation. The two projects I am associated with are:

1. Orfeo Toolbox SAR Interferometry support - This one is academically of most interest to me, there have been several attempts to develop the interferometry chain in open source, but they have not been properly software engineered, have dubious licensing or other shortcomings. More on this project and the teething issues encountered later.
2. iGlobe NetCDF support enhancements - This one is professionally of most interest to me. Patrick at NASA recently put us (CSIRO) together with NOAA, NASA and ORNL to build and application tying NetCDF to WorldWind. We are planning to explore large data visualisation techniques and get best possible performance and accuracy. As well as include some analysis capability. The project is code named iGlobe.

These projects will bring my student supervision quota up-to 5 for this year, I have only had them for 3 months and there has been a huge amount work delivered. I wish I had this kind of free labour and velocity at the real workplace.

Making NetCDF Faster, microbenchmarks on macro .nc

We have several very large NetCDF datasets with lots of variables to visualise. I have a gut feeling that file sizes are an issue and there is a bottleneck at the disk to RAM stage, but how do I prove it ? By writing a set of micro-benchmarks to test the various storage and access options. The options are:

1) Storage pattern - NetCDF allows vanilla NetCDF3, Lossy Packing, Lossless Deflation, Chunking and combinations of the aforementioned schemes for storage. If data fidelity is not an issue (as in the case of materialised overviews) , my benchmarks indicate that Lossy Packing is the best way to go.

2) Access pattern - The beast under scrutiny was 27GB RAMS Atmospheric model with 50+ variables. The biggest variable chews up 2GB of RAM if you load it all at once. Full data load takes 1000s, while 1/16th load in all dimensions takes 600ms. If just edges are read to perform a bounding box style render that is quicker as well. The exact access pattern used will depend on the volume rendering technique chosen. There are also asymmetries in accessing the beginning and ending of the variables. This indicates linear seeks across files.

3) Access Path - Still need to compare access via the C-library+JNA/JNI rather than pure Java. John Caron seems to have done it a while ago, but I am still agonising over how to activate this experimental code in the current version of the library.

PS: I have got the JNA code working with NetCDF-Java and started a dialog with John.