Tuesday, August 31, 2010

Wind profiling with Radars - ARRC

Had the inaugural lecture for the ARRC - Adelaide Radar Research Centre. It is trying to create collaboration between the Physics department and the Electronics Engineering department to make - well more radars of all sorts. The physics department at Adelaide has had several RF greats including the famous Braggs. Prof Iain Reid the Executive Director of ATRAD showed off the long history of wind profiling arrays at Buckland Park and other sites. They seem to have a huge number of installations at Chinese launch sites to control manned launches of the Long March.

Here are some interesting tid bits:

1) Full atmosphere coupled models work better for weather system modelling than lower atmosphere only models.
2) Atmosphere has Inertial,Bouyant and Viscous layers which influence its observability by VHF radar.
3) The region not observable by radar can be observed using lidar at multiple frequencies, there is a sodium band at the upper atmosphere which can be used for measurements. UV Lasers will have best response for most of the atmosphere.
4) Networked radars are the big new thing thanks to the popularity of the BOM site ( following on from the last post) and is driving the development of more wind profilers so that the atmospheric radars can be dedicated to precipitation observation.
5) Big radars are hard to build due to power constraints in remote areas like the outback and Antarctica, so they are being forced to become more economical. Pansy built by Japanese Universities in the Antarctica may include solar power or something similar, or a forest of Bonsai trees with electrical generation units and antennas plugged into them.

On that happy note we finished the talk on watching winds with electromagnetic waves.

Wednesday, August 25, 2010

Time to Rain with BOM Radar

BOM Radar loop is very handy no matter what you are planning, a bike ride, a dash to the bus stop or a picnic with your girl friend. The images are nice and you can use your good old brain to work out the windows of rain free time. I would like to convert the rain from spatial views to time slice views at my location and forward shift to work out when the rain will hit me, just so that I can time that 10 min slot to go and get some lunch.

This will involve working out the velocity of the rain band and current location to estimate the time of arrival. The loop provides 4 or more images, these can be used to estimate the direction of travel of the rain band together with wind velocities reported.

You can get individual Radar frames over Adelaide with the following format
  • URL Base: http://www.bom.gov.au/radar/IDR642.T [x.T where x is the resolution level - 2,3,4]
  • Date code: YYYYMMDD in UTC
  • Time code: HHMM in UTC 
Only last 4 frames can be directly obtained from the server. Not to worry, I am sure someone is running a cron job somewhere and backing them up. Once you have sufficient frames to establish a time series the fun part begins. Here is a Python script to fetch the last 4 frames over Adelaide, be conservative and run it every 5 minutes ( just to satisfy Nyquist) and fetch the latest frame.

The rainfall is colour coded so we will need to perform a look-up to convert from colour to rain density. Next a quick FFT based convolution for the 4 images to estimate x and y shifts. You can also use a shift and difference approach while minimizing the difference to establish the rain velocity. Statistically test the results to establish constant velocity . If the velocity is not fairly constant our prediction accuracy falls off. Then the time machine part, propagate the last image through the shift field and estimate the rain intensity at the location of interest for the next 30minutes to 1hour, whatever the horizon of the radar and rain velocity lets you do.

Obviously all this can be done much easier from within BOM since they have the data at hand, but it is much more fun mashing a separate application together.

Wednesday, August 18, 2010

MPICH2 - Joys of windows MPI

LAM/MPI is showing its age on our Ubuntu cluster. LAMD dies at times and we always seem to have more windows machines than linux pxeboot boxes. MPICH2 may allow heterogenous clusters. Here are runtimes with different communication protocols on MPICH2 and LAM with the trivial prime tutorial.
forest image
forest_image

MPICH2 : [FAQ]
sock: 4.562287
nemesis: 0.403781
ssm: 0.641682
mt: 4.793552
default: 4.489148
auto: 4.200369


LAM/MPI : [FAQ]
usysv: 0.260094
sysv: 1.899345
tcp: 1.965503
crtcp: 2.089337

The test app was built with CMake and the very handy MPI detection with MPI choice options.

On the other hand the CMake setup for polsarprosim is progressing very quickly thanks to the experience with Qgis, OTB and lots of advice from Norman and Emmanuel. Here is a what the automated testing says:

Test project C:/Program Files (x86)/PolSARpro_v4.1.5/Soft/build
 Start 1: compareOutput
1/2 Test #1: compareOutput ....................   Passed    0.07 sec
  Start 2: runSim
2/2 Test #2: runSim ...........................   Passed  3919.94 sec
100% tests passed, 0 tests failed out of 2
Total Test time (real) = 3920.05 sec



Now need to try some MPI to speed up the simulation time a bit and put the simulation run and output comparison to a standard in the proper order.

Wednesday, August 11, 2010

Writing PhD thesis and living in a hole

I have finally moved house to a crummy student digs to go with the PhD writing mood. Most of my time is spent with WinEdt and latex syntax. I might think of switching allegiance to LyX since Bibus has added Lyx support, I wonder if it will mangle my tex files into LyX specific format.

I have also been doing some TerraSAR-X quad-pol data analysis and comparing decomposition results to dual-pol case. There is a very nice overlap, but some things seem more coherent when the cross-pol scattering is not taken into account - in my case tree trunks standing on water.


I did manage to sneak in some time for open-source and check out the great work Manuel is doing to incorporate Monteverdi support into Qgis. Monteverdi can pull the layers from Qgis open files list and process them. The results currently cannot be passed directly from Monteverdi into Qgis. This issue can be solved by using a custom otbprovider which pushes Monteverdi streams into the QGis view by producing an appropriate QImage.

We have also been wondering whether an application is running inside our outside a NAT, what would be the best way to check this. Say ping an external server outside the NAT and see which source address the request came from, if the source is one particular IP (we have a static IP) then we are inside NAT and need to use internal IP's to connect to certain servers, otherwise we are outside and public IP is fine.