Open-source Sustainability (The tale of 2 package managers)

Last weekend I had the priviledge to attend the 10th PyconAU and listen to some amazing speakers. I went with my I will write markdown on the fly and make a blog-post at the end of the day mindset. Even though I did write a lot of markdown on the fly, I haven't gathered the courage to push these unedited notes into a public post. Excellent examples of live-blogging from conferences here.
What did happen was that the niggling doubts I had around how open-source works in the real world outside of just the code crystallized. This was as a result of 2 very good talks , one about how the PyPi project works and another around Open-source sustainability beyond money.

For the last year I have been writing and reviewing a lot of React Frontend, Python backend (Flask/Django) and Notebooks code. Both frameworks are super easy to buy batteries for where the included ones are running out of juice. Simply via pip install and npm install you can climb onto the shoulders of giants who are library maintainers and the life-blood of lean start-ups everywhere. However the maintainer burnout is a thing and start-ups when building their stack should be highly cognizant of this. Package repository burnout is also a thing. In my time in the software industry I have seen Maven repositories disappear. More recently NPM go through an identity crisis and the left-pad incident.

The PyPi talk gave me great background on a tool I use every single-hour without too much thought. It takes some dedicated volunteers to keep the dream alive. Who according to Dustin Ingram are :

• Unemployed and bored and poor (but super talented) 
• Paid for by their employer (thanks employers who support FOSS) 
• Not getting enough sleep (or in my case time with the family)

Vicky's talk covers another aspect. Developers and maintainers need more than money to keep going, they need back-up. The community need insurance against the bus-factor and burn-out. I have been guilty of this myself, putting a few dollars behind features I would like to see in BountySource instead of diving in. This has become more so as I have progressed in my career and become increasingly time-poor. Talking about this would anyone at VSCode like to claim the few dollars we put here ?

I love the longevity and discipline of project warehouse and will find some time to contribute to it. I also look forward to a similar alternative to npm, rather than a caching proxy with community behind it.

Nairobi 2nd OSHW meetup

After a many month hiatus we finally put together the 2nd Nairobi Open-source hardware meetup. This time we were at the Tav Irish Pub near Nairobi Garage on Waiyaki way. The pub was chosen due to proximility to Nairobi garage and possible use of that venue. As it turned out none of us were memebers paying the $60 per-month fee so we ended up co-opting the pub in Chicago 3H fashion.

PCB Lineup

I had collected PCB's over the last few month from Aisler, OSHPark and PCBWay all shipped to Kenya from Germany, US and China respectively. So we lined up to try various soldering techniques on 0603 Jellybeans and some 20pin TSSOP IC's.

Soldering Demo

The techniques we we tried were the tack, finish and reflow one using the iron and an experimental one with paste and dry-iron I had picked up recently. I didn't have a temperature profile controller for the dry-iron method. So we were just going to wing it by the eye.

Part Placement

We pasted up the boards using screwdrivers and tweezers. Started the reflow till the parts settled. Then simply turned off the iron. I am sourcing a K-Type thermocouple and SSR from KTechnics to get a better handle on this for next time. However the dry-iron does reach sufficient temperature for leaded solder reflow.

Dry iron reflow

By the end of the session we had assembled x2 CS5464 Breakouts. Next step from here will be suitable MicroPython and Arduino drivers to ensure these things actually work. We had a lot of fun over the meetup and I hope to continue having these sessions. Next time one of the participants has promised to bring his Weighbridge Automation and IoT set-up.

Scavenging a Coffee Machine

Following the meetup I have been hunting for accessories to make the dry-iron reflow less of a fire hazard and more controllable. (Un)fortunately the coffee machine at work has been fried by the spikes prevalent in the grid in Kenya (especially during the rainy season), so I have a spare K-type thermocouple I can work into the build. I have also co-opted a tissue holder we had on our dining table to stabilize the inverted iron during reflow.

Stand of opportunity

Onwards to the next Meetup.

Making PCB protos in Kenya

In Adelaide I made a lot of prototypes PCB's and filled up filing cabinets and shelves with them. Some had value to others and I sold them on Tindie and CrowdSupply. For the last year I have been hanging out in Kenya working at Lori. The output in terms of open-source hardware has taken a back seat compared to proprietary company code (we did release one open-source piece of ETL code).

This is near the end of my 3-month stint in Kenya and I decided to run an experiment in the feasibility of designing and assembling prototype batches of PCB's for an energy monitor ASIC I have been meaning to try out - the CS5490. This is a super low bar for a test run. The IC has very few dependent components and large pin pitch making it ideal of teaching manual SMD soldering.

PCB Design

Once the goal was set I broke out KiCAD and spent a night and a Sunday churning out a basic breakout board. The PCB Design and BOM are published here.

Order PCB

The harder part begins once the PCB design is done. Actually making this a physical reality in Kenya. I placed the usual order via PCBWay. A few days before Christmas before I went off to a round a country drive with some friends ( awesome content for another post). The shipping would be from Shenzhen via DHL. The logistics promises arrival by the 28th December.

Order Parts

The BOM for the design is not super complex either. Mouser covers the Kenya region. However there is no option for free DHL for orders over $60 which I get in Australia. I will be in Kenya only till the 20th January and the streets have no numbers or postcodes, which makes ordering via snail mail for any international shipping simply not fit the address template most places have. Perhaps global addressing via plus codes will become reality in future eCommerce systems. Mouser shipped the parts out of Texas immediately, I ended up paying more on shipping than for parts. The customs magic was yet to come.

Clear customs

Any packages with value below $1000 sail through customs in Australia. The Kenyan government will lose out on significant revenues if they followed this practice. On the flipside young engineers with limited resources are being stifled in self-driven learning and experimentation by this practice. I got a detailed/itemized bill from customs via DHL which ended up being the same cost as PCB + shipping for PCB's and around 20% of the cost of the parts. See file here for details and breakdown.

Source tools and materials

Thankfully this bit was much easier with a couple of local soldering iron , jumper wire and tweezers suppliers filling the gap. The ball was dropped by Fargo in delivering the stuff I ordered online again due to the addressing snafu, they attempted 5 deliveries to the wrong place before I had to redirect to the office address where a lot of parcels are received. I can recommend Nerokas and Ktechnics for getting parts in without hassle. Again better last mile logistics would have made things faster. I ended up attending the assembly meetup without the proper tools.

Find assembly space

To avoid annoying my landlord with solder fumes and messy paste workbench I started looking for a place to get together with a few interested parties and assemble the boards (at least one). I ended up starting an open-source hardware meetup and getting some Mang'u engineers together to hand solder the parts. 

1 PCB - x3 assemblers and a gas stove from a local shop. pic.twitter.com/S09EZJbZ5k

— Tisham Dhar (@whatnick) January 5, 2019

Test and publish drivers

Finally this Sunday I have put together some basic micropython code to read registers from the CS5490 over serial and display them in a micropython notebook. Nothing production ready but proof that the PCB design and assembly worked as expected and we can achieve custom products here, with a lot of bureaucracy and false starts.

I am happy to meet more people designing and building open-source hardware in Nairobi and hear about their experience.

International Space Apps Challenge - backing NASA

Last week I had the chance to participate in the inaugural NASA Space apps challenge, modelled after the random hacks of kindness which we see taking place whenever there are large natural disasters(earthquake in Hawaii) or to address malignant global social issues such as inequality and poverty. It is amazing to see the drive people at such events have to work on something that does not directly impact their everyday life.

Space exploration is one such outside context problem. It seems frivolous on first sight till we study a bit of history. Our current global economy is based on the last 500 years of exploration (and should I say colonial exploitation). The rules of economics are framed such that they require constant expansion of resources - raw materials, energy and ideas. Being in a closed system on the planet poses inherent limits to the amount of expansion we can achieve, in a similar way industrial revolution was bound by the border of colonial powers in Europe - they were forced to explore new worlds. Since the Earth is nearly fully discovered now and everyone agrees on very limited amount of exploitation being the norm, to achieve the growth our mindsets demand we need to expand beyond Earth - ergo space exploration and asteroid mining. By participating the the space apps challenge, we are taking a very small step towards pushing the material and energy exploitation off planet and relieving the pressure put on our bio-sphere and humans in highly populated countries by blind greed.



Philosophy and sociology aside, we worked on a very broad problem for the challenge - developing a Semantic Data Descriptor for NASA data holdings. NASA has a lot of past, present and planned missions addressing the need for better observations of our planet, our neighbours in the solar system and things far far away in both space and time (unravelling the mystery of how the universe came to be as it is today). These missions collect something broadly termed as data, mostly observations. NASA being a fairly siloed organisation, has teams focusing on particular missions and developing their own description regarding the data they collect aimed at subject matter experts. Science in its own way can develop its own priesthood, bent on excluding the "laymen". The aim here was to develop a schema to make sense of NASA data in everyday vocabulary, we basically described all data as being an observation of a target by a sensor, measuring a field or a particle at a point in space-time. Writing that up in RDF required some help from Altova SematicWorks and the presentation some form Prezi. In the end we made it to the global rounds, hopefully our efforts will help the average human being establish their stake on space data if not space itself.

ESA SOCIS - Coding in the Space Space

The summer of code pattern seems to have become rather popular, pity it is a mis-nomer for us Southern hemisphere folk, that's what you get for the earth being top heavy in landmass ( whoever decided they are on top anyway).

This year the European Space Agency (ESA) is running an SOC event, the catch line being a slight variant of the Copenhagen Interpretation. The two projects I am associated with are:

1. Orfeo Toolbox SAR Interferometry support - This one is academically of most interest to me, there have been several attempts to develop the interferometry chain in open source, but they have not been properly software engineered, have dubious licensing or other shortcomings. More on this project and the teething issues encountered later.
2. iGlobe NetCDF support enhancements - This one is professionally of most interest to me. Patrick at NASA recently put us (CSIRO) together with NOAA, NASA and ORNL to build and application tying NetCDF to WorldWind. We are planning to explore large data visualisation techniques and get best possible performance and accuracy. As well as include some analysis capability. The project is code named iGlobe.

These projects will bring my student supervision quota up-to 5 for this year, I have only had them for 3 months and there has been a huge amount work delivered. I wish I had this kind of free labour and velocity at the real workplace.

IGARSS 2011 Vancouver - Redux (So that you don't fall asleep)

It took a lot of messing around and changing flights to get there but thanks to the excellent folk at Flight Centre Melbourne Airport I arrived on time , left Melbourne 25th July Morning 10:30am and arrived in Vancouver 25th July 7:30am. Again proving the fact that a teleporter is indeed equivalent to a time machine.

This monday turned out to be full-on 48hour monday, lots of interesting sessions to go to. I attended the plenary by Shimada San introducing ALOS-2 and ALOS-3 with next generations of the great optical and SAR sensors we have come to expect from the Japanese. May be I can sneak into the CALVAL stage and score another trip to Japan.

More flight dramas continued in the background, the presentations blurred into a constant stream of words and diagrams on my sketchbook in the foreground. I learnt a lot about what has been done in the past year and what is coming over the horizon.

I saw science solving political problems with cross-boundary river basin monitoring with Poseidon, Jason-1 and in the future Jason-2. Important basins across unfriendly countries such as Indus, Ganges, Brahmaputra and the Mekong can be monitored by the international science community and appropriate alerts sent and action taken when one of the partner nations or the weather in the river basin begins to misbehave.

I saw the cold-gas (NO2) driven engines that keep TerraSAR-X and Tandem-X locked in their helical dance, not forever, only till the global elevation model project is complete or the gas runs out - whichever comes first. Some extra effort will be required to collect the elevation model of Antarctica, since the satellites will need to be rolled to face left. This is where the polar observation satellites - Cryosat-2 and more, will make a great impact.

I used the conference services a fair bit as well and got labelled "the troublemaker" - thanks people. If you want to get in touch with me just search for the troublemaker at IGARSS.

The real highlights of the week however were the impromptu dinner get-togethers. After the poster session on Monday I drifted off with SAR people from Spain and a few other places. Tuesday night was spent with the OTB team and a fun Columbian guy from ITC. Scientists on one night, engineers on another, we discussed the transition from Science to Engineering to Products, and the effort and mindsets it takes to go the next step, and how the last step provides continuing fuel for the first. The next night it was back to science setting up a collaboration aimed at just such a transition for the OTB ESA Summer of Code in Space (ESA SOCIS 2011). Thursday evening I had great stroll through the Sun Yat Sen garden, dinner at Shukaku and a movie at the Scotiabank Theatre - ultimate self indulgence. Friday, went to very few sessions and spent most of the time in Stanley park sketching totem poles, before catching a flight out. Thank you for a great time Vancouver - I will definitely be back.

GSOC 2011 - Ship Detection in Opticks

This is my second mentoring stint for the Google Summer of Code. I am advising on 3 separate projects in different capacities.

1. Ship + wake detection and parameter estimation using Radar images in Opticks
2. Wrapping the awesome OSSIM library in python
3. Displaying sub-surface features in a 3D globe - NASA WorldWind

This post is going to be the first in a series describing the goals of each of the projects and where we are so far in them. In my personal opinion given the complexity of the algorithms involved and the visual outputs the ship detection project has progressed the farthest. We are using a beautiful 3m TerraSAR-X image from DLR as the basis for our implementation.

The various algorithms we have thrown in include:

1) CFAR with k-distribution model of the clutter
2) Elliptical width, height and orientation estimation
3) Isodata clustering for ship points
4) Linear feature (wake) detection with radon transform
5) Conversion of ship to wake displacement to ship velocity using satellite trajectory and imaging mode

Vijay is in his 5th week now and has been making great progress, picking up algorithms from various examples and adapting them to Opticks.

Joys of cleaning the house - Nuking non-compliant code

The openkinect project is nascent and as such the win32 platform support is still stabilizing. Due to the naughty behaviour of libusb on windows several attempts have been made so far to create common API across platforms without a simple maintainable and working version emerging.



I decided to learn about kinect, coding on github and merging things in general by taking Zephod's code and putting it in a win32 folder under the library source. I got most of the demo code compiling and running with this approach. I took a master piece of my Ikea chair in depth on Vista 64, apparently a rather difficult OS to get to work on. By then my git code tree had become cluttered with lots of merges, I was working off 3 remotes (my own fork, Zephod and mainline). With some nudging from this video, I deleted all copies of my work into oblivion. I can't control Google cache and such but it felt liberating.

Getting the code out - from Academic to Open Source

I am at the IGARSS conference in Waikiki , Hawaii. It has been a busy few days. All done with my presentations today, 2 posters side-by-side are rather easy to handle. The last presentation I attended today was from Eric Pottier on PolsarPro - a very brave attempt to get academic research in polarimetric SAR processing to the non-specialist user community. He mentioned that new routines and improvements get incorporated into the platform every week, so I naturally asked about the release schedule and possible version control access. This is where the academic heritage showed up, the project in-spite of its research brilliance does not have a hard working software engineer to smooth the getting it out there process. Collation of the work from the various developers and building the final release is done manually it seems.

The folks at Rennes 1 should take a leaf out of the OTB (or even RAT) book and make collaboration easier. In the recent release the simulator that I frequently use is broken and I can't report a bug or track the changes that broke it - version control please.

PS: I created a version control for myself , only for PolSARProSim.