Any web application that targets a mass market and wants scalability ends up in the cloud, rather than in an individual data centre. Of all the cloud providers the biggest currently is Amazon. After running the metromap application with AWS for a year mostly on a self-taught basis I realized that it was time to rub shoulders with some experts and possibly bring in some help.
marketing conference in Sydney and unlike the academic conferences I am used to attending this one was free. So I signed up for it with some work backing.
On arrival I walked into a massive Kogan keynote speech, it had all the pomp and fanfare of a boxing match or WWF rumble. Felt very American.Moving data to and from the cloud was big sticking point as well and the AWS Snowball (80TB) was announced. I have recently ordered one, still waiting for it to turn up. Will write another article on this Data Exchange unit once I have had a real life play with it. Working at an aerial photography company I am used to huge amounts of data coming in from aircraft on similar hard drive units, but none of them feature 10GB ethernet and in transit encryption like the Snowball does.
There was a big focus on IoT capabilities in AWS as well including an Earth, Air, Fire and Water demo with Alexa playing stage manager.
I had a small blast from my CSIRO past. Peter Blaine had a talk about IMOS data sharing. 50Million NetCDF's are up in the cloud. The big challenge being the heterogeneity of the data. Apparently they have moved to the warehouse next to my old office and things are progressing okay. Even though CSIRO is rapidly shedding science staff, the engineering half is still alive and well.Sumo Logic and Puppet which were particularly relevant to my application of image serving and service monitoring.
After a couple of days of hard conferencing and networking I managed to get in some R&R and learn the weird game of shuffleboard and enjoy some cocktails at the Little Darlin'. It was an exciting couple of days in Sydney and since then I have been putting my knowledge to good use. I will write up another post from my mass of notes collected during the summit.