Fusing channel - array of cheap camera

I decided to do another experiment with my multitude of USB webcams and Gstreamer. This time I am capturing 2 shots using the stereo config I used before, but with an InfraRed filter with 850nm as the pass-band over one of the cameras. The rig looks nothing spectacular but costs only $40 net.


There is some parallax and neither the camera nor the mount is calibrated. Registration of the multi-band images is purely by trial and error at this stage. I have written it up with matplotlib, numpy and PIL to allow more automated transform estimation.


The overlapped section is cropped using array subsetting and channels are reassigned as InfraRed , Red , Green instead of plain RGB to form a False Colour Composite. Some of the misaligned sections can be seen highlighted. I also have 950nm and 750nm filters to test more detail in the spectral response. The compositing will obviously become harder and harder as a manual process till I build a stable mount, but I kind of like the extensibility with clip-on-mounting.

The solution is to use the well developed feature based image matching techniques and let feature descriptors like SURF, SIFT or Harris Corners pick up the bits I picked up by hand. The spectral variation provides additional challenge.

I finish off with the composite built so far.  Low costs systems like this can be used for environmental studies where lugging large cameras and spectrometers around rough terrain is not pleasant.