How do you quantify cloudy?

Its surprisingly difficult to autonomously define cloudiness. Even more if you subject is slightly reflective. Lets start with some pictures, everyone loves pictures.

This is the shard quite hazy, but clear:

This is the shard with the peak in the clouds, but with bright sunshine below: Then we get to night time, Here is a fairly clear night: And a slightly misty one: As you can see by this tiny sample set, there is a lot of change. I initially looked to do simple histogram matching. Get a sample of a sunny day, look at the histogram and boom a metric to tell you how cloudy it is. Well that doesn’t work. The outside world is far from uniform, the sun moves, the sky changes colours and the camera is quite noisy.

After looking at how to compare histograms properly my mind melted. I still needed to collect a good sample of cloudy picture. To do this I hacked the simple histogram script that ships with pyopencv: (usage: python scriptname.py imagefile.jpg)

#!/usr/bin/python
import cv2.cv as cv
import sys
#import urllib2
#the original author really likes "_" instead of camelCase, which is a pain.

hist_size = 64
range_0 = [0, 256]
ranges = [ range_0 ]

class DemHist:

def __init__(self, src_image):
self.src_image = src_image
self.dst_image = cv.CloneMat(src_image)
self.hist_image = cv.CreateImage((320, 200), 8, 1)
self.hist = cv.CreateHist([hist_size], cv.CV_HIST_ARRAY, ranges, 1)

#keep a copy of the original image so we can crop from it
self.orig_image = cv.CloneMat(src_image)

#define class globals
self.brightness     = 0
self.contrast         = 0
#make sure we've make the first crop the correct size
self.crop_height     = self.orig_image.height
self.crop_width        = self.orig_image.width
self.height         = self.orig_image.height
self.width        = self.orig_image.width
self.crop_pos_x        = 0
self.crop_pos_y        = 0

cv.NamedWindow("image", 0)
cv.NamedWindow("histogram", 0)
cv.CreateTrackbar("brightness", "image", 100, 200, self.update_brightness)
cv.CreateTrackbar("contrast", "image", 100, 200, self.update_contrast)
cv.CreateTrackbar("Crop Height", "image", self.height, self.height, self.update_height)
cv.CreateTrackbar("Crop Width", "image", self.width, self.width, self.update_width)
cv.CreateTrackbar("Crop X", "image", 0, self.width, self.update_crop_x)
cv.CreateTrackbar("Crop Y", "image", 0, self.height, self.update_crop_y)

self.update_brightcont()
def update_crop_y(self, val):
self.crop_pos_y = val
self.do_crop()

def update_crop_x(self, val):
self.crop_pos_x = val
self.do_crop()
def update_width(self, val):
self.crop_width = val
self.do_crop()
def update_height(self, val):
self.crop_height = val
self.do_crop()
def do_crop(self):
self.dst_image = cv.GetSubRect(self.orig_image, (self.crop_pos_x,self.crop_pos_y,self.crop_width,self.crop_height) )
self.src_image = cv.GetSubRect(self.orig_image, (self.crop_pos_x,self.crop_pos_y,self.crop_width,self.crop_height) )
cv.ShowImage("image", self.dst_image)
def update_brightness(self, val):
self.brightness = val - 100
self.update_brightcont()

def update_contrast(self, val):
self.contrast = val - 100
self.update_brightcont()

def update_brightcont(self):
# The algorithm is by Werner D. Streidt
# (http://visca.com/ffactory/archives/5-99/msg00021.html)

#if self.contrast > 0:
#    delta = 127. * self.contrast / 100
#    a = 255. / (255. - delta * 2)
#    b = a * (self.brightness - delta)
#else:
#    delta = -128. * self.contrast / 100
#    a = (256. - delta * 2) / 255.
#    b = a * self.brightness + delta

#cv.ConvertScale(self.src_image, self.dst_image, a, b)
cv.ShowImage("image", self.dst_image)

cv.CalcArrHist([self.dst_image], self.hist)
(min_value, max_value, _, _) = cv.GetMinMaxHistValue(self.hist)
cv.Scale(self.hist.bins, self.hist.bins, float(self.hist_image.height) / max_value, 0)

cv.Set(self.hist_image, cv.ScalarAll(255))
bin_w = round(float(self.hist_image.width) / hist_size)

for i in range(hist_size):
cv.Rectangle(self.hist_image, (int(i * bin_w), self.hist_image.height),    (int((i + 1) * bin_w), self.hist_image.height - cv.Round(self.hist.bins[i])),cv.ScalarAll(0), -1, 8, 0)

cv.ShowImage("histogram", self.hist_image)

if __name__ == "__main__":
# Load the source image.
if len(sys.argv) > 1:
src_image = cv.GetMat(cv.LoadImage(sys.argv[1], 0))
else:
sys.exit()
#else:
#    url = 'http://code.opencv.org/projects/opencv/repository/revisions/master/raw/samples/c/baboon.jpg'
#    filedata = urllib2.urlopen(url).read()
#    imagefiledata = cv.CreateMatHeader(1, len(filedata), cv.CV_8UC1)
#    cv.SetData(imagefiledata, filedata, len(filedata))
#    src_image = cv.DecodeImageM(imagefiledata, 0)

dh = DemHist(src_image)

cv.WaitKey(0)
cv.DestroyAllWindows()

This will hopefully allow you to see something like this:Be warned that you need to adjust the crop width and height before the X & Y, otherwise you’ll generate a lot of exceptions.

Next steps

Having failed to figure out fancy statistical function to solve all my problems, I went for a dirt simple approach. You’ll see in the image above that even with a loose crop it has a fairly wide histogram. It turns out that with proper cropping both daytime and night time have fairly robust histograms.  At night the histogram looks like a malformed W, with a collapsed “/”. As it gets cloudier the histogram gets compressed.

The day time is a bit more tricky, as there is the sun to factor in. Depending on the conditions, the shard can be reflecting the brilliant blue sky, scary thunderclouds, or just slightly translucent. The histogram bumbles about wildly, up and down. However its fairly narrow, but as soon as there is cloud it shrinks to almost a single line.

Putting it all together:

Having found a useful metric, and through testing found thresholds, I wrote a script to crop, test and save a thumbnail. The data is then collected, stored, normalised and then displayed on www.whatcaniseefromtheshard.com.