Progress Report #2

1)  In our last report we attempted to alter the lighting in an image by taking two pictures of the same scene, one in the afternoon and one in the evening, then finding the transfer function between them. When we applied this transfer function to other input images the resulting output image was rife with noise. Upon closer examination of image impulse responses or psf’s we concluded that the lighting and color effects we are trying to apply are not linear transformations. We have also come to find that while frequency domain analysis is good for the detection and manipulation of objects within the image it is not directly useful in the manipulation of color.

The averaging system we developed for comparison of the color content between images works reasonably well when applied between images where one color is not highly dominant. However when we apply this function between images in which one is dominant the following occurs:

Source: 1

Source: 2

Instead of sunny, tropical Seattle we get blue Seattle.

However we found that if we compose a more balanced color palette from the primary colors contained in the tropical scene and apply the same system we get a much more favorable result as seen in the examples below:

Source: 1

NO PALLETTE:

Source: 1

Source: 3

WITH PALLETTE:

Source: 1

We also found that if we applied the transformation using the palette to similar objects between images an even better result is obtained as can be seen below (foliage isolated):       

Source: 1

Source: 3

So the emphasis of our project has shifted to the following:

1. continue to fine tune the color transfer system. This will require further investigation into how colors are represented via rgb and hsv formats.
2. Develop a program that detects primary colors and can construct a viable color palette.
3. Potentially develop a system to detect coherent regions of the image (foliage, sky, etc.) and apply the transformation in a piecewise manner.

We have already begun developing the program listed under 2). So far we have developed the following techniques. The tropical image seen below contains too much information for easy analysis. What is more we are really only interested in the dominant colors.

                        

Source: 2

So our initial step involves running the image through a low pass filter to get rid of some of the edges. We then downsample and upsample the image to reduce the complexity even further as seen below:

     

Now instead of a very large number of sharp discontinuities between objects and colors we have much less overall complexity and smoother transitions between objects.

In order to locate the primary color peaks in the image we wrote a program that finds the gradient of the color change within the image as seen below in grayscale:

  

and in color (the color image became degraded after copying it from matlab into the report):

As can be seen above the gradient effectively highlights the regions containing the colors of interest.

We haven’t exactly worked out how yet (although we have ideas) but we plan to use this gradient image to select the colors of interest and then construct an effective color palette.

We feel that some of the other dsp techniques learned in class such as auto-correlation might turn out to be effective when attempting to locate colors and other regions of interest. Edge detection seems like it will also be useful for locating cohesive regions of the image.

4) Honestly the coolest thing we’ve done so far happened when we were initially attempting to simply detect changes in color/dominant colors along a single row vector. After figuring out how to do this we suddenly realized we could simply apply it over the entire image and obtain a gradient. This was especially cool since we had been looking for a gradient function in matlab just before this but couldn’t find one.

Sources:

1: Seattle skyline - http://i356.photobucket.com/albums/oo1/DizzyTai/1753506400_983b2c92be_b.jpg
2: Tropical Scene - http://wall.alphacoders.com/big.php?i=402163
3: River Scene - http://www.tapeteos.pl/data/media/597/big/wiosna_2560x1600_008.jpg