6.7.14

Making Colour

The digital reality that people surround themselves with these days differs from their physical reality in many ways but I'm only interested in the way that annoys me the most.

Digital displays do not accurately represent all colours people perceive in the real world. In digital images colours become colors.

What's worse, color combinations in digital images are severely distorted. It has to do with what is known to artists and scientists as primary colours.

When artists speak of primary colours they generally mean red, blue, and yellow. When scientists speak of primary colors they generally mean red, blue, and green.

It's amusing to watch artists and scientists quarrel over primary colour labels because both camps are correct in their assertions. Sort of but not really.

Artists made colours from crystals back in the day and considered yellow to be primary because they could not break down yellow pigments into other pigments yet when they combined yellow pigments with other pigments they produced new pigments. Yellow pigments mixed with red pigments produced orange pigments, red and blue pigment mixtures resulted in violet, while yellow and blue produced green.

Fig1.1 A spectacular exhibit.

Scientists consider green to be primary because, to put it simply, green occupies a greater portion of the rainbow as experienced by the average human than yellow does [as illustrated in the image below on Fig1.2, not the one above on Fig1.1].

Yet each cone cell in the human eyeball that's responsible for deciphering the rainbow responds to a range of colour wavelengths, not a particular colour, and the perception of colour is relative to the observer.

The rainbow, which illustrates the visible light region of the electromagnetic spectrum, is continuous and there are no set boundaries between different colours. Yellow is adjacent to green, which is adjacent to blue, and where each one begins and the other one ends is only a rough estimate. Different device manufacturers have different rough estimates.

Fig1.2 A slice of the rainbow
Regardless of which came first, green or yellow, both artists and scientists agree that the average human requires 3 primary colours to make and perceive all other colours on the rainbow/visible portion of the electromagnetic spectrum.

As I've established here already, not everyone experiences the visible portion of the electromagnetic spectrum as the average human but even the average human is shortlisted when it comes to digital displays.

Digital displays still cannot accurately display all colors experienced in reality by the average human. In other words, every kitten photo you clicked on today was only an approximation of the kitten in real life, at least colour-wise. Some manufacturers considered adding a fourth primary to compensate. However, even this four-primary tech does not yet reach the range of colours that average trichromat humans are capable of perceiving. Perhaps one of these primary colours was not really primary.

Take the digital image below. The left portion contains an image of a plant known as Hot Poker. The image is composed of pixels where each pixel is described by 3 integer values between 0 and 255 that represent the amounts of red, blue, and green in it. Green, not yellow, because where's the fun in displaying colours accurately. The right portion of the image contains the main colors of the left portion.  


Fig1.3 Hot Pokers and their digital colors 
In the physical world the yellow and red petals of the hot pokers flow more sinuously. Same goes for their green stems and leafs. Unfortunately, at this point there's not much that can be done to restore the original colours of the physical scene but something can still be done to harmonize the digital image.

When I first started working on Coloroid's harmonic template filters I attempted to recreate the magic that happens in this paper. But then that magic turned out to be too computationally expensive for a mobile web app so I had to come up with my own magic.

The 'Hot Pokers' digital image exists in a 3D color space where the combination of the 3 integer values of each pixel contains not only information about the pixel's color or hue, but also it contains information about how vivid that colour is and how bright it is, which means each one of the 3 variables: r, g, and b contains information about the pixel's hue, saturation, and brightness.

Trying to obtain a single color from a digital image could be a nightmare. Take green for example. Intuitively, one would imagine that if the values of  r and b are 0 then g would describe a green pixel, which is true but not the kind of green you typically experience. In the composite image below, the first two images illustrate the two main flavours of "green" from the "Hot Pokers" image, the 3rd one represents the "primary" green obtained from zeroing the r and b channels (note how once r and b are 0, the g channel is the kind of green you rarely experience in nature)

Fig 1.4 Flavours of green: the one on the far left is more like ochre than green

If the 3 channels are flattened so that the hue is separate from the brightness and the saturation, then the different colours can be categorized with greater ease and this information can be used to create different harmonious colour combinations of the image from its existing colour combinations.

In order to separate hue, saturation, and brightness into 3 separate channels one must convert from RGB to HSV for example (although HSV is lossy, it is also quick and inexpensive).

Below is an illustration of the first flavour of ochre green in "Hot Pokers", although to call this colour ochre green is probably an insult to someone somewhere. Also, the whole colour wheel below is an atrocity to colours everywhere 'cept for the interwebs.

Fig1.5 HSL wheel
Once the hue info is stored separately in a single variable it can be easily checked to see where each pixel stands on the spectrum and decide what to do with it. Standard implementations of the HSL and HSV algorithms return the hue, saturation and brightness or luminance as a float value between 0 and 1.

Fig1.5: A separate channel for the hue

Since h is between 0 and 1, it is easy to find then the complementary color range of any h.

This is useful in making harmonious combinations because regardless of whether people talk about expressionism or cubism, or surrealism and classicism, or whatever, what makes most of everyone's favourite paintings so appealing is that none of them contain red, blue, and green in equal amounts at the same time.

All of them fall into one of the harmonic colour templates as described on Figure 2 in this paper.

Fig2.0 Types of harmonic templates on the hue wheel


None of these templates uses 3 primary colours in equal amounts at the same time. The i-type, V-type , and T-type of harmonic templates correspond to monochromatic or dichromatic images (ones made of shades of the same colour, or shades of the same 2 colours respectively).  A favourite technique of great painters is to use complimentary colours or split complimentary colours, which correspond to the X & I types, and Y and L types of harmonic templates respectively.

A sample of famous paintings from different genres.

Since h is between 0.0 and 1.0, then the complementary colour range for a given pixel's hue h < 0.5 is h + 0.5, and for any h > 0.5 the complementary color is h - 0.5., the split complimentary is at h +  0.3 and h - 0.3 respectively, and the analogous is at +/- 0.1. With this in mind, the following algorithms shift every pixel of a particular colour specified in the if condition to its complementary hue.

In this particular case, the algorithm complements every pixel which isn't in the blue range and matches it to blue in order to force the image either into the i-type, the V-type , or T-type  where the entire image becomes either monochromatic (composed of shades of blue) or dichromatic (composed of shades of blue and shades of red).

 /* -------------------------------------------------  
 This content is released under the GNU License  
 http://www.gnu.org/copyleft/gpl.html 
 Author: Marina Ibrishimova (http://ibrius.net)  
 Version: 1.0  
 ---------------------------------------------------- */  
function match_blue($filename)
{
     $image = '/path_to/'.$filename;  
      if (file_exists($image))
      {   
           $im = imagecreatefromjpeg($image);
           $width = imagesx($im);  
           $height = imagesy($im);  
           //because you really don't want to loop through more that this  
           if ($width >= 1000)  
           return FALSE;  
           //let the looping begin  
           for($x = 0; $x < $width; $x++)   
           {  
                for($y = 0; $y < $height; $y++)   
                {  
                     $rgb = imagecolorat($im, $x, $y);  
                     $r = ($rgb >> 16) & 0xFF;  
                     $g = ($rgb >> 8) & 0xFF;  
                     $b = $rgb & 0xFF;  
                     $alpha = ($rgb & 0xFF000000) >> 24;  
                     //convert to hsl
                     list($h, $s, $l) = rgb2hsv($r, $g, $b);
                     //we're in red&green&yellow, convert to blue hue
                     if ($h > 0.7)
                     {
                         $h = $h - 0.5;
                     }
                     else if ($h < 0.5)
                     {
                         $h = $h + 0.5;
                     }
                     if($h > 1) {$h=1;}
                     //convert back to rgb
                     list($r, $g, $b) = hsv2rgb($h, $s, $l);
                     //piece everything together  
                     imagesetpixel($im, $x, $y, 
                     imagecolorallocatealpha($im, $r, $g, $b, $alpha));  
                }  
           }
           if(imagejpeg($im, 'path_to_temp/'.$filename))  
           {return TRUE;  }
           imagedestroy($im);  
      }  
 return FALSE;  
 }

Below is a demo of this algorithm in JS. Note that this algorithm does not take into account the spatial relation between different pixels with the exact same hue so it will shift all pixels of the same hue. Because where's the fun in solving all problems at the same time.

Also note that this filter is not in Coloroid. The harmonic filters there are slightly different. This one simply illustrates how to turn a trichromatic image into either a monochromatic or a dichromatic one.