Here’s a photo of a Christmas tree, as my camera’s sensor sees it:

It’s not even black-and-white, it’s gray-and-gray. This is becuase while the ADC’s output can theoretically go from 0 to 16382, the actual data doesn’t cover that whole range:

The real range of ADC values is ~2110 to ~136000. Let’s set those values as the white and black in the image:

V_new = (V_old - Black)/(White - Black)

Much better, but it’s still more monochromatic then I remember the tree being. Camera sensors aren’t actually able to see color: They only measure how much light hit each pixel.

In a color camera, the sensor is covered by a grid of alternating color filters:

Let’s color each pixel the same as the filter it’s looking through:

This version is more colorful, but each pixel only has one third of it’s RGB color. To fix this, I just averaged the values each pixel with it’s neighbors:

Applying this process to the whole photo gives the lights some color:

However, the image is still very dark. This is because monitors don’t have as much dynamic range as the human eye, or a camera sensor: Even if you are using an OLED, the screen still has some ambient light reflecting off of it and limiting how black it can get.

There’s also another, sneaker factor causing this:

Our perception of brightness is non-linear.

If brightness values are quantized, most of the ADC bins will be wasted on nearly identical shades of white while every other tone is crammed into the bottom. Because this is an inefficient use of memory, most color spaces assign extra bins to darker colors:

As a result of this, if the linear data is displayed directly, it will appear much darker then it should be.

Both problems can be solved by applying a non-linear curve to each color channel to brighten up the dark areas… but this doesn’t quite work out:

Some of this green cast is caused by the camera sensor being intrinsically more sensitive to green light, but some of it is my fault: There are twice as many green pixels in the filter matrix. When combined with my rather naive demosaicing, this resulted in the green channel being boosted even higher.

In either case, it can fixed with proper white-balance: Equalize the channels by multiply each one with a constant.

However, because the image is now non-linear, I have to go back a step to do this. Here’s the dark image from before with all the values temporarily scaled up so I can see the problem:

… here’s that image with the green taken down to mach the other channels:

… and after re-applying the curve:

This is really just the bare minimum: I haven’t done any color calibration, the white balance isn’t perfect, there’s lots of noise that needs to be cleaned up…

Additionally, applying the curve to each color channel accidentally desaturated the highlights. This effect looks rather good — and is what we’ve come to expect from film — but it’s has de-yellowed the star. It’s possible to separate the luminance and curve it while preserving color. On it’s own, this would make the LED Christmas lights into an overstaturated mess, but combining both methods can produce nice results.

For comparison, here’s the image my camera produced from the same data:

Far from being an “unedited” photo: there’s a huge amount of math that’s gone into making an image that nicely represents what the subject looks like in person.

There’s nothing that happens when you adjust the contrast or white balance in editing software that the camera hasn’t done under the hood. The edited image isn’t “faker” then the original: they are different renditions of the same data.

In the end, replicating human perception is hard, and it’s made harder when constrained to the limitations of display technology or printed images. There’s nothing wrong with tweaking the image when the automated algorithms make the wrong call.