More thoughts on Multi Image Capture: PART 2

In the last post I started discussing multi image capture for post processing frame averaging. In this post I’ll look at using frame averaging as an alternative to exposure bracketing, ie to extend dynamic range (DR).

[BTW the first post was amended to tidy up my language, thanks to some comments I received on the DPReview forum: many thanks to @Entrophy512].

In standard exposure bracketing we would set our camera to a base ISO, say, ISO 100 on my Canons, and capture as many images, separated, say, by 2Ev, to ensure our exposures capture enough information, down into in the shadows, and contain at least one image with no blown out highlights, for post processing through one of the following typical workflows:

• Auto tone mapping based blend
• Auto fusion based blend
• Manually blend

Although auto tone mapping had a bad reputation in the early days, these days most tone mapping software can achieve a reasonable natural look. With fusion based auto blending, eg LR/Enfuse, potentially creating an even more nature blend. 

However, both these approaches, despite auto correction tools, can introduce unsightly artifacts associated with movement, say, of trees, between images.

Manually blending, with or without luminosity masks, potentially offers the highest quality result, but does require more skill/effort in post processing.

All the above have a similar capture workflow:

• Capture one image for the non blown out highlights
• Capture enough images, at varying exposures, to ensure the shadow details you are interested in are captured

Or, put another way:

• Capture one image for the non blown out highlights
• Ensure the noise in the shadows detail areas is acceptable for post processing

Without going down a rabbit hole of detail, many photographers will recognise the following sources of noise:

• Shot noise, related to the statistical nature of light. Shot noise follows a Poisson distribution, but may be approximated to a normal distribution well away from the shadows, ie in 'good' exposure. Shot noise can be reduced in post processing as the noise fluctuates around the mean exposure.
• Dark current or thermal noise is mainly a problem for astro and/or extremely long exposures, which leads to the use of additional technologies that keep the sensor cool
  
• Readout noise is, as it implies, generated after the sensor has gathered its signal. The main impact here is the 'extra' gain the photographer introduces, ie the ISO
  
• Finally there are other noise sources that can impact the image, eg:
• So-called 'reset noise', where the sensor pixel does perfectly zero itself after capturing an image
• Fixed pattern (column) noise, which is normally seen as vertical lines in the shadow areas of an image. Fixed pattern noise will tend to be additive between two identical images, taken next to each other, without pixel shifting, say
• Quantisation noise that is introduced in the analogue to digital conversion process

Not all noise sources are the same, and some we can ignore, ie as photographers we have no/little control over them; although in the end we observe an integrated impact of all the various noise sources. 

For most photographers I believe it is worth understanding two sources of noise in particular, as 'optimum' camera settings and post processing can be used to reduce these noise sources, if needed, ie extended the DR of the post processed image.

Ignoring image to image motion for now, and keeping things simple, shot and ISO noise reduction can be achieved by simply averaging multiple images: either in the camera, in the case of the Phase One IQ4; or in post for the rest of use ;-)

As noise reduction goes as the square root of the number of images processed, we can thus reduce the noise by, say, a factor of 4, by taking 16 images and averaging then, either through opacity or smart object statistics. 

This approach shows the most benefits in the shadow areas of an image, where the signal (number of photons captured/converted), and the tonal content, are low. However, it can also help to clean up a 'perfectly’ exposured scene, as shot noise is still visible at, say, just below maximum well capture, and can be clearly seen if you zoom in on a flat, featureless, well lit, surface.

At this point many will be saying: why bother?

After all, 'normal' exposure bracketing is fine and it's simple. But as stated above, it does come as a price:

• Integrated capture time, eg a base (ETTR) exposure of a high dynamic scene, say needing a 4Ev lift for the shadows, will typically be achieved with three exposures, for tonal overlap, at 1s, 4s and 16s, ie an integrated capture time of at least 21s
• During the capture wind movement may become a problem
• Post processing may be OK, ie if there was no movement or movement is an artistic element of the image, but it might require some post processing effort to eliminate unsightly artifacts, ie to try and make the image look more natural or organic
  

If, on the other hand, we took a burst of 16 images at an exposure of 1s, the noise in the shadows and highlights will be reduced by a factor of 4. Thus, if shooting at ISO 1600, the post processed image would have noise characteristics of an ISO 100 image. Or if shot at ISO 100, the photon noise will look like an image shot at around ISO 6.

Shooting at high ISOs may be a useful tool to reduce fixed pattern noise, however, the ISO that you need to shoot at will be camera dependent. From my experience, a useful rule of thumb is to seek out an ISO just above where your camera exhibits ISO invariant like characteristics and above where the fixed pattern noise becomes less noticeable. Thus on my Canon M I might use an ISO of 800 - 1000, but on my 5D3 I would use an ISO of 1600. 

Exploiting the dynamic range of modern cameras and using the above insight, allows us to consider an alternative high dynamic range, ie where we would normally bracket, exposure capture strategies, based on using one or two fixed exposures capture sets, ie one set for the scene; or one set for the highlights and another for the shadows. 

If you also wish to exploit ISO, ie for a camera that is not fully ISO invariant through its ISO range and where the scene’s DR is not too 'extreme', the capture workflow could go like this (composition and focus ignored):

• Set an ETTR exposure for the highlights at the base ISO
• Capture 4 images
• Increase the ISO to, say, ISO 800 (ie set at your ISO invariant point)
• Capture the appropriate number of images, eg 8 in this case
• Post process
  

Note that for cameras that are fully ISO invariant, ie from their base ISO, you only need to capture sufficient images to address the noise in the shadows. That is shoot at a single ISO value.

To further emphasise the potential practicality of the above, consider where cameras are at the moment. The new kids on the block can capture images at 20-30 images per second, with no buffering.

As for the above example, ie a two set ,4+8 image capture, for the highlights and shadows appropriately; if the base exposure was 1s, as in the exposure bracketing above, the integrated capture time would now be 16s, as opposed to 21s. 

As for a post processing workflow it could go like this:

• Ingest in Lightroom and adjust the RAW exposures in each of the two image sets. For the ETTR image set, use curves, ie keep the highlights fixed. Keep RAW sharpening set to zero as we don’t wish to add additional ‘noise’
  
• Export both data sets to Photoshop
• Merge each of the two image sets into their own Smart Object and use Mean statistics, or adjust the individual layer opacities: manual or use the merge script (on the right)
  
• Stack the two flattened image sets and use manual blending to bring the 'best bits' of each forward

The attached three images are a simple test of the above I just took with my EOS M, at a focal length of 11mm:

The top image is one of four ETTR exposures taken at a ISO 100, f/8 and at 1/15s. The middle image is one of eight taken at the base exposure, but ISO shifted to ISO 1600: note I 'only' took 8 images in this case, ie rather than 16. Thus the shadow noise reduction will result in the image taken for the shadows looking like it has noise as if it was around a ISO 200 image.

The last image is a quick development of the other two, where I processed the two stacks in Photoshop, manually blended in the appropriate details from each image, and finished toning and colour grading back in Lightroom.

As usual for me, this has been a 'bit of fun'. Will I use it as an alternative to 'normal exposure bracketing? 

Maybe: as I have all my Canon cameras running in-camera Lua scripts that greatly speed up, capturing image sets like the above. Having said that, I believe most modern cameras are able to capture the above in a very lean manner. Plus the frame averaging approach to extending DR has the potential advantage of creating more natural or organic looking images where there is movement in the scene, eg wind.

As usual I welcome any comments on this post or any of my posts.