Monday, November 1, 2021

More thoughts on Multi Image Capture: PART 2

In the last post I started discussing multi image capture for post processing frame averaging. In this post I’ll look at using frame averaging as an alternative to exposure bracketing, ie to extend dynamic range (DR).

[BTW the first post was amended to tidy up my language, thanks to some comments I received on the DPReview forum: many thanks to @Entrophy512].

In standard exposure bracketing we would set our camera to a base ISO, say, ISO 100 on my Canons, and capture as many images, separated, say, by 2Ev, to ensure our exposures capture enough information, down into in the shadows, and contain at least one image with no blown out highlights, for post processing through one of the following typical workflows:

  • Auto tone mapping and blend
  • Auto fusion blend
  • Manually blend

Although auto tone mapping had a bad reputation in the early days, these days most tone mapping software can achieve a reasonable natural look. With fusion based auto blending, eg LR/Enfuse, potentially creating an even more nature blend. 

However, both these approaches, despite auto correction tools, can introduce unsightly artifacts associated with movement, say, of trees, between images.

Manually blending, with or without luminosity masks, potentially offers the highest quality result, but does require more skill/effort in post processing.

All the above have a similar capture workflow:

  • Capture one image for the non blown out highlights
  • Capture enough images, at varying exposures, to ensure the shadow details you are interested in are captured

Or, put another way:

  • Capture one image for the non blown out highlights
  • Ensure the noise in the shadows detail areas is acceptable for post processing

Without going down a rabbit hole of detail, many photographers will recognise the following sources of noise:

  • Shot noise, related to the statistical nature of light. Shot noise follows a Poisson distribution, but may be approximated to a normal distribution well away from the shadows, ie in 'good' exposure. Shot noise can be reduced in post processing as the noise fluctuates around the mean exposure.
  • Dark current or thermal noise is mainly a problem for astro and/or extremely long exposures, which leads to the use of additional technologies that keep the sensor cool
  • Readout noise is, as it implies, generated after the sensor has gathered its signal. The main impact here is the 'extra' gain the photographer introduces, ie the ISO
  • Finally there are other noise sources that can impact the image, eg:
    • So-called 'reset noise', where the sensor pixel does perfectly zero itself after capturing an image
    • Fixed pattern (column) noise, which is normally seen as vertical lines in the shadow areas of an image. Fixed pattern noise will tend to be additive between two identical images, taken next to each other, without pixel shifting, say
    • Quantisation noise that is introduced in the analogue to digital conversion process

Not all noise sources are the same, and some we can ignore, ie as photographers we have no/little control over them; although in the end we observe an integrated impact of all the various noise sources. 

For most photographers I believe it is worth understanding two sources of noise in particular, as 'optimum' camera settings and post processing can be used to reduce these noise sources, if needed, ie extended the DR of the post processed image.

Ignoring image to image motion for now, and keeping things simple, shot and ISO noise reduction can be achieved by simply averaging multiple images: either in the camera, in the case of the Phase One IQ4; or in post for the rest of use ;-)

As noise reduction goes as the square root of the number of images processed, we can thus reduce the noise by, say, a factor of 4, by taking 16 images and averaging then, either through opacity or smart object statistics. 

This approach shows the most benefits in the shadow areas of an image, where the signal (number of photons captured/converted), and the tonal content, are low. However, it can also help to clean up a 'perfectly’ exposured scene, as shot noise is still visible at, say, just below maximum well capture, and can be clearly seen if you zoom in on a flat, featureless, well lit, surface.

At this point many will be saying: why bother?

After all, 'normal' exposure bracketing is fine and it's simple. But as stated above, it does come as a price:

  • Integrated capture time, eg a base (ETTR) exposure of a high dynamic scene, say needing a 4Ev lift for the shadows, will typically be achieved with three exposures, for tonal overlap, at 1s, 4s and 16s, ie an integrated capture time of at least 21s
  • During the capture wind movement may become a problem
  • Post processing may be OK, ie if there was no movement or movement is an artistic element of the image, but it might require some post processing effort to eliminate unsightly artifacts, ie to try and make the image look more natural or organic

If, on the other hand, we took a burst of 16 images at an exposure of 1s, the noise in the shadows and highlights will be reduced by a factor of 4. Thus, if shooting at ISO 1600, the post processed image would have noise characteristics of an ISO 100 image. Or if shot at ISO 100, the photon noise will look like an image shot at around ISO 6.

Shooting at high ISOs may be a useful tool to reduce fixed pattern noise, however, the ISO that you need to shoot at will be camera dependent. From my experience, a useful rule of thumb is to seek out an ISO just above where your camera exhibits ISO invariant characteristics and above where the fixed pattern noise becomes less noticeable. Thus on my Canon M I might use an ISO of 800 - 1000, but on my 5D3 I would use an ISO of 1600. 

Exploiting the dynamic range of modern cameras and using the above insight, allows us to consider an alternative high dynamic range, ie where we would normally bracket, exposure capture strategies, based on using one or two fixed exposures capture sets, ie one set for the scene; or one set for the highlights and another for the shadows. 

If you also wish to exploit ISO, ie for a camera that is not fully ISO invariant through its ISO range and where the scene’s DR is not too 'extreme', the capture workflow could go like this (composition and focus ignored):

  • Set an ETTR exposure for the highlights at the base ISO
  • Capture 4 images
  • Increase the ISO to, say, ISO 800 (ie set at your ISO invariant point)
  • Capture the appropriate number of images, eg 8 in this case
  • Post process

Note that for cameras that are fully ISO invariant, ie from their base ISO, you only need to capture sufficient images to address the noise in the shadows. That is shoot at a single ISO value.

To further emphasise the potential practicality of the above, consider where cameras are at the moment. The new kids on the block can capture images at 20-30 images per second, with no buffering.

As for the above example, ie a two set ,4+8 image capture, for the highlights and shadows appropriately; if the base exposure was 1s, as in the exposure bracketing above, the integrated capture time would now be 16s, as opposed to 21s. 

As for a post processing workflow it could go like this:

  • Ingest in Lightroom and adjust the RAW exposures in each of the two image sets. For the ETTR image set, use curves, ie keep the highlights fixed. Keep RAW sharpening set to zero as we don’t wish to add additional ‘noise’
  • Export both data sets to Photoshop
  • Merge each of the two image sets into their own Smart Object and use Mean statistics, or adjust the individual layer opacities: manual or use the merge script (on the right)
  • Stack the two flattened image sets and use manual blending to bring the 'best bits' of each forward

The attached three images are a simple test of the above I just took with my EOS M, at a focal length of 11mm:

The top image is one of four ETTR exposures taken at a ISO 100, f/8 and at 1/15s. The middle image is one of eight taken at the base exposure, but ISO shifted to ISO 1600: note I 'only' took 8 images in this case, ie rather than 16. Thus the shadow noise reduction will result in the image taken for the shadows looking like it has noise as if it was around a ISO 200 image.

The last image is a quick development of the other two, where I processed the two stacks in Photoshop, manually blended in the appropriate details from each image, and finished toning and colour grading back in Lightroom.

As usual for me, this has been a 'bit of fun'. Will I use it as an alternative to 'normal exposure bracketing? 

Maybe: as I have all my Canon cameras running in-camera Lua scripts that greatly speed up, capturing image sets like the above. Having said that, I believe most modern cameras are able to capture the above in a very lean manner. Plus the frame averaging approach to extending DR has the potential advantage of creating more natural or organic looking images where there is movement in the scene, eg wind.

In the next post I will carry on exploring (non-bracketing) multiple image capture for extended dynamic range.

As usual I welcome any comments on this post or any of my posts.

Friday, October 29, 2021

More thoughts on Multi Image Capture: PART 1 (updated)

Anyone who has read my blog knows that I love playing with technology and especially creating in-camera scripts. In this post, although there is a link to CHDK and Magic Lantern scripting, the ideas I'm discussing can be adopted by anyone with a camera.

Multi Image Capture is used to gather data beyond what is possible in a single image, eg: achieving greater depth of field; covering a larger dynamic range than the sensor can capture in a single image; realising an artistic vision, for example a long exposure; for reducing noise; to eliminate people from your image...etc etc

In other words, multi image capture is an essential tool for all serious photographers.

The two most common multi image capture use cases are, of course, increasing the depth of field and/or covering a large dynamic range. Focus and exposure bracketing, where we change a camera or lens prosperity between captures, are well know techniques and I'm not going to repeat the basics here. A search of this blog will bring up previous posts on both subjects.

Before I develop my multi image capture ideas further, it is useful to remind ourselves what most are brainwashed to believe as they start their photography: The so-called Exposure Triangle.

Yes it's a triangle, but it doesn't really represent the 'photon-based' exposure, ie formed from the aperture size, that controls the 'flow' of light on the sensor, and a shutter time, that allows the flow to fall on the sensor for a fixed amount of time.

ISO is simply a gain that gets added to increase the brightness of the captured scene, ie both the signal and the noise.

Many modern cameras are ISO invariant, and applying an ISO setting above the base ISO in the camera, is broadly no different to applying it in post. Other cameras, like my Canon M or my Canon 5D3, are only ISO invariant above a certain (camera specific) ISO.

Another thing that follows from the above is that shutter time is coupled to capture time. For example, if I wish to create a long exposure artistic look, eg smooth out some flowing water, then I need to match my shutter speed to the capture time that I need for my artistic vision.

This could be achieved by closing down the aperture, but as we know, the downside of this is that diffraction blur will increase, to the potential detriment of the image quality we are seeking. 

To overcome such problems we usually introduce neutral density filters to reduce the amount of light we can capture, allowing us to run with longer shutter times at the ‘optimum’ aperture. But NDs come at a cost.

First, in our pockets, as one ND is not going to cover all our needs; and also in terms of image quality, ie that extra glass (or even plastic), especially with stacked NDs, will only degrade your image quality. Thus, the thought of buying an expensive camera and lens, and then putting a cheap ND in front of it, is something to think about.

It would be far better if we could decouple the aperture, which is our primary depth of field tool, from the shutter time, and our capture time needs.

This is where multi image photography comes in and, in particular, image or frame averaging, where we don’t change camera or lens properties between shots, which gives us an alternative 'exposure/capture triangle' - note this is just a visual alliteration to the original exposure triangle:

Here we see, on the left, our driving vision for the image, which, on the right, which we wish to create. 

We will set the aperture (N) to achieve the depth of field we wish to realise, and set focus appropriately, eg hyperfocally, non-hyperfocally or multiple times if focus bracketing.

We choose a total/integrated capture time, eg for: long exposure impact; or reducing noise; or extending the captured dynamic range. Plus set the camera to the base ISO, to ensure maximum dynamic range coverage.

If the shutter time, to achieve an acceptable exposure, is similar to the capture time we are seeking, including, if required, an small ISO lift from the base, then we are there: all in a single exposure.

We could also iterate a bit with aperture, needing to play off depth of field with shutter-induced motion blur: but that's detail.

If, on the other hand we don't want to change our aperture, and our shutter time falls short of the capture time we wish, then we may decide to try a low strength ND, ie trying to stay away from high strength NDs, with their colour casts and image degrading glass/plastic. 

Once again, this may do it. However, if it doesn't, then we take multiple images at a single shutter time to ensure the integrated shutter time matches the desired capture time. For example, with a 1/10s shutter and a 2 second total capture goal, we will need 20 images; which, in passing, will help us reduce the noise in post capture processing.

For now we will ignore motion-strobe effects etc and assume the images are captured in a seamless/gapless fashion.

Although ISO is still there to be used, ie as a moderator, for the kind of photography we are talking about in this post, eg tripod based city/landscapes etc, we will usually wish to remain at the base ISO.

With the above we thus only need to carry one or two (maybe) low strength NDs and/or a circular polariser, which we will use to reduce the number of images we take and help take 'near' seamless/gapless images. That is, we need to match/respect the camera's image buffer limitations.

Let's now finish this post and look at the best way to achieve the above. Simply go out and buy yourself a Phase One XT IQ4 150Mb camera, which does all the above in-camera and delivers you a single (gapless) RAW image of any capture time.

OK, you don't have over 50,000 pounds to spend: then we will need to create an in field and post processing multi image capture work flow, to achieve a very similar result: for free!

But that's a story for future parts of this post.

As usual I welcome any comments on the above or any of my posts.

Tuesday, October 19, 2021

M3 LBS minor update

In the last post I introduced the LE/NR and Super-Resolution features in the M3 Landscape Bracketing Script. As a result of some testing, I've refined/simplified the UI.

The top half of the menu now looks like this:

Here we see that a Super Resolution bracket set has been requested under the Focus Bracket? menu item. The number of super-res images to be taken is now defined via the negative ND feature. A -3 value, as above, takes the script to take 8 images. A -5 value would result in 32 images.

There are now two focus move options that attempt to introduce image to image pixel dither. In the above Off is shown, meaning that no focus dithering will be attempted, ie the super-res bracket set 'just' captures the required number of images for either LE or NR post processing.

The two differ schemes are -+ or ++.

In the above we see that the -+ scheme has been selected, which will adjust focus either side of the point of focus, ie half and half. Each time adjusting focus towards infinity, having first re-positioned away from infinity. 

In the ++ scheme, shown below, focus differing is attempted from the point of focus, towards infinity. As stated in the last post, pixel dithering via refocusing is not a guaranteed approach, however, if the angle of view doesn't change, at  least you will have a LE/NR bracket set.

As a reminder, the UI looks like this, ie giving you constant focus feedback until you run a focus bracket set:

In the above we see that the console option is switched on, thus giving feedback from the script. This option can be disabled if required.

In this above, the top bar shows:

  • We are focused at 259cm from the sensor
  • That 2 focus brackets are required to get to the hyperfocal
  • That we are at 13mm focal length
  • That the shutter is set to 1.3s
  • That the near Depth of field is at 130cm
  • That the defocus infinity blur is at 16micons, in other words we are focused just short of the hyperfocal, which is based on a 15um overlap setting, as shown above in the menu
To capture the super-res bracket set, all we need to do is to push the the M-Fn button, which acts as the script's auxiliary shutter button. In this case, because we are not requesting a focus bracket set, the script will keep running, ie you can repeat the super-res bracketing as many times as you wish, adjusting focus and exposure in between captures.

As implied above, the noise reducation or long exposure bracket set operates in a similar way to that of the super-res one: but in the LE/NR case focus is not changed at the LE/NR bracket set is captured at the 'infinity' focus position, as is the sky bracket, ie after focus bracketing. 

Friday, October 15, 2021

M3 Landscape Bracketing Script: Now with ND and Super-Resolution bracketing

I've discussed multi image bracketing for ND and Super-Resolution in previous posts: that is taking a bracket set of images for post processing to simulate a long exposure or provide the data to create an up-scaled and/or noise reduced image.

The idea is simple. For LEs just divide the required time by the shutter speed and take this many images for 'averaging' in post, eg to simulate movement in water. This feature works well when you either don't have an ND filter with you, or you don't have a stronger enough one.

For Super-Resolution processing the procedure is similar, but we need to introduce small amounts of sensor movement between images. That is take at least 4 images, or as many as you wish, that are up-scaled and statistically averaged in post. This technique is best applied to images with little/no movement. The Super-Resolution bracketing exploits the focus breathing that occurs when changing focus, ie to simulate pixel shifting at the sensor. But note, this effect may not manifest itself on every lens or at every focus position or focal length, as it is dependent on the lens exhibiting lens breathing, ie a change in the angle of view, in the area of focus. However, even with no movement, you will still have a noise reduction bracket set.

I'm pleased to announce that I've incorporated both bracketing schemes into my M3 Landscape Bracketing Script: which can be downloaded from the right as usual. The top part of the menu now looks like this:

@subtitle Bracketing Options
#mode = 0 "Focus bracket?" {Off X2INF Min2INF Manual Min2X SupRes}
#bracket = 0 "Exposure bracket?" {Off 1(4Ev) 1(3Ev) 1(2Ev) 2(2Ev) 3(2Ev) ISO Auto Wind3Ev Wind4Ev}
#sky = 0 "Sky Bracket?" {Off 2Ev 3Ev 4Ev 5Ev 6Ev ETTR}
#ndf = 0 "ND?" [-5 30]

@subtitle Bracketing Settings
#infinity = 3 "Infinity Focus (xH)" [2 4]
#overlap = 15 "Overlap (um)" [5 30]
#hilit = 5 "% Histo HiLit" [1 30]
#start = 1 "Histo Upper Quartiles" [1 4]
#pmag = 1 "Assumed Pupil Mag" [1 6]
#isoval = 0 "ISO Bracketing Value" {400 800 1600 3200}
#supres = 4 "# SupRes shots?" [4 16]

To use the ND/LE feature, simply set ND to a non-zero number. If the number is negative the bracket set will simulate that ND value, eg -2 will take 4 images for processing a simulated 2 stop ND filter; whereas -5 will take 32 images to simulate a 5 stop filter.

Note that the negative mode can also be used to create a noise reduction bracket subset. That is a value of -4 will create 16 images, for a NR of 4 after processing in post.

If the ND value is positive, eg 2, this is asking the script to take the required number of images to simulate a 2s exposure. In order to keep the number of images to a minimum, the script first adjusts the aperture to the maximum it can, ie up to f/22 if the base shutter allows. The shutter is then adjusted to ensure the exposure is the same as the base. 

The Super-Resolution technique will only work on a lens whose focus motor can be controlled, eg the EFM lenses. This new feature works best when you set the base focus at the hyperfocal, as the lens moves towards infinity, at the smallest step it can, to simulate sensor pixel shifting.

The number of images is set in the supres menu item, ie between 4 and 16. Also, the super-resolution bracket set can only realised with focus bracketing set to manual. Whereas the ND/LE/NR bracket set will be created as part of a focus bracket capture, eg X2INF.

Both bracket sets are post processed in, say, photoshop, using the following workflows:

  • For LE/ND processing ingest the images into Photoshop and stack into a single image. Align the images then either use the Photoshop Merge script on the right or manually adjust the individual image's opacity, ie halving the opacity each time with the bottom image at 100%. Then flatten the image.
  • For Super-Resolution the process is similar, however,after stacking and before aligning, upscale the image by 200%. After processing you can then export the Super-Resolution image back to Lightroom, or downscale first. See here for more information on the Super-Resolution technique.

You can also use Smart Object statistics, eg median, but this route is slow if you have a large number of objects.

The following is a Super-Resolution test image I just took with my M3 using the 11-22mm M lens at 11mm. The aperture was set to f/8 and the shutter to 1/40s. I requested a 10 image Super-Res bracket set on this occasion, and was lucky that there was nearly no wind to ruin the test.

If you zoom in on the image it is easy to see the lack of noise in the shadows and the general cleanness of the image, noting that no NR was used in post.

With the two new bracketing features, it is easy to create complex bracket sets via one or two button pushes. For example you could set up focus bracketing for a deep focus capture, plus taking an ND/LE bracket set to layer in in post, plus an ETTR sky shot to, once again, layer in in post.

I will discuss post processing in a future post.

As usual I welcome any comments on this post or any of my posts.

Monday, September 20, 2021

Landscape Bracketing Script (M3): now with a simulated ND filter option

Just a quick post to say I've brought the M3 version of my CHDK Landscape Bracketing Script, into line with the non-M3 version.

The M3 version, downloaded from the right as usual, now has a menu item called "ND Filter?", with values of 0 to 5.

If 0 is selected then no ND brackets will be taken. If the value of this menu item is between 1 and 5, then a simulated ND bracket set will be taken, as part of the focus and exposure bracket set.

An ND2, will take 2 images that you can post process, whereas an ND5 will take 32 images. Each ND bracket set can be processed in Photoshop, for example, to create a Long Exposure image for blending with the others in the bracket set, eg the focus images and the sky ETTR capture.

The ND images are taken at the foreground exposure value, ie the one used for focus bracketing.

As with the sky exposure bracket, the ND filter bracket set is taken at the infinity focus point that you set, eg three times the hyperfocal, say.

Once captured, the bracket set is easily identifiable in Lightroom, as the script makes use of dark frame bookends at appropriate places, eg:

In the above we see that the entire bracket set has bookends at the start and end of the bracket set, and the ND set is also delineated, eg the 4 images towards the end, in the above example. Also seen are (4) focus brackets, at the start: two before the hyperfocal, one at the hyperfocal and one at, in this case, 3 times the hyperfocal, for a defocus infinity blur of about a third of the overlap CoC, which was set at 15 microns in this test capture.

The script automatically took the focus brackets, as, in this case, I had selected the X2Inf option. Thus all I needed to do was to focus on the foreground point of interest, note the number of focus brackets and accept or adjust the aperture, focus or focal length.

Then, with a single button press, the M3 captured a perfect focus bracket set, plus capturing a simulated ND bracket set for 'statistical processing in Photoshop, and an ETTR sky image at infinity. 

Once the focus and ND bracket sets are pre-processed, these two images can be blended with the sky ETTR image, eg using masks.

As usual I welcome any comments on this post or any of my posts.

Wednesday, September 1, 2021

If it's good enough for a Large Format camera; then it's good enough for my Canon M3

Once you get into photography you begin to look at the various categories of equipment. From large format film cameras down to point and shoot digital technology.

From my perspective, with a bias towards the science and technology of photography, I'm always interested to look backwards for 'new' ideas, ie ideas that we can exploit today.

This post is about one of those necessities from the large format, view camera world, that is virtually ignored today. Namely a dark cloth, which is also called a focusing cloth or hood, The dark cloth being a piece of light-proof material that covers your head and the back of the camera. It eliminates stray light to allow for proper viewing of the relatively dim image on the camera ground glass during framing and focusing.

Although today's digital cameras don't require us to inspect focus on a piece of ground glass, we often find ourselves looking at the LCD screen , eg to read camera info, carry out focus peeking or, in my case, drive Magic Lantern or CHDK.

But try and look at a camera's LCD screen in bright sunlight, or if your eyes are 'getting older', as mine are.

I've tried various 'solutions', but I found all of them lacking at some level. Plus things can get complicated if your camera has an articulating screen, rather, say, than the fixed screen on my 5D3.

Also, as a photographer who likes getting low, I envy the cameras of old, where you can look down into the camera:

As my 5D3 has a solution in the form of the Swivi S5 (see below), I decided to 'hack' my M3.

As I didn't wish to buy more 'stuff' I decided to use what I had and not to compromise my base Canon M3 configuration, which looks like this:

That is any solution needed to fit around a cage and a handle, and enhance my low level photography needs, when I have the screen deployed like this:

The base set up ended up looking like this, where I adapted a collapsible Hoodman Loupe, with a diopter adjustment, via a cold shoe mounted, miniture ballhead:

But this base arrangement still suffers from light hitting the screen.

The final configuration benefited from me having a wife who is an expert sewist. She quickly knocked up the following Mark 1 version of my idea, which will likely end up the final version for me. Thank you Jean :-x

Here we see my 'dark cloth' for your M3 camera :-)

So far it's a 100% success: allowing me to use the camera in bright sunshine, low on the ground, whilst focus peeking and controlling CHDK scripts.

As usual I welcome any feedback on this post or any of my posts.

Monday, August 30, 2021

M3 Landscape Bracketing Script: minimum focus to focus position mode

Just a short post to say I've updated my M3 bracketing script to include a new auto focus mode, to cover when you wish to focus bracket from the minimum focus distance of the lens, to a fixed point. For example, when you don't want a sharp image throughout the scene.

To enable the new feature simply select it in the menu. But remember the caveat with this script, it’s not designed for macro focus bracketing at high magnification.

Once selected, to use this new mode simply:

  • Set the lens to the minimum focus and focus the camera on the nearest part of the object of interest, ie by moving the camera - the front of the tea bag in the example below;
  • Then move focus, with the lens, to the far part of the object of interest, ie the top of the tea bag in this example;
  • Then run the script.

To get an estimate of the number of focus brackets, simply take note of the difference between the number of brackets reported at each focus station, ie the minimum and the farthest focus point of interest.

Finally, you can also use sky bracketing with the new Min2X feature.

In this example, shot at F/4 at an 11mm focal length, the script took 6 images, which I processed in Helicon Focus:

As usual I welcome any feedback on this post or any of my posts.