Monday, January 2, 2017

Further experiments in Super Resolution Photography

In a previous post I wrote about how to use a Magic Lantern Lua script (on the right) to create auto super resolution brackets. In this post, I offer some insight into why SR photography may be of interest to you, and a Photoshop action to automate PS-CC post processing: well at least for a four layer SR stack.

As usual my test image is a view in our house.

I first used the SR Lua script to create a four bracket SR set, using the jiggle and move option.

After ingesting into LR, I simply exported as layers into Photoshop.

I had previously created a script that took the four layers, scalled up by 200%, aligned the layers, reduced the opacity according the usual recipe for SR processing (100%, 50%, 33%, 25%), flattened the layers, and reduced the image scale back down, ie by 50%. For those interested here are the script commands:

Action: 4 Layer SR

Select back layer

Without Make Visible

3

Select front layer

Modification: Add Continuous

Without Make Visible

1, 3, 4, 5

Image Size

Width: 200%

With Scale Styles

With Constrain Proportions

Interpolation: nearest neighbor

Align current layer

Using: content

Apply: automatic

Without Vignetting Removal

Without Geometric Distortion Correction

Select back layer

Without Make Visible

6

Select forward layer

Without Make Visible

3

Set current layer

To: layer

Opacity: 50%

Select forward layer

Without Make Visible

4

Set current layer

To: layer

Opacity: 33%

Select forward layer

Without Make Visible

5

Set current layer

To: layer

Opacity: 25%

Flatten Image

Image Size

Width: 50%

With Scale Styles

With Constrain Proportions

Interpolation: nearest neighbor


As for a comparison, I made good use of the new LR-CC compare mode (in the Library module).

First, here is the base (RAW) image:


Here is the LR comparison after processing both the a single RAW .CR2 and the PS-CC processed SR image with the same settings, ie lens correction and detail sharpening.


I hope you agree, that if you are interested in getting the maximum image quality and detail out of an image, a simple 4-layer SR approach appears to have real benefits (accepting the subject shouldn't be moving). 

The SR (on the left) clearly exhibits detail and the single RAW is softer and lacking the detail in the SR version.

As usual I welcome any feedback on this approach to SR photography, ie using an ML script and a PS-CC script (at least for a four image bracket set).




Friday, December 30, 2016

How far can you push things?



As I prepare for a photography trip in Feb (more about this soon), I’ve used the Christmas break to reset my Magic Lantern configuration. Up until now I’ve been using the Canon 5D3-113 firmware, as, in ML, this is the ‘best’; but only if you are a videographer - which I’m not.

The downside of using the 113 Canon firmware is that it doesn’t support F/8 AF.

Why is this important? Because my 70-200 F/4L with a x2 extender means that the minimum (widest) aperture is F/8. So I took the decision to move the 123 ML version.

Moving back and forth between 113 and 123 is a simple matter for loading thee appropriate Canon firmware, doing and in-camera update and then following this with the appropriate in-camera ML update.

Having assured myself that all my ‘go to’ scripts work, ie: Auto Bracketing, Auto HFD, LE Simulation and Super Resolution bracketing (see on the right), I’ve spent a few hours testing the F/8 AF capability.

As I’ve been on a Christmas (garden) safari for a week [:-)], I had lots of willing subjects: as long as I fed them!

As many will be aware, unless you have a camera with an ISO invariant sensor (and I don’t), then you need to appreciate the ‘limitations’ of the sensor as you set exposure. For example, recognising you may need to prioritise something, eg shutter speed; use ETTR for the case where the camera electronics limit things, eg below about 1600 for my 5DIII, and simply push whatever image comes out of the camera in the ISO invariant zone, eg about 1600 for my 5DIII. I’ve used this chart for a 5DII from here before: http://www.clarkvision.com/articles/iso/
So my ‘test case’ from my garden safari, on a foggy, dull, sunless days, was a real extreme shot. I was aiming to handhold with the 70-200 at F/8 with a x2 Extender, to achieve a very fast shutter of 1/1600. This resulted in an ISO of 12800, ie 7 stops from ISO 100.

I processed the image in LR and used Piccure+ as well. The resulting image would not win me any prizes, but it does show what an ISO 12800 image looks like on the 5DIII.


Next step: more testing :-)

Tuesday, December 20, 2016

A new technique for Super Resolution Photography...maybe?

As readers of my posts know, I try and get as much as I can out of the (Canon) camera technology; helped, of course, by Magic Lantern.

So far, accounting for my bias towards landscapes, cityscapes and architecture photography, through various scripts (see the right hand side), we can automatically:

  • Take exposure brackets, ie according to the dynamic range of the scene. The camera works out the number of brackets to take;
  • Take focus brackets, from a focus point to the macro end of a lens, or the infinity end, or to the hyperfocal distance (HFD) or from one focus point to another. The camera handles the diffraction and the bracket to bracket overlap;
  • Take both exposure and focus brackets at the same time;
  • Take a simulated long exposure brackets, in Full Resolution Silent Picture mode, ie no shutter actions, or with the ‘normal’ mechanical shutter;
  • Take any number of brackets, ie to use when you are trying to remove people from the scene;
  • Move the lens to the HFD at the push of a button.

In this post, we are going to take the next step in automatic bracketing, namely: automatic bracketing for super resolution photography.

At the heart of super resolution photography (SRP) is trying to use the equipment you have, like a Canon 5D3 in my case, to emulate a, say, Canon EOS 5DS R. In this post I will show how to enhance the resolution of your camera without spending a penny on a new camera. In my case, using a 5D3, to a camera equivalent to 90 megapixels (MP)

SRP is not new. For example, Hasselblad had a 200 MP camera system, but that 200 MP images was made by a 50 MP sensor, using a special sensor-shift mechanism inside the camera. Typically it would take six separate images, each with slightly different sensor positions with only a pixel of difference between each shot. The camera would then automatically re-align those images and combine them together to produce a photo with four times the amount of resolution.

Last year the Olympus OM-D E-M5 II was announced as the first consumer level camera to feature this, sensor shift, technology. Similar to the Hasselblad , the OM-D E-M5 II takes 8 consecutive photographs with its 16 MP sensor. After shooting these 8 photos, once again each with a different sensor position, it then combines the data from all 8 images into a 40 MP plus image.

But what if you don’t have a camera that can shift the sensor? Are there other ways to achieve a similar result?

Bluntly: yes!

First, you can hand hold and shoot the 8, say, images and post process those images into a SR image in Photoshop. The technique requires hand holding, as taking the images on a tripod would result in image to image perfect alignment; and we need to the images to be different. Having said this, the multi stacking no sensor movement approach has some advantage in reducing noise, ie 8 images (albeit perfectly aligned) will give you a three stop reduction in noise, eg an ISO 800 shot will look like an ISO 100 one.

But such perfect alignment will not help us achieve the SR ‘uplift’.

The problem with hand holding is that you are restricted in your shutter speed, eg to 1/FL, or for an average photographer, say, faster than 1/40s.

To overcome the handholding limitations, I offer the following as an alternative technique, well at least for the Magic Lantern Canon shooter. I’m not aware of anyone else coming up with this idea: but I’m prepared to eat humble pie :-)

The key to my technique is to exploit a known ‘weakness’ in still photography lenses, ie the technique may not work with cine lens (although I can’t test this as I don’t have one).

The feature we are going to exploit is a normal lens’s focus breathing or focal length shortening, feature; which is the name given to the change of focal length (and hence angle of view and magnification) when the focus distance of a lens is changed. It is easy to see the impact of focus breathing by simply looking at an image on the Live View screen at say the closest focus point, and then move to infinity. The image changes on the sensor: which is exactly what we want!

Knowing all lenses exhibit focus breathing, led me to speculate if I could exploit this as form of sensor shifting, ie take multiple images at different focus points.

The immediate thought may be: no way, as the images will be differently out of focus between images. But this is where a pragmatic limitation comes in: use wide angle lenses and ‘limit’ things to the HFD.

As a reminder, if one focuses at the HFD, then every thing between the 1/2 HFD to infinity will be in focus, according the out-of-focus criterion you have chosen, eg the circle of confusion and, if you are being critical, the diffraction blur added in quadrature. Luckily for Magic Lantern shooter, all this Depth of Field maths is built in to ML and accessible via Lua scripting.

The other advantage of focusing at the HFD is that moving to infinity means that only 1/2 of the distance to the HFD is ‘lost’. Thus, say, if the HFD was at 2m, ie the DoF went from 1m to infinity, then moving the focus to infinity would result in an acceptable DoF from 2 to infinity, ie all we lost 1m in the near field.

This illustration shows the depth of field visually, with diffraction accounted for. It is taken from an excellent article here: http://www.georgedouvos.com/douvos/Image_Sharpness_vs_Aperture.html

Conceptually the ‘new’ (I think {;-}}) technique I’m offering goes like this:
  1. Compose your scene on a Tripod;
  2. Set the exposure for your scene, ie any exposure you like;
  3. Set the lens to the HFD and take an image;
  4. Move the lens towards infinity and take another image, and repeat until you are at infinity and have taken the required number of images, each with a slightly different image because of the focus breathing OR simply jiggle the lens via the IS;
  5. Ingest the images into your PC and stack them in Photoshop;
  6. Increase the image size by 200%;
  7. Auto align the images;
  8. Turn the image stack into a smart object;
  9. Run the stack mode as medium or mean (and wait a long time!);
  10. Collapse the stack an finish post processing.
The only real problem with the above is step 4, ie repeatedly touching the lens; as manual control of focus position is not good/repeatable as the lens gets towards infinity. The IS jiggle trick works, ie moving the images relative to each other 'randomly' by a pixel, but you have to keep manually intervening. 

But help is on hand in the form of my latest Auto Super Resolution Stacking Script (see below).

My Lua script runs under ML and will automatically control the lens (which must have AF). The script will find the HFD, and, in the updated version below, jiggle the lens.

The script only requires three inputs:

  • The user selected number of super resolution images to take, ie 4 to 20;
  • A user selected delay, 0 to 5 seconds;
  • What sensor shift strategy to use;
  • And whether to add a bookmark image at the start and end, ie a dark frame to differentiate the SR stack.
So what are the results?

Well you will have to wait to see as, so far in my experiments all I have done is to get the auto stacking script (see below) running and take a few SR image stacks in our back garden, using my 24mm F/4L at 24mm. The stack comprised of 9 images, covering the HFD (about 1.8m) to the ‘infinity’ end of the lens. Each images was the usual 30MB sized CR2 file, ie 5760 x 3840 pixels. The resultant SR image was 516MB and had 11645 x 7756 pixels. In future posts I will explore some test images.

So why would you bother?

Apart from the curiosity factor, which drives me, there are occasions when you need all that pixel real-estate, eg a super large print on a bill board. Another reason photographer’s enjoy having a lot of megapixels to play with is because it gives us a lot more room to crop and manipulate our images in post production.

Of course, the downside is the shear workload in simulating a super large ‘mega pixel’ sensor, ie a 90MP equivalent on my 5D3. Thus the technique is one to use with care!

For now, without evidence, I offer this ‘new’ approach to super resolution photography, ie shooting on a tripod at any shutter speed, with any (Canon EOS) camera, and look forward to hearing from others regarding their thoughts on this post, especially the Canon Magic Lantern users.


Auto Super Resolution Bracketing Script

--[[
This script simulates a super resolution stack for post processing...
by simulating sensor shifting by changing focus. Because of this, the script is 'limited' to taking...
sharp images between the HFD and infinity.
This script assumes you are on a tripod, ie not handholding as 'normal' in super resolution stacking (without a special sensor)
This version does NOT work with FRSP...at the moment :-)
Should be in LV
Canon review should be set to OFF
Usual caveat: script was written for my workflow and enjoyment, and 'only' tested on a 5D3

Version 0.2

*******************************************************************************************************
*                                                                                                      *
* If you reference this script, please consider acknowledging me: http://photography.grayheron.net/   *
*                                                                                                      *
*******************************************************************************************************
--]]

function my_shoot() -- to grab an image
    camera.shoot(false)
end

function jiggle(around)
    for i = 1, around do
    key.press(KEY.HALFSHUTTER)
    msleep(100)
    key.press(KEY.UNPRESS_HALFSHUTTER)
    end
end

function move() -- to HFD position irrespective of lens starting position
    if lens.focus_distance ~= 0 and lens.af then -- lens is OK
        if lens.dof_far ~= lens.dof_near then -- ok to move
            if lens.focus_distance < lens.hyperfocal then
                repeat
                    lens.focus(-1,2,false)
                until lens.focus_distance >= lens.hyperfocal
                repeat
                    lens.focus(1,1,false)
                until lens.focus_distance <= lens.hyperfocal
                repeat
                    lens.focus(-1,1,false)
                until lens.focus_distance >= lens.hyperfocal
            else
                repeat
                    lens.focus(1,2,false)
                until lens.focus_distance <= lens.hyperfocal
                repeat
                    lens.focus(-1,1,false)
                until lens.focus_distance >= lens.hyperfocal
            end
        else
            beep (3, 200 , 500)  -- warning message
            display.notify_box("Check aperture", 3000)
        end
    else
        beep (3, 200 , 500)  -- warning message
        display.notify_box("Check lens settings", 3000)
    end
end  

function check_lens_ready() -- just in case
        if lens.focus_distance ~= 0 and lens.af then -- lens is OK
            repeat
            msleep(100)
            until lens.focus(0,1,false)
        end
end

function check_bookend() -- adds an over exposed FRSP frame
    if supermenu.submenu["Bookends?"].value == "yes"
    then
        local tv = camera.shutter.ms
        local av = camera.aperture.value
        camera.shutter.ms = 1
        camera.aperture.apex = 9
        my_shoot()
        camera.shutter.ms = tv
        camera.aperture.value = av
    end
end

function go() -- and run the script
    local lens_ok = true
    local num_step = 0
    local del = 0
    local count = 0
    local inf = 100000 -- pseudo infinity
    local pos = 0
    menu.close()  -- just in case
    if lens.af and lv.enabled then -- alright to use script
        msleep(supermenu.submenu["Delay"].value * 1000)
        check_bookend() -- requested or not
        move() -- back to HFD to start super resolution bracketing
        count = 0
        for count = 1, supermenu.submenu["Number of images?"].value do
            count = count + 1
            my_shoot()
            if count ~= supermenu.submenu["Number of images?"].value then
                if supermenu.submenu["Sensor shift mode"].value == "IS Jiggle" then
                    jiggle(2)
                elseif supermenu.submenu["Sensor shift mode"].value == "Lens Move" then
                    check_lens_ready() -- just in case
                    lens.focus(-1,1,false) -- move towards infinity
                else
                    check_lens_ready() -- just in case
                    lens.focus(-1,1,false) -- move towards infinity
                    jiggle(2)
                end
            end
            msleep(200)
        end
        check_bookend() -- requested or not
        beep (3, 200 , 1000)  -- and display message
        display.notify_box("Script Finished Running", 5000)
    else -- something is wrong
        beep (3, 200 , 500)  -- and display message
        display.notify_box("Sorry, can't use the script", 5000)
    end
end

supermenu = menu.new
{
    parent = "Shoot",
    name = "Super Resolution Bracketing",
    help = "Works best with wide lens: be warned!",
    depends_on = DEPENDS_ON.LIVEVIEW,
    submenu =
    {
        {
            name = "Run Script",
            help = "Does what it says after pressing SET",
            depends_on = DEPENDS_ON.LIVEVIEW,
            select = function(this) if lv.enabled then task.create(go) end
            end,
        },
        {
                    name = "Number of images?",
                    help = "Number of super res images to take",
                    min = 4,
                    max = 20,
                    value = 4,
                    icon_type = ICON_TYPE.BOOL,
                    select = function(this,delta)
                        this.value = this.value + delta
                        if this.value > this.max then
                            this.value = this.min
                        elseif this.value < this.min then
                            this.value = this.max
                        end
                    end,
               },
        {
            name = "Delay",
            help = "Delays script start by stated number of secs",
                    min = 0,
                    max = 5,
                    value = 0,
                    icon_type = ICON_TYPE.BOOL,
                    select = function(this,delta)
                        this.value = this.value + delta
                        if this.value > this.max then
                            this.value = this.min
                        elseif this.value < this.min then
                            this.value = this.max
                        end
                    end,
        },{
            name = "Sensor shift mode",
            help = "Choose one of three strategies",
            choices = {"IS Jiggle","Lens Move","Both"},
        },
        {
            name = "Bookends?",
            help = "Places an underexposed frame at start and end of bracket set",
            choices = {"no","yes"},
        }
    }
}

Sunday, November 20, 2016

One of the UK's jewels

Wiki tells us that the Lake District National Park includes nearly all of the Lake District, though the town of Kendal and the Lakeland Peninsulas are currently outside the park boundary. 

The area, which was designated a national park on 9 May 1951 (less than a month after the first UK national park designation — the Peak District), is the most visited national park in the United Kingdom with 15.8 million annual visitors and more than 23 million annual day visits, the largest of the thirteen national parks in England and Wales, and the second largest in the UK after the Cairngorms. Its aim is to protect the landscape by restricting unwelcome change by industry or commerce. Most of the land in the park is in private ownership, with about 55% registered as agricultural land. Landowners include:
  • Individual farmers and other private landowners, with more than half of the agricultural land farmed by the owners.
  • The National Trust owns about a quarter of the total area (including some lakes and land of significant landscape value).
  • The Forestry Commission and other investors in forests and woodland.
  • United Utilities owns 8%
  • Lake District National Park Authority (3.9%) is based at offices in Kendal. It runs a visitor centre on Windermere at a former country house called Brockhole, Coniston Boating Centre, and Information Centres. It is reducing its landholding.
In common with all other national parks in England, there is no restriction on entry to, or movement within the park along public routes, but access to cultivated land is usually restricted to public footpaths, bridleways and byways. Much of the uncultivated land has statutory open access rights, which cover around 50% of the park.

The lakes and mountains combine to form impressive scenery. Farmland and settlement add aesthetic value to the natural scenery with an ecology modified by human influence for millennia and including important wildlife habitats. The Lake District failed in a previous attempt to gain World Heritage status as a natural World Heritage Site, because of human activities, such as commercial forestry, which have adversely impacted the park's assessment. However, in 2016 the English Lake District bid for World Heritage Status was submitted to UNESCO in the category of cultural landscape. A decision is expected in 2017.

The image below was taken on the eastern shore of Thirlmere, looking north. The word Thirlmere, appears to have its roots in 'Leathes-Water', which was also 'Thirlmere' or 'Wythburn-water' 1769. Probably 'the lake with/at the narrowing' from OE þyrel 'aperture', pierced hole' plus OE mere 'lake'.

Thirlmere was constructed in the 19th century by the Manchester Corporation to provide the burgeoning industrial city of Manchester with water supplies. The 96 mile-long Thirlmere Aqueduct still provides water to the Manchester area.

The image was captured with my Canon 5D3 using a Sigma 12-24mm lens at 12mm. Nine images were taken: three exposure brackets at three different focus points. All the brackets, exposure and focus, were automatically calculated and set in the camera, using Version 4 of my Magic Lantern Lua Auto Landscape Bracketing script (see link on the right).


Tuesday, November 15, 2016

Another little tweak

For those that value getting the best out of your Canon ML enhanced camera, you may be interested in the latest version of my Auto HFD script: download it from the right.

This version first optimizes the aperture for maximum depth of field, before moving the lens to the HFD position.

The lens must report focus distance and be in AF mode. The camera does not need to be in LV, but can be. Also the lens position can be anywhere and the aperture set to any value.

The script will run by simply pressing the MENU key, followed by the INFO key (but you can change these two keys if you wish).

After the script has run, you need to tweak the exposure or run the auto ETTR via ML or the ML auto bracketing for a fully optimized capture.

As usual, I welcome feedback on the script, which I have tested on my 5D3.

Wednesday, November 2, 2016

Auto Hyperfocal Distance

Things have been busy for a while and photography has suffered. Nevertheless, I did find time to create a small script that some may find of value.

All it does is provide an automatic way of moving the lens to the hyperfocal focus point.

The usual caveat applies, ie you need a Canon camera running Magic Lantern and the Lua module enabled.

In addition, the lens, which can be non-Canon, needs to report focus distance and be in AF mode. The camera also needs to be in Live View, and you should set the Depth of Field option in ML, ie put this in diffraction aware mode for best results.

The script can be downloaded from the link on the right.

To use, simply use the modified key sequence of your choice, in the script you download I use MENU and INFO keys, which work well on my 5D3.

The use the MENU key as normal, simply press the MENU key twice. 

The use the MENU key in the modified state, ie moving the lens to the HFD point, simply press MENU, followed by INFO.

As usual, I welcome any feedback.

Monday, October 10, 2016

Dorset: Long Exposure Trip

After several weeks of 'camera inactivity', this weekend was a chance to get away and ‘do some photography’: in particular Long Exposure photography with my new ND filter system.

For the wide angle photographers out there, you know that ‘normal’ ND filters break down as you start to go wide. For instance, my Sigma full frame 12-24mm WA lens, can take a screw in front element ND filter, but only at 24mm, ie at 12mm you just get the circular cut off of an 82mm ND.

To get round ultra wide angle lens problems, I have adopted two solutions. The first is to use a Vizelex M645-EOS ND Throttle adapter on my Canon 5D3, with, in my case, a Mamiya-Sekor 45mm prime. This combination allows me to put a variable ND filter at the back of the lens, instead of on the front. This way, I get no strange ND effects, such as the dreaded X effect. The down side of this arrangement is that I’m ‘limited’ to 45mm focal length, ie not very wide.

The second solution is what I wanted to experiment with this weekend. That is my new Fotodiox Wonderpana system, that puts a 145 diameter filter system on the front of most WA lenses, including my Sigma 12-24mm. https://www.fotodioxpro.com/wonderpana-66-freearc-essentials-nd-kit.html


Although I have the 6.6 inch square filter adapter, this last weekend I only explored the 145mm diameter ND1000 filter; and as we were in Dorset, the obvious target was to see what I could do with a seascape. So I chose Durdle Door and St Oswald's Bay, which is sometimes called the Man of War Cove, on the UK’s Jurassic Coast.

This first image was taken, handheld, at F/8, ISO 100, 1/60s and at 24mm. I first used Magic Lantern to establish a RAW ETTR setting and took the image with Dual-ISO switched on, ie an inter-laced ISO 100/800 image. I first processed the Dual RAW in Lightroom and then undertook a round trip to Photoshop to carry out some luminosity mask work.


As for Durdle Door, I managed to get there early enough, so that my photography was not complicated by too many tourists. However, as you can see from the elevated view of Durdle Door that I took walking down to the beach, one family still managed to beat me to the shoreline :-)

For the long exposure, I used my Sigma 12-24 with the Wonderpana attachment, to hold the 145mm diameter ND 1000 filter. I also used the Magic Lantern LE ND Bulb module to make exposure setting easier. For instance, I simple composed and set the exposure without the ND filter, then screwed on the filter and used the ML ND Bulb module to grab the LE. In this case a 91 second exposure, which I processed in B&W.

One mistake I made, was to only take a travel tripod with me, ie not my heavy duty tripod. So I’m aware that the image is soft in places, which I attribute to a small amount of tripod-relate ‘movement/vibrations’, ie the tripod was placed on a shingle beach. This created issues with this second LE, taken at 124 seconds down on the shoreline at St Oswald's Bay.

 
The rest of the weekend was spent relaxing in a very nice hotel, The Priory at Wareham, http://theprioryhotel.co.uk/, where we enjoyed great food and wine. It was also a chance to visit Kingston Lacy.

Wiki tells us that Kingston Lacy is a country house and estate near Wimborne Minster in Dorset. It was for many years the family seat of the Bankes family who lived nearby at Corfe Castle until its destruction in the English Civil War after its incumbent owners, Sir John Bankes and Dame Mary, had remained loyal to Charles I.

The house was built between 1663 and 1665 by Ralph Bankes, son of Sir John Bankes, to a design by the architect Sir Roger Pratt. It is a rectangular building with two main storeys, attics and basement, modeled on Chevening in Kent. The gardens and parkland were laid down at the same time, including some of the specimen trees that remain today. Various additions and alterations were made to the house over the years and the estate remained in the ownership of the Bankes family from the 17th to the late-20th century.

For me it was a chance to take a few snaps:







dd