August 2009: Integrated Software Distortion Correction Reduces Lens Complexity

Background

Most dSLR zoom lenses use physical design (lens elements) to compensate for lens distortions near the ends of the zoom range.  Typically, a lens needs to be designed such that barrel distortion is well controlled at the wide angle end of the zoom range and pincushion distortion is controlled at the telephoto end of the range.  The most challenging aspect of lens design is often being able to control barrel distortion on lenses that have super-wide-angle focal lengths.  This is why many online review sites test lens distortions by taking photos of a grid, particularly at the widest angle of a zoom lens.  If the grid appears bowed out near the edges of the photo at wide angles for example, distortion is said to be noticeable.  Usually the review will go on to indicate whether the distortion disappears after zooming in and at what zoom range the distortion is noticeable to viewers.  Panasonic with their latest 14-140 Micro Four Thirds lens, however, may be starting a new trend in lens design.  Instead of designing the lens to optically control distortions at wide angle, they depend on software to correct barrel distortions at wide angle focal lengths.  While the lens optics actually suffer from severe barrel distortion from about 14-25mm, the camera compensates for the distortion by applying a software barrel distortion filter to the image (note that the Panasonic G1, GH1 and the Olympus E-P1 are currently the only cameras compatible with the Panasonic 14-140 lens).  This allows the lens to be less complex, lighter, and in the long run: cheaper.  Is this a trend that will catch on?  Will other manufacturers start "outsourcing" distortion correction to software, allowing lenses to be less complex and cheaper?  Let's take a look at this concept.

As discussed above, optical control of barrel distortion in lenses involves increasing the complexity of the lens.  Lenses that are more complex are heavier, more expensive to manufacture, and the very methods used to control distortions can cause other unwanted effects such as a reduction in light transfer, reduction in (particularly edge) sharpness, and front/back focus issues as better tolerances are required to achieve good focus.  So why not delegate lens distortion correction to software every time, reducing lens complexity and cost?  Well, not so fast.  There are trade-offs involved with software lens distortion correction as well and the method is not viable for traditional dSLR cameras that utilize a mirror.

Any time you apply a software correction that moves pixels around and resizes, expands, or contracts parts of the image, you are going to lose resolving power and hence lose detail.  Everything comes at a price.  Fortunately, software lens distortion corrections are quite good at what they do and when you compare any sharpness/detail lost in software correction to the sharpness/detail loss that you will get with optical correction (and there will be some), software correction seems like a viable solution.  Think about it this way.  The optical complexity that you must add to a lens to control barrel distortion will result in some loss of detail, usually more at the edges of the photo.  If your (let's say) 12 MP image sensor has enough resolution to be able to pick up that edge softness, will letting the lens distort and correcting the distortion in software result in just as much softening?  Less?  More?  As the resolution of image sensors increases, delegating some of the lens distortion correction to software starts to make more sense.  A good balance is being able to develop lenses at a more reasonable cost (and weight) while putting some of those pixels on that high resolution sensor to good use, allowing them to do some of the work that has traditionally been forced upon the optics.

 

The new Micro Four Thirds lenses are interesting because they depend on software for some of their optical characteristics.  Panasonic doesn't want users to have to deal with (or see) how badly the 14-140 lens distorts optically so they've built the software correction into the digital pipeline.  This means that the electronic viewfinder and LCD on compatible Micro Four Thirds cameras (right now, the Panasonic G1, GH1, and Olympus E-P1) must correct the distortions in real time using software inside the camera!  Yes, if you set the lens to 14mm and look through the viewfinder or LCD, you see a nice image with very little barrel distortion.  That's because the camera is correcting the distortion in real time taking what is hitting the sensor, correcting it, and then passing the distortion-corrected version along to the EVF or LCD.  The Micro Four Thirds cameras can do this because you are seeing a rendition of the image that is hitting the sensor: it can be "modified" or "fixed" before it is displayed in the EVF or on the LCD.  On a typical dSLR, this cannot be done: the lens optics must control barrel distortion at wide angles because you are looking through the lens... there is no opportunity to "intercept" what you see and fix it.

What this means is that this type of software auto-correction is probably only viable for mirrorless dSLR-like cameras such as those in the Micro Four Thirds family.  It does give the mirrorless cameras an advantage, however.  By being able to offload some of the optical responsibilities of the lens to software, ultimately you end up with better lens quality for a given price.  In practical terms, the 14-140 lens is incredible.  Due to its reduced complexity, it produces incredibly sharp photos and the software lens corrections take over where needed at the wide angle end of the zoom range.  The bottom line is that for about $800, you get a lens that can deliver optical quality comparable to a lens costing double (if not more) that amount.  But... as mentioned, the lens is part of a system.  Without the software lens distortion correction, most people would find performance of the lens at wide angles quite disappointing due to the rather extreme amount of optical distortion.  Part of the "system" that I refer to must also be able to handle raw photos!  Yes, the raw files are really raw and if you decode them in their raw form, they'll show the severe distortion of the lens at wide angles.  I took a photo of some window shades just to prove this point and here are the results:

JPEG straight from the Panasonic GH1	Raw photo decoded with Qimage

Raw photo decoded with dcraw

The above shot was taken in raw+JPEG mode with the GH1 and 14-140 lens set to 14mm.  Notice how the JPEG, when downloaded straight from the camera, already has the software lens distortion applied.  When shooting in JPEG mode, the photographer would not even be aware that the lens actually has some pretty significant optical barrel distortion.  The Qimage developed raw photo also has the proper lens distortion correction applied.  Remember, the lens distortion is part of the image acquisition process now, so for the raw file to be properly decoded, the optical data that is stored in the raw photo for that lens must be applied.  Take a look at the bottom example that shows how dcraw renders the same photo.  Since dcraw doesn't use the lens distortion correction data and displays the raw file as truly raw, you can see how much distortion the lens itself really has.  Most "high end" raw tools such as PhotoShop, SilkyPix, and Qimage will render the proper image but raw utilities like dcraw that are designed to decode the "truly raw" data will show exactly what the sensor sees.  As you can see from the above, significant optical distortions are mitigated effectively by software.  In addition, the software fix is so good that very little detail is lost in the final/fixed photo!