improving contrast with the local laplacian filter

sometimes difficult lighting situations arise which, when taking photographs, result in unappealing pictures. for instance very uniform lighting on a cloudy day may give dull results, while very contrasty illumination (such as back lit) may require to compress the contrast to embrace both highlights and shadows in the limited dynamic range of the output device.

refer to the following two shots as examples:

Have you ever over-exposed your images? Have you ever noticed that your images look flat and dull after you apply negative exposure compensation? Even though the over/underexposed warning says there is no overexposure? Have you ever wondered what is going on? Read on.

the problem

First, why would you want to know which pixels are overexposed, clipped?

Consider this image:

… Why is the sky so white? Why is the image so flat and dull?

modern sensor capture an astonishing dynamic range, namely some sony sensors or canon with magic lantern’s dual iso feature.

this is in a range where the image has to be processed carefully to display it in pleasing ways on a monitor, let alone the limited dynamic range of print media.

example images

use graduated density filter to brighten foreground

using the graudated density iop works well in this case since the horizon here is more or less straight, so we can easily mask it out with a simple gradient in the graduated density module. now what if the objects can’t be masked out so easily?

[update 2016/07/31: there was a section about intermediate export to csv and manually changing that file. this is no longer needed, exporting the style directly from darktable-chart is fine now.]

motivation

for raw photography there exist great presets for nice colour rendition:

• in-camera colour processing such as canon picture styles
• fuji film-emulation-like presets (provia velvia astia classic-chrome)
• pat david’s film emulation luts

unfortunately these are eat-it-or-die canned styles or icc lut profiles. you have to apply them and be happy or tweak them with other tools. but can we extract meaning from these presets? can we have understandable and tweakable styles like these?

Today we are happy to announce a big new feature that we will not only ship with the big 2.0 release later this year but also with our next point release, 1.6.4, which is due in about a week: even more darkroom modules!

One of the big strengths of darktable has always been its varied selection of modules to tweak your image. However, that has also been one of the main points of criticism: too much, too many and too complicated to grasp. To make it easier for the user to deal with the flood of tools darktable has had the “more modules” list since many years. It changed its appearance a few times, we added module categories, allowed to select favorite modules, and all of that has proven to be useful. Thus there have always been people that approached us with great new ideas for new modules, especially since we moved to GitHub a while ago with its powerful Pull Request system, yet we couldn’t accept many of them. Some were not that great codewise, some didn’t really fit our product vision – and then there were some that looked nice and certainly benefited some users, but we felt it wasn’t generic enough to justify polluting our module list even more. Of course this was a bad situation, after all these people invested quite some time into providing us with a new feature that we turned down. No one likes to waste their time.

If you overexpose a photo with your digital camera you are in trouble. That’s what most photography related textbooks tell you – and it’s true. So you better pay close attention to your camera’s metering while shooting. However, what to do when the “bad thing” happened and you got this one non-repeatable shot, which is so absolutely brilliant, but unfortunately has some ugly signs of overexposure?

In this blog article I’d like to summarize how darktable can help you to repair overexposed images as much as possible. I’ll cover modules which have been part of darktable for a long time but also touch the brand new module “color reconstruction”.

In darktable, selective editing was a long awaited feature. Our development version now allow limiting module effects to a region of the image.

Remember the old times, the red light of the darkroom, the smell of the developing bath …

Remember when you were using your hands or a small piece of cardboard to achieve some masking …

Now you can do that in darktable.

example

let take this photo as an example.

One of the upcoming new feature in darktable is the ability to use the same development module several times. By applying the same module multiple times and combining them with blendif it is possible to do some effects that could not be achieved previously without using external tools like the gimp.

Modules that can be instantiated multiple times have a new icon in their header, next to the “reset” button. Clicking that icon will open a pop-up menu that allows you to create a new instance of the module, change the order in which the different instances are applied, or delete an instance of the module. Each instance can have its own parameters, can be activated or deactivates separately and can use presets. Note that the last instance of a module can’t be deleted, it can only be deactivated.

… and how to get rid of it.

[update 02/05/2018 The information how to create camera noise profiles is outdated please read this tutorial instead!]

[update 20/12/2012: ‘how to profile your camera’ includes instructions with the new gen-profile script]

[update 15/12/2012: no more recompile needed, updated the instructions in the benchmark section and how to run make.sh.]

to summarize the current situation in dt: we have a lot of cool tools wrapped around great algorithms with almost all the knobs you need to get perfect results. while you can actually get really great results it’s this sheer number of knobs that makes finding a good parameter set quite a time consuming task. even creating per-iso presets is not straight forward, as most of the current modules depend on a lot more stuff early on in the pipe (whitebalance, exposure, basecurve, etc).

in an ideal world, an image is piecewise smooth. it has soft gradients, some detail and edges. in particular there’s no noise and the edges are sharp. given these assumptions, you can do a lot of cool things to your pictures, using techniques like frequency space editing, wavelets, or local histograms.

darktable’s equalizer module demonstrates some of this, using the wavelet approach. you can use it to sharpen and denoise, enhance or attenuate certain frequencies in your image, while keeping the edges intact.

Recent builds from git will bring you a new executable, “darktable-cli”. With this tool you can export images using the processing power of darktable on the command line.

The simplest way to call the utility is

darktable-cli <input> <output>

This will take the input image, look for the XMP file associated with it, process it at maximal resolution and write the output to output, trying to guess the output format from the output filename. You can also explicitly give a XMP file by running

darktable is all about providing you the best tools in order to get the most out of your photographies. This blog entry will explain how an existing feature can help you get more detailed exports and it will try to give you a glimpse of what is cooking in an unscheduled but upcoming version of darktable for even better detail preservation.

Make sure to enable High Quality Resampling for exporting your photographies

In darktable, the pixel pipeline is responsible for processing your photography from demosaic up to the point it is passed over to the output subsystem (saving in tiff, jpeg…).

For quite some time darktable supports tethering your camera. What was missing all the time was live view. This is about to change though. If you use master (and have a camera supporting it) you can now either use the eye button in the “camera settings” module or hit ‘v’ on your keyboard to start live view from within tethering mode. Since the preview is scaled to fit your screen it might be a good idea to hide some of the side panels. If you are using a Canon EOS (I only tested this with my 40D) you can also use your middle mouse button to zoom into the preview. Another click brings you back to regular live view.

or “If one slider is not enough”

Diligent readers of our small blog series are already aware of the blending feature that darktable offers as part of many modules. Instead of just handing over their result to the subsequent module in pixelpipe, “blending modules” take a moment to reconsider. Based on the blend setting they will take their original output together with their input and do a re-processing. As an example refer to here, where we took blend mode “overlay” with module “lowpass” to do shadow recovery.

Many readers will have already heard about GPU processing and the fact that darktable can make use of OpenCL to improve performance. As we still lack a detailed documentation of that topic, please find here a few explanations and howtos.

The Background

Processing high resolution images belongs to the more demanding tasks in modern computing. Both, in terms of memory requirements and in terms of CPU power, getting the best out of a typical 15, 20 or 25 Megapixel image can quickly bring your computer to its limits.

disclaimer: this is only to tease you and will not make it into the next release, but the one after …

when reading gui-guidelines, most of them seem to be too general, or too specific for a certain kind of programming environment (gnome and gtk, qt, etc).

for our purposes, i found the fundamental principles of the bauhaus school to be more appropriate. radical simplicity, no unnecessary shape or line, such as a pseudo 3d-bevel-border around ui elements. the underlying grid should be visible because all elements are aligned, not because it is drawn. only simple shapes are allowed, and everything should integrate seamlessly into the background of the panel.

One of the remaining shortcomings of digital cameras is their rather low dynamic range in comparison to analog – especially black-and-white – film. Scenes with strong differences between highlights and shadows are very difficult to capture. Even if they are exposed properly with no blown-out highlights they will too often only give acceptable results after extensive post-processing.

Fortunately, darktable is progressing with a high pace. Some days ago I wrote an article on how to recover shadows with a technique using lowpass filter plus blend mode (" Using lowpass filter to recover shadows "). In between a new, even better module called “shadows and highlights” was integrated into darktable, that obsoletes this technique.

or “How to bring the jungle back”

Since its early beginnings darktable has a tone curve module that is able to alter the gray level distribution of an image. Recently we did an enhancement: tone curve is now able to control the full Lab color space with separate curves for the L, a and b channel. People who are used to curve tools in RGB, at first might get puzzled over the results of these three curves; they show marked differences to the typical RGB curve. Especially a and b channels need to be dealt with in the right way; not doing so will give you strong off-colors. To spare you frustration here are some explanations and examples.

you might have noticed our equalizer tool, and been confused by it and the many controls. that’s probably partly because you didn’t see a similar thing before, we had to develop it first.

very short history

behind the ui is a powerful frequency domain processing technique, based on wavelets. the most commonly used wavelets are based on the lifting scheme [swe97], work in a data-independent way, and are decimated (i.e. the coarse coefficients are much more sparse than the fine ones). while this allows for very fast implementations, data-independent wavelets can lead to blurring or ringing around edges (depending on what you do to the coefficients during image enhancement). raanan fattal had quite an inspiring paper at siggraph 2009 [fat09] introducing edge weights into the lifting scheme to overcome this. while the method is fast, produces okay results (the legacy equalizer version I was based on this), it has some problems caused by the decimation. in particular, decimated wavelets are not shift-invariant, which means your results will change if you slightly crop the image for example. actually the same author also had a solution for that earlier already [far07], in a different context, and not quite as fast.

since i probably tend to make this more technical than any reader would like to, here’s the take home message:

• much faster import of folders
• much faster thumbnail creation for first-time images
• much improved scalability wrt concurrency
• much improved scalability wrt total number of images in your database (should be good up to around 500 million images)
• much improved robustness (no more deleting ~/.cache/darktable/mipmaps all the time, yay)

some context:

darktable’s light table mode shows you your image collection in arbitrary order and filtered by arbitrary queries to the underlying database. that means that there is quite some uncertainty about which thumbnails you are going to need in the next second. so we rely on a caching mechanism that stores a few thumbnails, regenerates them as needed, and evicts not so frequently used thumbnails from the cache.

The last time I posted to this blog it was my April Fool’s Joke about a file manager (which happened to be just an embedded shell). Since a few people didn’t like that at all I want to assure you that this is no joke at all. However, if you are a Windows user and feel easily offended, then stop reading now.

Still here? Great. I managed to compile darktable for Windows.

Up till recently the usermanual has been only available for those who got the source and the necessary tools to generate and even than it would generate a HUGE pdf file due to the amount of high resolution of screenshots. I revisited the build system with the goals of shrink-en the size of the final PDF and to produce an html output that could be integrated in our website.

“Why does darktable need a blog?” you might ask. The answer is quite simple: While we do have a mailing list (actually there are three) which is quite active, most decisions and background informations happen in IRC (#darktable on FreeNode). This is great for fast communications, but it doesn’t allow interested users to follow what we actually (want to) do.

I hope that this blog will be an annotated commit log on the one hand showing the progress we make and a persistent archive of important decisions taken by us on the other hand.

Every now and then the question arises why we don’t have a button in darktable to open the current image in GIMP. Everytime I answer more or less the same.

The arguments of those requesting the button are along the line of “$PROGRAM has it, so it shouldn’t be hard to do” and “I really need to do some small retouching, so it would save me lots of time”.

Well, to understand why we don’t have it I have to stress two features of darktable. First of all, every action is done non-destructive. What that means is that you never edit the actual data in your raw file, but “record” a list of actions (the history in darkroom mode and the XMP files) which shall be executed to get the final image. This list can be changed afterwards without negative side effects since those actions can just be recomputed. The second nice thing in darktable is that we work with high bit depths (32 bit floating point) to get the best possible result.