Chapter 9. Scripting with Lua

darktable comes with a versatile scripting interface for functionality enhancement.

9.1. Lua usage

Lua can be used to define actions which darktable will perform whenever a specified event is triggered. One example might be calling an external application during file export in order to apply additional processing steps outside of darktable.

darktable uses Lua, which is an independent project founded in 1993, providing a powerful, fast, lightweight, embeddable scripting language. Lua is widely used by many open source applications, in commercial programs, and for games programming.

darktable uses Lua version 5.2. Describing the principles and syntax of Lua is beyond the scope of this usermanual. For a detailed introduction see the Lua reference manual.

9.1.1. Basic principles

At startup, darktable will automatically run two Lua scripts:

  • a script called luarc in $DARKTABLE/share/darktable

  • a script called luarc in the user's configuration directory

$DARKTABLE is used here to represent your system's darktable installation directory.

This is the only time darktable will run Lua scripts by itself. The script can register callbacks to perform actions on various darktable events. This callback mechanism is the primary way to trigger lua actions.

9.1.2. A simple lua example

Let's start with a simple example. We will print some code on the console. Create a file called luarc in darktable's configuration directory (usually ~/.config/darktable/) and add the following line to it:

print("Hello World !")

Start darktable and you will see the sencence Hello World ! printed on the console. Nothing fancy but it's a start...

At this point, there is nothing specific to darktable in the script. We simply use the standard print function to print a string. That's nice and all, but we can do better than that. To access the darktable API you first need to require it and save the returned object in a variable. Once this is done you can access the darktable API as subfields of the returned object. All of this is documented in darktable's Lua API reference manual (see Section 9.2, “Lua API”).

local darktable = require "darktable"
darktable.print_error("Hello World !")

Run the script ... and nothing happens. The function darktable.print_error is just like print but will only print the message if you have enabled lua traces with -d lua on the command line. This is the recommended way to do traces in a darktable lua script.

9.1.3. Printing labeled images

This first example showed us the very basics of lua and allowed us to check that everything is working properly. Let's do something a little bit more complex. Let's try to print the list of images that have a red label attached to them. But first of all, what is an image?

local darktable = require "darktable"
local debug = require "darktable.debug"
print(darktable.debug.dump(darktable.database[1]))

Running the code above will produce a lot of output. We will look at it in a moment, but first let's look at the code itself.

We know about requiring darktable . Here, we need to separately require darktable.debug which is an optional section of the API that provides helper functions to help debug lua scripts.

darktable.database is a table provided by the API that contains all images in the database (currently visible or not, duplicate or not...). Each entry in the database is an image object. Image objects are complex objects that allow you to manipulate your image in various ways (it is all documented in the types_dt_lua_image_t section of the API manual). To display our images, we use darktable.debug.dump which is a function that will take anything as its parameter and recursively dump its content. Since images are complex objects that indirectly reference other complex objects, the resulting output is huge. Below is a cut down example of the output.



          
toplevel (userdata,dt_lua_image_t) : /images/100.JPG
   publisher (string) : ""
   path (string) : "/images"
   move (function)
   exif_aperture (number) : 2.7999999523163
   rights (string) : ""
   make_group_leader (function)
   exif_crop (number) : 0
   duplicate_index (number) : 0
   is_raw (boolean) : false
   exif_iso (number) : 200
   is_ldr (boolean) : true
   rating (number) : 1
   description (string) : ""
   red (boolean) : false
   get_tags (function)
   duplicate (function)
   creator (string) : ""
   latitude (nil)
   blue (boolean) : false
   exif_datetime_taken (string) : "2014:04:27 14:10:27"
   exif_maker (string) : "Panasonic"
   drop_cache (function)
   title (string) : ""
   reset (function)
   create_style (function)
   apply_style (function)
   film (userdata,dt_lua_film_t) : /images
      1 (userdata,dt_lua_image_t): .toplevel
      [......]
   exif_exposure (number) : 0.0062500000931323
   exif_lens (string) : ""
   detach_tag (function): toplevel.film.2.detach_tag
   exif_focal_length (number) : 4.5
   get_group_members (function): toplevel.film.2.get_group_members
   id (number) : 1
   group_with (function): toplevel.film.2.group_with
   delete (function): toplevel.film.2.delete
   purple (boolean) : false
   is_hdr (boolean) : false
   exif_model (string) : "DMC-FZ200"
   green (boolean) : false
   yellow (boolean) : false
   longitude (nil)
   filename (string) : "100.JPG"
   width (number) : 945
   attach_tag (function): toplevel.film.2.attach_tag
   exif_focus_distance (number) : 0
   height (number) : 648
   local_copy (boolean) : false
   copy (function): toplevel.film.2.copy
   group_leader (userdata,dt_lua_image_t): .toplevel
   


        

As we can see, an image has a large number of fields which provide all sort of information about it. We are interested in the red label. This field is a boolean, and the documentation tells us that it can be written. We now just need to find all images with that field and print them out.

darktable = require "darktable"
for _,v in ipairs(darktable.database) do
  if v.red then
    print(tostring(v))
  end
end

This code should be quite simple to understand at this point, but it contains a few interesting aspects about lua that are worth highlighting:

  • ipairs is a standard lua function that will iterate through all numeric indices of a table. We use it here because darktable.database has non-numeric indices which are functions to manipulate the database itself (adding or deleting images, for example).

  • Iterating through a table will return both the key and the value used. It is conventional in lua to use a variable named _ to store values that we don't care about.

  • Note that we use the standard lua function tostring here and not the darktable specific darktable.debug.dump . The standard function will return a name for the object whereas the debug function will print the content. The debug function would be too verbose here. Once again, it is a great debug tool but should not be used for anything else.

9.1.4. Adding a simple shortcut

So far, all our scripts have done things during startup. This is of limited use and doesn't allow us to react to real user actions. To do more advanced things we need to register a function that will be called on a given event. The most common event to react to is a keyboard shortcut.

darktable = require "darktable"

local function hello_shortcut(event, shortcut)
darktable.print("Hello, I just received '"..event..
       "' with parameter '"..shortcut.."'")
end

darktable.register_event("shortcut",hello_shortcut,
       "A shortcut that print its parameters")
      

Now start darktable, go to preferences => shortcut => lua => A shortcut that print its parameters assign a shortcut and try it. You should have a nice message printed on the screen.

Let's look at the code in details. We first define a function with two parameters. These parameters are strings. The first one is the type of event that is triggered ( "shortcut" ) and the second one is what shortcut specifically ( "A shortcut that print its parameters" ). The function itself calls darktable.print that will print the message as an overlay in the main window.

Once that function is defined, we register it as a shortcut callback. To do that we call darktable.register_event which is a generic function for all types of events. We tell it that we are registering a shortcut event, then we give the callback to call and last, we give the string to use to describe the shortcut in the preference window.

Let's try a shortcut a little more interactive. This one will look at the images the user is currently interested in (selected or moused-over) and will increase their rating.

darktable = require "darktable"

darktable.register_event("shortcut",function(event,shortcut)
    local images = darktable.gui.action_images
    for _,v in pairs(images) do
      v.rating = v.rating + 1
    end
  end,"Increase the rating of an image")
      

At this point, most of this code should be self explanatory. Just a couple of notes:

  • Instead of declaring a function and referencing it, we declare it directly in the call to darktable.register_event this is strictly equivalent but slightly more compact.

  • image.rating is a field of any image that gives its rating (between 0 and 5 stars, -1 means rejected).

  • darktable.gui.action_images is a table containing all the images of interest. darktable will act on selected images if any image is selected, and on the image under the mouse if no image is selected. This function allows to easily follow darktable's UI logic in lua.

If you select an image and press your shortcut a couple of time, it will work correctly at first but when you have reached five stars, darktable will start showing the following error on the console:



          


LUA ERROR : rating too high : 6
stack traceback:
[C]: in ?
[C]: in function '__newindex'
  ./configdir/luarc:10: in function <./configdir/luarc:7>
      LUA ERROR : rating too high : 6
  

          


        

This is lua's way of reporting errors. We have attempted to set a rating of 6 to an image, but a rating can only go as high as 5. It would be trivial to add a check, but let's go the complicated way and catch the error instead.

darktable.register_event("shortcut",function(event,shortcut)
    local images = darktable.gui.action_images
    for _,v in pairs(images) do
      result,message = pcall(function()
        v.rating = v.rating + 1
        end)
      if not result then
        darktable.print_error("could not increase rating of image "..
          tostring(v).." : "..message)
      end
    end
end,"Increase the rating of an image")

pcall will run its first argument and catch any exception thrown by it. If there is no exception it will return true plus any result returned by the function; if there is an exception it will return false and the error message of the exception. We simply test these results and print them on the console...

9.1.5. Exporting images with Lua

We have learned to use lua to adapt darktable to our particular workflow, let's look at how to use lua to easily export images. darktable can easily export images to quite a few online services but there are always more. If you are able to upload an image to a service via the command line then you can use lua to integrate it into darktable's user interface.

In this next example we will use lua to export via scp. A new storage will appear in darktable's UI which will export images to a remote target via ssh's copy mechanism.

darktable = require "darktable"

darktable.preferences.register("scp_export","export_path",
  "string","target SCP path",
  "Complete path to copy to. Can include user and hostname","")

darktable.register_storage("scp_export","Export via scp",
  function( storage, image, format, filename,
     number, total, high_quality, extra_data)
    if coroutine.yield("RUN_COMMAND","scp "..filename.." "..
      darktable.preferences.read("scp_export",
         "export_path","string")) then
      darktable.print_error("scp failed for "..tostring(image))
    end
end)

darktable.preferences.register will add a new preference to darktable's preference menu. scp_export and export_path allow us to uniquely identify our preference. These fields are reused when we read the value of the preference. The string field tells the lua engine that the preference is a string. It could also be an integer, a filename or any of the types detailed in API manual relating to types_lua_pref_type . We then have the label for the preference in the preference menu, the tooltip when hovering over the value and a default value.

darktable.register_storage is the call that actually registers a new storage. The first argument is a name for the storage, the second is the label that will be displayed in the UI and last is a function to call on each image. This function has a lot of parameters, but filename is the only one we use in this example. It contains the name of a temporary file where the image was exported by darktable's engine.

This code will work but it has a couple of limitations. This is just a simple example after all:

  • We use preferences to configure the target path. It would be nicer to add an element to the export UI in darktable. We will detail how to do that in the next section

  • We do not check the returned value of scp. That command might fail, in particular if the user has not correctly set the preference.

  • This script can't read input from the user. The remote scp must use password-less copy. Scp can't be provided a password easily, so we will leave it that way

  • There is no message displayed once the example is done, only the progressbar on the lower left side tells the user that the job is done.

  • We use coroutine.yield to call an external program. The normal os.execute would block other lua codes from happening.

9.1.6. Building User Interface elements

Our previous example was a bit limited. In particular the use of a preference for the export path wasn't very nice. We can do better than that by adding elements to the user interface in the export dialog.

UI elements are created via the darktable_new_widget function. This function takes a type of widget as a parameter and returns a new object corresponding to that widget. You can then set various fields to that widget to set its parameters. You will then use that object as a parameter to various functions that will add it to the darktable UI. The following simple example adds a lib in the lighttable view with a simple label

local my_label = darktable.new_widget("label")
my_label.label = "Hello, world !"


dt.register_lib("test","test",false,{
    [dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_LEFT_CENTER",20},
    },my_label)
    

There is a nice syntactic trick to make UI element code easier to read and write. You can call these objects as functions with a table of key values as an argument. This allows the following example to work. It creates a container widget with two sub-widgets. A label and a text entry field.

	local my_widget = darktable.new_widget("box"){
		orientation = "horizontal",
		darktable.new_widget("label"){ label = "here => " },
		darktable.new_widget("entry"){ tooltip = "please enter text here" }
	}

Now that we know that, let's improve our script a bit.

darktable = require "darktable"

local scp_path = darktable.new_widget("entry"){
  tooltip ="Complete path to copy to. Can include user and hostname",
  text = "",
  reset_callback = function(self) self.text = "" end
}


darktable.register_storage("scp_export","Export via scp",
  function( storage, image, format, filename,
     number, total, high_quality, extra_data)
    if coroutine.yield("RUN_COMMAND","scp "..filename.." "..
      scp_path.text
    ) then
      darktable.print_error("scp failed for "..tostring(image))
    end
    end,
    nil, --finalize
    nil, --supported
    nil, --initialize
    darktable.new_widget("box") {
    orientation ="horizontal",
    darktable.new_widget("label"){label = "target SCP PATH "},
    scp_path,
})

9.1.7. Sharing scripts

So far, our lua code was in luarc. That's a good way to develop your script but not very practical for distribution. We need to make this into a proper lua module. To do that, we save the code in a separate file (scp-storage.lua in our case):

--[[
SCP STORAGE
a simple storage to export images via scp

AUTHOR
Jérémy Rosen (jeremy.rosen@enst-bretagne.fr)

INSTALLATION
* copy this file in $CONFIGDIR/lua/ where CONFIGDIR
is your darktable configuration directory
* add the following line in the file $CONFIGDIR/luarc
  require "scp-storage"

USAGE
* select "Export via SCP" in the storage selection menu
* set the target directory 
* export your images

LICENSE
GPLv2

]]
darktable = require "darktable"
darktable.configuration.check_version(...,{2,0,0})

local scp_path = darktable.new_widget("entry"){
  tooltip ="Complete path to copy to. Can include user and hostname",
  text = "",
  reset_callback = function(self) self.text = "" end
}


darktable.register_storage("scp_export","Export via scp",
  function( storage, image, format, filename,
     number, total, high_quality, extra_data)
    if coroutine.yield("RUN_COMMAND","scp "..filename.." "..
      scp_path.text
    ) then
      darktable.print_error("scp failed for "..tostring(image))
    end
    end,
    nil, --finalize
    nil, --supported
    nil, --initialize
    darktable.new_widget("box") {
    orientation ="horizontal",
    darktable.new_widget("label"){label = "target SCP PATH "},
    scp_path,
})

darktable will look for scripts (following the normal lua rules) in the standard directories plus $CONFIGDIR/lua/?.lua . So our script can be called by simply adding require "scp-storage" in the luarc file. A couple of extra notes...

  • The function darktable.configuration.check_version will check compatibility for you. The ... will turn into your script's name and {2,0,0} is the API version you have tested your script with. You can add multiple API versions if you update your script for multiple versions of darktable.

  • Make sure to declare all your functions as local to not pollute the general namespace.

  • Make sure you do not leave debug prints in your code. darktable.print_error in particular allows you to leave debug prints in your final code without disturbing the console.

  • You are free to choose any license for your script but scripts that are uploaded on darktable's website need to be GPLv2.

Once you have filled all the fields, checked your code, you can upload it to our script page here.

9.1.8. Calling Lua from DBus

It is possible to send a lua command to darktable via its DBus interface. The method org.darktable.service.Remote.Lua takes a single string parameter which is interpreted as a lua command. The command will be executed in the current lua context and should return either nil or a string. The result will be passed back as the result of the DBus method.

If the Lua call results in an error, the DBus method call will return an error org.darktable.Error.LuaError with the lua error message as the message attached to the DBus error.

9.1.9. Using darktable from a lua script

Warning: This feature is very experimental. It is known that several elements don't work in library mode yet. Careful testing is highly recommended.

The lua interface allows you to use darktable from any lua script. This will load darktable as a library and provide you with most of the lua API (darktable is configured headless, so the functions related to the user interface are not available).

As an example, the following program will print the list of all images in your library:

#!/usr/bin/env lua
package = require "package"
package.cpath=package.cpath..";./lib/darktable/lib?.so"

dt = require("darktable")(
"--library", "./library.db",
"--datadir", "./share/darktable",
"--moduledir", "./lib/darktable",
"--configdir", "./configdir",
"--cachedir","cachedir",
"--g-fatal-warnings")

require("darktable.debug")

for k,v in ipairs(dt.database) do
	print(tostring(v))
end

Note the third line that points to the location of the libdarktable.so file.

Also note that the call to require returns a function that can be called only once and allows you to set darktable's command line parameter. The :memory: parameter to --library is usefull here if you don't want to work on your personal library.