supermoon-ss

Darktable and Solaris: It Just Works(tm) .... and there are some nifty benefits too

I'm the self-appointed maintainer of Darktable on Solaris, which is a fairly easy gig to keep on top of.

Here's why that is so: Darktable's codebase is very portable. It's not riddled with operating system-specific assumptions; it uses standard C (with some C++), and apart from the OpenCL support every prerequisite library is buildable on Solaris with gcc or g++. I'd prefer to use Oracle Solaris Studio because that's my work compiler, but there's no great incentive for me to beat up on all the prerequisites to make them behave.

To date I've contributed more by way of packaging metadata and bug updates than code changes and I like that. If you're curious, I wrote a blog post about how to generate a new Darktable package in the Solaris 11 IPS format.

My plans for future Solaris involvement are chiefly around how we scale to larger memory configurations (specifically, the mipmap cache) and our database usage - with DTrace I can see very clearly how long queries and updates are taking in real time. While the command line debug flags are handy, they do slow down interactive operation with printing output to a file.

For instance, on my system (dual-core/dual-processor 3.0GHz Opteron cpus with 32Gb ram, 512Mb nVidia framebuffer with a development build of a successor to Solaris 11), the gui response when I'm typing in a characters from a tag is (subjectively) awful. To get an idea of how awful, I added a new tag to a photo I took last night of the SuperMoon:

$ dtrace -n'pid517::dt_tag_new:entry{self->traced = 1; self->timestamp = vtimestamp; self->tag=copyinstr(arg0); self->tagid= (int)arg1;}' \
-n'pid517::dt_tag_new:return/self->traced/ {self->traced = 0; printf("dt_tag_new(%s, %d) took  %d nsec", self->tag, self->tagid, vtimestamp - self->timestamp);}' 
dtrace: description 'pid517::dt_tag_new:entry' matched 1 probe
dtrace: description 'pid517::dt_tag_new:return' matched 1 probe

CPU     ID                    FUNCTION:NAME
  3 167336                dt_tag_new:return dt_tag_new(clouds, 134506508) took  9226022 nsec

That's a fairly simple one-liner for a very simple use-case. It's a bit of a worry that it took 9 milliseconds (thankyou to the commenters below for picking up my error with the order of magnitude) for that function to return. Now that I've got some hard data I can get started with drilling down into what the possible causes are, using the analytical troubleshooting system (an implementation of Kepner-Tregoe's Problem Solving and Decision Management methodology) which I was taught while working at (the late) Sun Microsystems.

Looking through the code for dt_tag_new, I think we've got the possibility of some small optimisations. I've been trying out the idea of keeping the tags and tagxtag tables in memory and updating the on-disk versions using a SQLite3 trigger. However.... more investigation is needed. I'll be using Dtrace to help me see what works best.

Solving little things like that is where I think I can bring most to this project, and where having Solaris as a tier-1 development platform becomes very useful.

Self-promotion: when I blog about Darktable, I post at http://www.jmcpdotcom.com/blog/category/darktable/.

4 thoughts on “Darktable and Solaris: It Just Works(tm) .... and there are some nifty benefits too

  1. Maybe I misunderstand the dtrace output, but 9226022 nsec is rather 9 milliseconds and not 9 seconds?

  2. Hi there !

    Insightful post but there is something I don’t understand: you say the tagging function lasts 9 seconds but your dtrace results show only 9,226,022 nsecs. Aren’t nsecs nanoseconds ? If so, the function only lasts 9 milliseconds, which sounds quite good to me.

    Cheers

    Fred

  3. Ah, you’re correct. I hate being off by an order of magnitude :(

    That said, there is significant latency in the gui (to the extent that the window greys out) when I do tagging operations; using DTrace I’m narrowing down where and why that problem occurs.

    I don’t think that 9milliseconds is speedy, though.

    Thankyou for your comments, I’ll correct the post shortly….

  4. An order of magnitude is usually a factor of 10. s -> ms is therefore 3 orders of magnitude ;-)

    9ms is ok, more so if it involves a round-trip do disk (a seek).