Who else?

Continuing the series of postings about Libre Graphics Meeting. This one is all about Raw software.
At LGM3 We had almost everybody related to Raw software:

• Dave Coffin, creator of DCRaw
• Udi Fuchs, creator of UFraw
• 3×Anders, creators of Rawstudio
• Gilles Caulier, creator of digiKam

We were missing only Hubert Figuiere of libopenraw/exempi/niepce and Alex Tutubalin/Ilia Borgh of LibRaw. But the spirit of Tutubalin flew across the top of the waters of Oder for four days, so naturally Gilles couldn’t resist embracing the heavenly emanation and just few weeks later introduced LibRaw based libkdcraw Jokes aside, this library is now used across KDE4 graphics applications: digiKam, kphotalbum, DarkRoom and Krita. This is because Gilles heard about it at LGM in Wrocław.

You might as well ask, what is LibRaw and why it is better than the original DCRaw from which it is recreated. Alex’s “Goals and objectives” article pretty much covers covers this question, but you probably would like to hear from developers who use LibRaw. So I asked Gilles to write a short explanation.

And here it is:

— begin of quote —

1/ Introduction.
—————–

libkdcraw is a C++ wrapper to decode RAW images in different case and using a lots of demosaicing settings. It can also extract some metadata.

The first implementation of libkdcraw used an embedded dcraw binary program installed at the same time than library. This version of DCRaw was tested and valided to run properly with libkdcraw interface ad all options available from API for clients program. Why not to use the DCRaw binary version provided by a Linux distro? Because DCRaw options compatibility wasn’t preserved between main versions, and libkdcraw workflow could be broken with some combinations of settings.

DCRaw program is executed in a separated process and not threaded. A complex interface have been designed into libkdcraw to be able to observe progression of demosaicing and stop Raw decoding if necessary.

As more than one version of libkdcraw can be installed at the same time, a complex control of DCRaw install path must be done for each libkdcraw version to prevent conflict.

2/ Problems with dcraw. Why we need a low level library ?
———————————————————————

Using dcraw program in a wrapper library is a wrong solution to trying to have an homogeneous C++ interface:

• Performance: running DCRaw as a separated process is long to init by the system (more than to use children treads.)
• multithreading lock: a complex mechanism need to be done if more than one threads from client applications query libkdcraw at the same time. QProcess used to run DCRaw is not re-entrant, and application needed to wait than first query is done to ask another one.
• the DCRaw options interface is not homogenous and sometimes even obscure. For example, you cannot use all options available with 8 bits color depth demosaicing in 16 bits.
• There is no auto gamma adjustment in 16 bits color dept (a black image in 16 bits against a fine adjusted image in 8 bits). You cannot automate this workflow without big puzzle. Of course, it’s better to use customized settings to import Raw, and you can do it in digiKam, but some users want a fast way to open Raw images, as LightZone does.
• libkdcraw packaging is the hell, especially with DCRaw program install path, customized for each library release.
• Binary compatibility: if something changes in DCRaw option interface, it can be difficult to follow these changes without breaking binary compatibility.
• Metadata extraction is very limited and complex/slow to handle. DCRaw talks to libkdcraw true strings written/read from/to a command line pipe. It’s a hell to maintain and test.
• Due to using a pipe mechanism, following demosacing progress or stopping it is really complex to do.
• Due to using a pipe mechanism, memory allocation is not optimum.
• Last, but not least: DCRaw’s author has always refused to make a library and did not want to hear all recommendation from users, especially regarding improvements for worflow/interface (e.g. improving Raw decoding performance using parallelization of code).

3/ Using low level library : the ultimate solution
——————————————————-

Using a low level library as libraw instead dcraw, fix a lots of points:

• LibRaw is re-entrant. There is no lock mechanism to use anymore. We can use multithreading everywhere and query more than one demosaicing a the same time. Client application is faster.
• The interface have been reviewed with LibRaw developpers. Demosaicing settings are now homogeneous everywhere, especially in 8 and 16 bits mode. Users cannot see any difference between these modes.
• Demosaicing options compatibility will be preserved. Its more easy to maintain libkdcraw.
• All options are available directly by a C interface. Bye Bye pipe mechanism. It’s simple, fast and easy to maintain.
• Same with all metadata extracted. All are available thrue C interface.
• More metadata are extracted, as white balance, ICC color profile, photograph informations, color matrix (used with DNG converter), etc.
• Libraw is now parallelized using openmp library. Not all demosaicing sections are optimized in this way yet, but it’s in progress…
• libkdcraw packaging is really simple now. We don’t use an external libraw for the moment, until Libraw 1.0 will be published (current version is libraw 0.7.0). To update Due to use a pipe mechanism, libraw in libkdcraw, some C source code file need to be copied in an internal and private section. BC is preserved…
• New demosaicing algorithm have been introduced with libraw 0.7.0. It’s not only dcraw algorithms backported in a library, but a collection of code taken from another projects, as Rawtherapee for ex.
• Memory footprint from libkdcraw is very small as pipe mechanism disappears.
• Progress info is fully implemented in Libraw. We can provide a details of each operations processed (which can be long).
• Stopping demosaicing is easy to do, and very fast.
• LibRaw team is very open to discussions, so our work became very productive.

The migration from DCRaw to LibRaw was done just after LGM2008, where we spoke about LibRaw project. Rewriting libkdcraw code was done in a month, and optimization that to users feedback was done 2/3 month later. Now we have a powerful wrapper to play with Raw files. libdcraw is now included in KDE core and used by others KDE projects.

— end of quote —

DigiKam 0.10 was released earlier this week. Check it out and put it to a good use! They say that some of 0.11 features would be a new sophisticated batch manager and copy/pasting Raw development settings

By the way, some obviously mysterious winds have blown Hub’s libopenraw source code to Klaus Post who created RawSpeed library that is making its way to the next version of Rawstudio which will feature internal revamp, modularization and some great new stuff.

Who is going to win the Raw workflow battle — digiKam or Rawstudio? Or will Niepce Digital rise and take over? Support Libre Graphics Meeting and be the first to find out!