Digital Asset Management
with digiKam

...in the end, photographs need a lot of care.

I hope it's you who said this.

Welcome!

to this article by Gerhard Kulzer.
This article treats many aspects of how to preserve your digital assets, in particular your photographs and scans. Much of what is being explained here is equally and readily applicable to manage other digital assets such as music files, movies, documents, art work, etc.

Digital Asset Management DAM

Can you find your digital photographs when you need them? Or do you spend more time sifting through your hard drive and file cabinets than you would like? Do you have a systematic approach for assigning and tracking content data on your photos? If you make a living as a photographer, do your images bear your copyright and contact information, or do they circulate in the marketplace unprotected? Do you want your future grandchildren to admire your photographs you have taken yesterday? How do you ensure backup and the correctness of your data? How to prepare to change your computer, your hard disk, the software, the operating system and still manage to find your pictures or movies?

What is digital asset management - apart from a buzz word? Digital Asset Management (DAM) refers to every part of the process that follows the taking of the picture, all the way through the final output and permanent storage. Anyone who shoots, scans or stores digital photographs is practicing some form of DAM, but most of us are not doing so in a systematic or efficient way.

I present a tool, a plan and practical advice on how to file, find, protect and re-use photographs, focusing on best practices for digital photographers using digiKam. I cover downloading, renaming, culling, converting, grouping, backing-up, rating, tagging, archiving, optimizing, maintaining and exporting image files.

A generic definition:
"Digital Asset Management ingests, indexes, categorizes, secures, searches, transforms, assembles and exports content that has monetary or cultural value."
And since we're at it, another important one:
"Metadata is defined as data about data. Metadata is definitional data that provides information about or documentation of other data managed within an application or environment."
In our context here it stands for all information ABOUT a photograph.

DigiKam with its libraries and plugins is a unique and comprehensive tool to cover most of DAM tasks, and it does it fast and transparently. Based on open standards on all fronts it will not confine you to a platform or application, rather it puts you into a fast track to manage and find your photographs and to move on if you so please to any other platform, application, system without losing any of your work be it as an occasional user, enthusiast or professional.

The one thing that differentiates the archiving capabilities of film vs digital is that with digital you can make as many new originals as you want. With film you only have one original. All copies will have a slightly lower quality, and both originals and copies are more or less slowly aging and disappearing. The only way to keep it "forever fresh" is to make a digital copy of it. And that is also the only way to protect it from all hazards.

Even if digital media today may last shorter than film it is just up to you to make new copies every year, 5, 10 years or whenever necessary, and to always keep at least 2-3 copies of the files, preferably in different physical locations. You never had that opportunity with film. It could always be damaged in a fire, floods or similar - or even be stolen. The good and bad news then is this: if you lose digital images/data it is only your own laxity.

Build a system to organize and find your images

Themes: hierarchy, tags, rating, captions, geolocation, date, albums, filenames, versioning, exporting

I dare-say if you have more than 1000 photographs on your computer in no-DAM fashion it takes you too long to find any particular image. And if you don't know how many images are in your files you're surely not using digiKam. The dual approach to store metadata in a database AND in the image files guarantees ultra fast searching and secure archiving freely accessible to other applications, platforms and formats.

But as much as there is no such thing as a free lunch, there is no free cataloging or DAM - those who spend the initial time of building a systematic method of their own will be better off as time passes and the number of photographs multiplies. The ROI (return on investment) of DAM has been estimated in different studies to be better than 10. Keep in mind to be concise, plan for the future (30-50y), do it once. The upcoming semantic web will totally integrate into and add value to a DAM environment.

A case for doing DAM with digiKam.

DigiKam provides a number of methods to classify photographs:
filenames, albums, collections, date + time, tags, rating, GPS position and captions. As if this was not enough, in the KDE4 version of digiKam you can search many standard metadata items like camera model, lens, coordinates, image size and many more. Metadata categories as listed here are in fact different 'views' of your photo library. Combining these views is THE very powerful method to narrow down the search for a file and to find it quickly. Imagine having 800 photos of your loved one. Searching for 'Mary', having more than '***' rating, shot in 'France' will surely leave you with very few candidates. In terms of selection criteria for a DAM system, digiKam fares very well in terms of completeness, versatility, speed, scalability, accuracy and openness.

The key thing to remember is that you don't know how you or somebody else will try to find an image 2 years ahead of our time. You will remember past events in a different context, it's a fact of life. So if can narrow down your search by remembering place or time or camera or theme or rating or owner you stand an infinitely better chance to find it quickly than by just one of those criteria or none. At the beginning, at the time of taking a photograph, all metadata is in your head (except for the EXIF data). If you do not transcribe some of it into your DAM system, it will be lost eventually as much as every event fades into oblivion over time.

One distinction has to be interjected here between private and public metadata. One could say that all file-embedded attributes are potentially public since the images may be exported, sold, copied to other places and people. On the other hand all non-embedded metadata in the database can be considered private as they stay in the database and nowhere else. By adjusting digiKam's settings accordingly you can control what kind of data remains private and what will be embedded.

Build the archive: Folder organization, physical layout as information

The first thing to do and to know before you put anything onto your system is a to build an information structure (as opposed to data structure). Your image files have to be somehow organized within the computer, you have to decide if others should have access to your photographs (sharing), if you put them on a dedicated drive, on a network drive etc. Keep in mind that you have to migrate one day onto some bigger volume.

The organization should be simple, unified and scalable1, and it should be independent of the storage medium on which you host them. Do not make the folders too small, several thousand images in one folder is not too much to ask for, but keep them small enough so that they can fit into a backup medium like a DVD 4.7 of 9.4 GB for double sided ones. Remember that the archive will grow all the time! The concrete type of structure depends on your use case of course: Lets take a simple yet frequent example: you are a casual photographer taking pictures of your private life, your family, holidays and so on. It could be efficient to create a structure based on years plus some holiday and export containers. It could look like this:

2006
2007
2008
Holidays
- A
- B
- C
Export
Fun stuff

Maybe you'll be happy with this structure. Holiday pictures can be quickly found by its location (unless you go to the same place every year), the rest will be organized by date. If you shoot enough pictures you want to create sub folders below the years as months e.g. 2008-01, 2008-02 etc. 'Export' would be a container for images to print or to put onto a website.

The more professional photographer will have very different needs as there will be versions of photographs, archives, workflows, a constant influx of images of diverging themes, and a large quantity of everything. Within 10 years you'll have 95% archives and 5% work space files and you don't want to organize your structure around content!

The consideration are these:

  • What kind of files go together? Segregation of file type makes batch processing easier. Keep new and old files separate.

  • How can you make that structure scalable?

  • Segregation of original and working files makes it easier to allocate the backup strategy and migration. You will always know if you look for an original or a derivative.

Automatic metadata generation

How to go about all this metadata business? Firstly, there are already a lot of automatically generated metadata: EXIF data and Makernotes. If you have configured digiKam with your identity section all imported images will be imprinted with this data set which includes copyrights, all automatic. If you have a GPS track recorded in parallel to your taking the photographs, you can geolocate those images in a single action using the Geolocation plugin. Even if you brought back 1000 images from a shooting session, so far you'll not have spent more than 10 minutes to do all that. And by now you have all camera settings of every shot, lens data like zoom, focus, aperture etc., date and time, shooting location, copyrights, authorship, program used, and more. Not bad, isn't it? But we could have done more during the importing, we could have changed the file names to include the date, or place or theme, we could have changed the format to a lossless 16 bit per channel format, we could have automatically separated JPEG and RAW files into their folders. I actually recommend to auto-rename to match an event, a place or a theme. digiKam provides all date/calendar related grouping so that there's hardly a need for coding the date into the file name. Unless you'd like to do just that to browse your albums with another application that is not calendar savvy. You will buy a new camera one day, sooner than you believe. The numbering scheme of that new camera will start over at typically img_0001.JPG again, creating identical file names to the one you have already if you do not rename them.

I also recommend to switch-on the 'save metadata' options in the digiKam settings page for metadata. This will ensure that EXIF and IPTC data is written into the file. If you forgot to do that you can always catch up by copying the metadata in the database to the files in one go (from the album menu).

Now we have a lot of stuff already in our database, but what if I need to change some of it? digiKam provides a metadata editor for a selected number of attributes, the most important ones of course.

The real works begins here as we will apply tags, captions and a rating to every photograph. Of course, all images requiring the same attribute can be treated as a selection in one action. Lets start with rating or ranking. It's best to start with ranking because for further work you can concentrate on the good shots.

Rating/Ranking

A ranking systematic is implemented in digiKam by the 5 star rating tool. In fact there are 6 levels, zero through five stars (*) can be attributed (when saving them into IPTC metadata a translation of levels ensures compatibility with other programs). Rating is rapidly applied with digiKam using keyboard shortcuts or the mouse on single photographs or whole selections. The rating can then be entered as a search criterion or directly from the status bar quick filters. However, before you start attributing stars everywhere take a moment to establish personal criteria for ranking. Best practice is to write down your personal match of stars to some qualitative expression, that will define what you actually mean when giving 5 stars. Generally there should be much less images rated with increasing star assignment. A ratio of 3-10 between each level has proven useful. That will get you quite far in distinguishing your rating pyramid. Say, you choose a ratio of 7 between levels. For every 5 star image you'll then have 7 4 stars, 49 3 stars and so on, resulting in almost 20000 pictures. Amazing? Yes, and 16807 of them you didn't have to rate at all! You even can define a different rating scheme depending on the kind of use, 2 stars for commercial use, may mean something else than 2 stars holiday photos. It is also a good practice to define a neutral rating, everything below is actually a negative rating. This will help you culling and thinning your collection very efficiently. Or you could define purposes to ratings, say 0 stars for 'can throw away', 1 star for images in quarantine (decide later), 2 stars for gallery export, 3 stars for printing, 4 stars for selling, 5 stars for 'have to work on', as you please. It must suit YOUR needs. The following table illustrates a possible evolution for a professional photographer using a ranking ratio of roughly 7 over the next 12 years. It is evident that the good shots can be easily found, even within millions of photos.

Rating

Year 2008

Year 2020

Total images 132241 1503320
Neutral (0 stars) 55000 640000
* 72020 810000
** 4880 49000
*** 301 3200
**** 38 400
***** 2 720
ranking

Lets continue with tags (or keywords as called by other applications, or categories, they are all synonymous).

Tagging, Keyword assignment

Tags are a hierarchical labeling system that you create as you add to it. The important thing to do is to create a system that suits your needs and habits. Are you a (semi)professional who wants to sell photographs to agencies, do you want to publish on a web gallery, or are you just the occasional amateur managing the visual family memory? For all these different use cases you want to design a tag structure that is adapted to it. If you configure it so, digiKam will write the whole hierarchy into IPTC fields so that they can be used by your photographic agency using a different application of to automatically create title and caption for web exports. In any case it will serve you well to quickly find a specific picture again.

The hierarchy will provide you with automatic groupings. For example, if you start a typical private use hierarchy with 'Activities', 'People', 'Places', 'Themes' and 'Projects' on the top level, everything you tag with a sub-tag of these will be grouped together into a virtual album. digiKam has a dedicated view in the left sidebar for these virtual albums. But it comes even better! As you continue adding sub-tags into the hierarchies, not only will you be able to search and quick-filter for them, the right sidebar tag filter allows you to select combinations of tag groups. Lets say in the left sidebar tag panel you select the virtual album 'People' and you have 12 different tags for people in there, then you can combine it with the right sidebar and just choose 'Peter', 'Paul' and 'Mary' out of the 12.

In the long run you will not remember the details of your pictures and their subject (essentially the metadata in you brain will break down). It is therefore paramount that you choose general and generic categories. You will aways remember that a particular shot was set at a river bank in a country or continent (-> river, continent), but you'll have forgotten which river it was. Instead of only tagging it with 'Okavango' you tag it with river/Africa or river/South Africa. The details you can either put into a tag as well or into the captions. A trick may help you: How would you search for that river with an internet search engine? That's the way to go!

Another categorization might be task-oriented as in 'print jobs', 'web export', 'personal', 'galleryXYZ', 'clients', 'slideshow' etc. Create groups as you need them but not more, you should be able to remember by heart the top level tags at least, otherwise the differentiation will become useless. Don't forget that you have all the other attributes to narrow down the search. The right sidebar tag filter combines with any view of left sidebar (albums, calendar, timeline, tag and search).

metadata

When you import cataloged images from other sources having embedded tags already, digiKam will automatically create the trees for you, respectively insert it into the right place. Rearranging the hierarchy within the tree is no problem, you can do that easily by dragging and dropping a sub-tree to another place in the hierarchy. The changed tags will be updated as digiKam ripples down the branches.

The graphics here shows how different metadata overlap. This is a very coarse representation, as each block of metadata will in itself be subdivided into many sections. File names and calendar data are properties of all images.

Enough of tags - lets move on to captions or comments, the third major tool for metadata cataloging.

Captions/Comments

This is already the 4th kind of metadata presented here. What is the distinction of captions compared to tags (comments can be used synonymously, but the IPTC vocabulary stipulates the term 'caption'), keywords? Where tags owe to a hierarchical and generalized description, captions are the opposite: prose description, details, anecdotal stuff. Tags foremostly serve the finding, retrieval and grouping of assets, whereas captions shall entertain, inform, touch the beholder. Naturally they can also be used to filter the catalog, but this is just a byproduct. Captions are to remember the story, the event, the emotions, it's what makes photographs much more interesting to look at, captions put photographs into a context and meaning. If the pictures are an aesthetic statement, caption should be the emotional and informational complement.

You rarely want nobody to see your photographs. You rather want to share them with friends, your family, other photographers, agencies, put them onto the internet. And don't tell me you're not interested as to how your photos are being received!

So you might have to most beautiful portrait, sunset or landscape and nobody seems to care. Why is that? Look at some good photographs yourself without reading the title, comment or background information. How many of you are interested in depth of field, exposure time, white balance etc.? Some, of course. But anybody will be interested in the story the pictures tell, you want to remember a photograph, meaningless images bombard us too much anyways. You have to give the viewer something that explains it all.

Lets look at this panorama. From far it is not even a nice beach panorama. If you go closer you start to see some details, people, the space.

beach

And now I tell you that this is the Allies landing site “Omaha Beach” in the French Normandie 60 years after the disembarkation. Wow! One starts to dream, have associations, memories, the historical time span is present, you may hear the silence. The caption has totally reframed to perception of this panorama.

For others to appreciate your photographs, the title is probably more important than the image itself for the interest it creates. When you show pictures, tell a story. Remember that the key is to convey the meaning to viewers, to help them understand what you understand about the subject and what moved you.

  • Let people know what you understand about the subject, why you love it

  • create a red line between the photographs

  • oppose or relate them to different epochs

  • Take notes shortly after shooting to remember

  • contemplate, research, watch, and talk - but mostly listen.

  • it's okay if the image is less than perfect because it has the strength to stand on its own merit described in the caption.

With digiKam you can enter unlimited amounts of text using internationalized alphabet (UTF-8) as caption. You can enter it for a selection of photos at the same time. KDE even provides a spell checker. When you export images to web galleries, the captions will be exported at choice into either/or/and caption/title of the web gallery system, no need to re-write the story for publishing.

Geolocation (geo-tagging)

Do you still remember the times before GPS? When you would find your way to another city without navigation system? Wasn't the earth a dull blue ball before GoogleEarth? Well then, with images, the train of spatial representation is running at cruising speed alright.

A few cameras have a GPS receiver built-in, the images come tagged with 3-dimensional coordinates. And with almost any GPS device you're able to extract a trace (of course the receiver needs to be switched-on and carried with you whilst taking the photographs, and for good matching the camera time must be accurately set) and save it onto a computer. You have to store it in gpx format, that's easily done with gpsbabel, gpsman and other tools. You then can automatically match a whole bunch of photos with that track using digiKam. The coordinates are written into the JFIF part of JPG files (settings choice) and into the database. The KDE4 version will enable searches based on locations and coordinates, you can create virtual albums of geographical areas! In the right sidebar under the metadata tab you'll find your image located on a local zoom of the world map. A further click brings on anyone of several mapping services on the web, zooming in on details. Even if you don't have a GPS trace you can geo-tag multiple images with a geo-editor. Just navigate on the map to the spot of shooting and click to fix it as a geo-tag.

e.g. conversion of a Garmin track with file name 'xyz':
$ gpsbabel -w -i mapsource -f xyz.mps -o gpx -F xyz.gpx
The possibilities of exploiting this geo-tagging are already innumerable and will become pervasive in the future. I'm sure one day not too far away we can revisit in a virtual reality our travels through geo-tagged pictures. The digiKam features include exporting to kml files that can be opened by GoogleEarth (which in turn will show the photos on their shooting site), exporting to gallery2, picasaweb, flickr etc. with GoogleMaps viewer and more.

Protect your authorship and copyleft/right

Themes: watermarking, IPTC and XMP authorship data, export size

This will be the last chapter and step to mark your digital library with authorship, ownership and copyright or -left information. More than in 'the good(?) old days' of paper copies, the ubiquitous internet makes it just too easy to 'steal' a picture from a web site. At the very least, for all images that will be exported and/or published in any form, the authorship and copyright information should be part of their metadata. Nothing more simple to do with digiKam: you can setup the default identity, and any images ingested be digiKam will be automatically informed. I put copyleft in the title for a reason (citation from Wikipedia):

"Copyleft is a play on the word copyright and is the practice of using copyright law to remove restrictions on distributing copies and modified versions of a work for others and requiring that the same freedoms be preserved in modified versions.
Copyleft is a form of licensing and may be used to modify copyrights for works such as ... music, and art. In general, copyright law allows an author to prohibit others from reproducing, adapting, or distributing copies of the author's work. In contrast, an author may, through a copyleft licensing scheme, give every person who receives a copy of a work permission to reproduce, adapt or distribute the work as long as any resulting copies or adaptations are also bound by the same copyleft licensing scheme. A widely used and originating copyleft license is the GNU General Public License. Similar licenses are available through Creative Commons — called Share-alike."

And here follows a description of what should be supplied to digiKam's setup page as information:

Author (synonymous with Creator and By-line): This field should contain your name, or the name of the person who created the photograph. If it is not appropriate to add the name of the photographer (for example, if the identify of the photographer needs to be protected) the name of a company or organization can also be used. Once saved, this field should not be changed by anyone. This field does not support the use of commas or semi-colons as separator.

Author title (synonymous with By-line title): Linked to Author. This field should contain the job title of the photographer. Examples might include titles such as: Staff Photographer, Freelance Photographer, or Independent Commercial Photographer. Since this is a qualifier for the Author field, the Author field must also be filled out.

Credit (synonymous with Provider): Use the Provider field to identify who is providing the photograph. This does not necessarily have to be the author. If a photographer is working for a news agency such as Reuters or the Associated Press, these organizations could be listed here as they are 'providing' the image for use by others. If the image is a stock photograph, then the group (agency) involved in supplying the image should be listed here.

Source: The Source field should be used to identify the original owner or copyright holder of the photograph. The value of this field should never be changed after the information is entered following the image's creation. You should consider this to be a write-once field. The source could be an individual, an agency, or a member of an agency. To aid in later searches, I suggest to separate any slashes '/' with a blank space. Use the form 'photographer / agency' rather than 'photographer/agency.' Source may also be different from Creator and from the names listed in the Copyright Notice.

Copyright Notice: The Copyright Notice should contain any necessary copyright notice for claiming the intellectual property, and should identify the current owner(s) of the copyright for the photograph. Usually, this would be the photographer, but if the image was done by an employee or as work-for-hire, then the agency or company should be listed. Use the form appropriate to your country. For the United States you would typically follow the form of © {date of first publication} name of copyright owner, as in 'copr 2005 John Doe.' The word 'copyright' or the abbreviation 'copr' shall be used in place of the © symbol as ASCII characters only are allowed. In some foreign countries only the copyright symbol is recognized and the abbreviation does not work. Using something like (c) where the parentheses form a partial circle is not sufficient. For additional protection worldwide, use of the phrase, 'all rights reserved' following the notice above is encouraged. In Europe you would use: Copyright {Year} {copyright owner}, all rights reserved. In Japan, for maximum protection, the following three items should appear in the copyright field of the IPTC Core: (a) the word, Copyright; (b) year of the first publication; and (c) name of the author. You may also wish to include the phrase 'all rights reserved.'

Whereas it is paramount to fill-in the author and copyright sections, they represent no protection against fraud. Anyone with a bit more than basic computer knowledge is able to delete or modify image embedded metadata. The solution to this problem is called 'digital watermarking'. To private persons this might be of little interest for the majority of photographs, but for professionals ans semi-professionals this protection is really important.

Digital Watermarking (DW)

Digital Watermarking refers to an invisible digital watermark that is being impressed on photographs as an element of digital rights management (DRM). The watermark contains the same information of authorship and copyright as described above, but the metadata is encrypted and saved in the actual image data (as opposed to the metadata section which is a separate section within the image file). This invisible imprint has holographic properties so that modifications done to an image (size, color, crop, up to a certain limit) will not destroy the copyright information. Only when an image is resized to a very small fraction like a thumbnail will the embedded information be lost, but then the image is of no value anymore to the copyright infringing party.

The digital watermark will be unique per image. digiKam will provide a plugin for DW in the near future that features batch processing.

1 Refer to Scalability for explanation

-> continue with part 2