Based on a talk to the Singapore chapter of the International Society for Knowledge Organization (ISKO). A recording of the talk is available at: http://www.iskosg.org/magic_with_metadata_haynes.html

In my last blog I wrote about the purposes of metadata. Metadata is pervasive and plays a vital role in the effective management of information.  Here are some of the challenges associated with metadata management that I have come across in my 30-year career as information management consultant.

Challenges of metadata

Let’s start with the challenges of using metadata

The human element

Many great metadata projects are scuppered by people! It’s a bit like the joke that goes “This would be an excellent service, but for the customers”. It is easy for service providers to fall into the trap of seeing people as the problem. People should be the focus for the service. They are the reason for the service. For instance, a manager may be convinced of the benefits of metadata. But if they then impose onerous indexing tasks on the staff to generate metadata, it becomes a barrier. Index terms need to be applied consistently and accurately. This is difficult to police. Of course, there’s always automatic indexing – more on that later.

Maintenance of metadata

Change is constant in any business area or area of government policy. This means that indexing and classification systems used to describe these activities have to be maintained.  Many organisations have successfully built their own taxonomies. But this only works if there are funds for on-going maintenance. Otherwise the taxonomy quickly becomes out of date.

User training

It’s no good having a sophisticated search capability based on metadata, if no-one uses it.  Although people are becoming more adept at querying systems, there is still a tendency to use simple search strategies. With a little training more sophisticated searching will improve the precision of searches.

Automation

If you are an information manager you may well have heard the refrain: “Oh, we’ll automate”.  Undoubtedly search engine technology has made many advances since the early days of the internet. The advent of the semantic web allows us to label individual elements so that the context is clear.  However, someone has to design these systems, maintain them and mark up resources (such as web pages and documents) so that they can be handled by apps. Humans are incredibly good at recognising context and interpreting meaning.

Ways forward

How do we deal with these challenges? Fortunately there are some established approaches which can help.

Metadata standards

Metadata standards such as Dublin Core and mark up systems such as XML allow elements within digital documents to be labelled. This opens the door for semantic web applications where the marked up data is given a context. The semantic allows more flexible descriptions of information. This in turn can improve retrieval and use of information resources.

Consistency

More consistent descriptions improve the user experience, save time and lead to better quality interaction. If we use pre-defined categories or descriptions for resources, the resulting indexing is more consistent. Controlled vocabularies, taxonomies or encoding systems are ways of achieving this.

Marking up

Text can be marked up for automatic processing, which allows development of new services that draw on multiple resources. For example, the Google Knowledge Graph depends on recognising marked up fields in online resources. This enables Google searches to deliver answers to queries rather than lists of links to websites. This is a good example of the semantic web in operation.

Automation

Automation still depends on human intervention. For instance, machine learning systems need to be designed. They then need to be trained with a standard corpus of data before they can be applied to new data sets. An example of automation would be for a system to suggest indexing terms, which are then confirmed by a human operator.

In my next blog I will write about the need for information and technology professionals to engage with users and vendors to make better use of metadata.

Teaching what we research

We explore many of these themes in a new module on Information and Knowledge Organization here at Edinburgh Napier University – as part of the MSc programme in Business Information Technology which I lead.

References and further information

Gartner, R. (2016) Metadata: shaping knowledge from antiquity to the semantic web. Cham, Switzerland: Springer International Publishing. doi: 10.1007/9783319408934.

Haynes, D. (2018) Metadata for Information Management and Retrieval: understanding metadata and its use. 2nd edn. London: Facet Publishing.

International Society for Knowledge Organization. ISKO website. www.isko.org (accessed 2021-10-22)