A work-centred approach to evaluation of a digital library: annotation system for knowledge sharing in collaborative work

This paper presents an expert evaluation of a collaborative annotation system named DiLAS (Digital Library Annotation Service). It can be globally accessed by individuals as well as different user communities, and knowledge is created and shared within English-speaking communities. It contains a collection of textual documents on information science and software science, and it gives access to all kinds of related material such as authors’ home pages, photos, articles, and so on. An analytical evaluation was conducted as a participatory group evaluation, which involved presentation beyond the objectives and rationale papers and development of the prototype. The empirical evaluation of DiLAS consisted of two experiments. The first evaluation was a bottom up evaluation that began at the elementary level with an evaluation of the usability of the interface. A Cognitive Walkthrough approach was chosen using a qualitative approach. The next evaluation moved towards the broader work context with a user- and work-centred evaluation involving an entire collaborative task situation, which required knowledge sharing on a common real life work task. This evaluation resulted in a large number of user requirements for the next prototype.