Closure Report

Project Summary

DiscoverEd is the University Library’s primary user-facing service.  It allows users to search the Library’s collections of print and electronic resources, to access the full text of e-books and journal articles, and to view their personal Library accounts in order to carry out a variety of self-service functions.  DiscoverEd has been a live service since July 2015, when it replaced both the previous Voyager Library Catalogue and the Searcher discovery tool.   In July 2017 a new interface for DiscoverEd was launched. 

DiscoverEd uses the Primo discovery product from Ex Libris, which works alongside the Alma Library Management System to provide real-time availability information for physical items, access to subscription-based full-text resources, and access to personal account details and self-service functions.  Primo comes with a suite of analytics tools, which allow the Library to access statistics recording levels of use, both of the system as a whole and of specific functions.  Analysis of these statistics indicates that use of DiscoverEd is substantial and is increasing year-on-year.  However, these statistics do not allow the Library to gauge levels of user satisfaction with the service.

Evidence gathered anecdotally suggests that some academic staff and students choose to use other online sources of information in preference to DiscoverEd, but the Library has very little information to help explain why this is the case.  The Library receives a low volume of feedback on DiscoverEd via an online form.  Much of this feedback is either very negative, focusing on specific issues and problems (often relating to other Library services), or very positive but containing very little useful detail.  Users often submit feedback anonymously, making it impossible to engage further with respondents about what they do and don’t like about DiscoverEd.  Furthermore, the Library currently has no method of engaging directly with non-users of the service.

The DiscoverEd Improvements project aimed to investigate what existing users think of the service in much more detail than we were hitherto able to ascertain, and to identify aspects of the service which are perceived to be frustrating, difficult to use, or are currently under-used.  The project also aimed to improve the Library’s understanding of why potential users choose not to use DiscoverEd.

The project had three main strands:

  1. An online survey of users and non-users of the service.
  2. Focus groups with targeted groupings of users and non-users.
  3. Usability testing of both the service as a whole and of specific aspects of the service.

 The objectives of the project were:

  1. To improve users’ experience of DiscoverEd.
  2. To better understand why users use DiscoverEd and why they might use other sources in preference to DiscoverEd.
  3. To encourage existing users to make more effective use of the service.
  4. To increase the use of DiscoverEd by academic staff and students within the University.

 

Analysis of Resource Usage:

Staff Usage Estimate: 300 days

Staff Usage Actual: 280 days

Staff Usage Variance: 0.93%

Other Resource Estimate: £0

Other Resource Actual: £560

Other Resource Variance: 560%

 

Outcome

The project ran successfully, despite the issues encountered around surveying students.  A detailed project report, covering the findings of the focus groups, usability testing sessions and staff survey is available by request from the Project Manager.

Actions taken to date:

A number of changes were made to the DiscoverEd interface in October 2018 as a direct result of the feedback gathered in the project.

  • Citation Trails:  The Citation Trails links have been repositioned in the Full Record display to make them more obvious users.  The Primo Analytics indicate that use of Citation Trails has doubled since this change was made. 
  • Persistent Facets:   The usability testers commented that the persistent facet function was potentially very useful, but almost impossible to find.  A change has been made to make the “make this facet persistent” padlock icon appear constantly when a facet has been selected, making it much more obvious to users.  Use of this function has increased by over 400% since the change was made.
  • Personalised Results:    The focus groups revealed that there was a desire from students to make search results more specific to their areas of study.  As a result, a feature of Primo which allows users to change the ranking of search results based on preferred subject disciplines has been switched on.  Use of this feature has been limited thus far, but it is hoped that, with additional promotion, this will increase and that the functionality will meet an expressed need from our users.

Proposals for further action:

  • Results ranking:  Much of the negative feedback received centred on the results ranking algorithm used by Primo.  Unfortunately, this is an area over which the Library has very little control.  However the Library can and will continue to lobby Ex Libris to improve this key functionality within Primo, and will reference the results of this project as evidence that the current results ranking algorithm is inadequate.  Full-text searching can often be a cause of irrelevant search results, and the Library will also recommend to Ex Libris that searching full-text should be an optional feature, as opposed to a mandatory setting.
  • Pre-Search Filters:    Feedback suggested that users were often frustrated by the limitations of the Basic Search, but that relatively few used the Advanced Search, which allows for greater specificity.  Pre-Search Filters will be added to the Basic Search, which will allow users to apply a filter to a search before searching, rather than conducting the search and then using facets to limit the results after the search has been made.
  • Newspaper Search:  Many users reported that the presence of Newspaper Articles in search results was a hindrance to searching.  Ex Libris are introducing new functionality to Primo which will remove Newspaper Articles from default search results and make them available through a separate Newspaper Search function.   The Library will activate this functionality as soon as it is available in early 2019.
  • Book reviews:  The volume of book reviews in search results was something that many users disliked.   Ex Libris have indicated that they will change the way that reviews are ranked in search results, giving them less prominence.  In the meantime the Library will explore ways of excluding book reviews from default search results, with an option to include them if desired.
  • Citations Trails:  The usability testers indicated that this was useful functionality, and this was echoed by some of the Focus Group participants, who mentioned that this was something they valued in other search tools.  Although the Library has tried to increase the visibility of the Citation Trails links, they are still not obvious in the interface.  In 2019 Ex Libris will be changing the way the Citation Trails are presented in the interface, making them visible on the results list, and the Library will be introducing this functionality at the earliest possible opportunity.
  • Promotion and User Education:   A recurrent theme in the Focus Groups, Usability Testing session and the Online Survey was that users were unaware of many useful features and functions of DiscoverEd.  It is clear that the Library’s promotion of DiscoverEd needs to be improved.  A series of short “how to” videos will be produced to highlight both core functions and more advanced functionality, and the existing FAQ pages will be reviewed and added to.  Where possible, extra help options will be added within the interface itself.  The Library will also increase the promotion of DiscoverEd via MyEd and social media channels, to raise users’ awareness of existing functions and of new functionality, as it becomes available.
  • Continued User Engagement:  The project has been very valuable in assessing the strengths and weaknesses of DiscoverEd, and this should be an ongoing process.  The Library will conduct further Usability Testing and Focus Group sessions, with both users and non-users.   These will largely be focussed on new and enhanced functionality, although there is also a requirement to continually evaluate the usability of core functionality with non-users.  The results of this testing will help to ensure that DiscoverEd continues to develop to meet our users’ needs and expectations.

Explanation for variance

The main issue encountered during the project related to the distribution of the online survey, which resulted in unforeseen costs. 

After some delay waiting for approval from the Student Surveys Unit/Student Ethics Committee, the project was eventually denied permission to disseminate the survey to students by any means.  This was unfortunate, as students make up the largest section of the user community for DiscoverEd.  As a result, the project had to deviate from the original methodology in the following ways:

  • The online survey was distributed to Staff only, via the Academic Support Librarians (ASLs) for the various Schools.  As an incentive to taking part, all respondents were entered into a prize draw, with four £25 book vouchers as prizes.
  • Since no students could respond to the survey, it was not possible to use it as a method of recruiting student participants for the focus groups and usability sessions.  Instead the project had to recruit students via MyCareerHub.  The participants were each paid £20.   £340 was paid to 17 focus group participants, and £120 was paid to 6 usability testing participants.

 

Key Learning Points

When DiscoverEd was launched in Summer 2015, there was an expectation amongst Library staff that it would provide a “one-stop shop” for study and research at the University.  The Project shows that this is clearly not the case.  Many users see DiscoverEd as just one of a range of search tools available to them.  Google Scholar, Google and subject-specific databases are often legitimately preferred to DiscoverEd, either because of the search features they provide, or because they allow access to content which is not available in DiscoverEd.  On the other hand, DiscoverEd is particularly valued for accessing known items, either in physical or electronic format, and the Library should concentrate its efforts on making these functions as intuitive and user-friendly as possible.  

The Usability Testing sessions also highlighted the variety of different ways our users interact with DiscoverEd, in terms of the search options they use and the search strategies they employ.  DiscoverEd users find their own way of getting to what they need, and tend not to deviate from their preferred approach.  Rather than restricting the search options available, for fear of confusing users, the Library should endeavour to provide as many different search options as possible, provided that they are clearly signposted and do not interfere with the usability of the interface.

The final lesson learned from the project relates to the difficulties around surveying the student body.  The Project Team were unaware of the procedures and timescale for getting approval to run a student survey, and this was to the detriment of the objectives of the project.   Any future Library surveys will need to be planned for, and have approval sought well in advance, if they are to be successful.

Outstanding Issues

 

Project Info

Project
DiscoverEd Improvements
Code
ISI031
Programme
IS Innovation - IS Corporate (ISI)
Management Office
ISG PMO
Project Manager
Gordon Andrew
Project Sponsor
Gavin McLachlan
Current Stage
Close
Status
Closed
Start Date
01-Dec-2017
Planning Date
n/a
Delivery Date
n/a
Close Date
29-Jan-2019
Overall Priority
Normal

Documentation

Close