Discovery: Plenary session 4

The first plenary session of the last day of UKSG offered 4 perspectives on discovery. Below is a summary of each of the presentations followed by a round up of the discussion.

Discovery or Displacement?: a large-scale longitudinal study of the effect of discovery systems on online journal usage

Michael Levine-Clark, University of Denver and John McDonald, University of Southern California.

The survey focused on whether implementing a discovery tool within a library increases usage of content. It looked at measuring if there is an effect, not why. The survey focused on the use of Primo, Summon, EBSCO Discovery Service (EDS) and WorldCat Local.

Methodology:

Contacted libraries they knew had implemented a discovery system; asked implementation date, search box location and marketing effort used to promote the service.

Selected 33 libraries for 1st phase; 6 from each of 4 major discovery services and 9 control libraries with no discovery service. Of these 28 were in the US, 2 in CA, 1 in UK,1 in AUS and in 1 NZ, most of whom had implemented discovery services in 2011. 

The survey included data from 6 publishers and 9,206 journals, with 36% of observations from the largest publisher. Usage was compared using the COUNTER JR1 (full text downloads per title) reports the year before and the year after implementation of the discovery service. 

All publishers available in all of the discovery services and to all of the libraries. 

Each library only had one discovery service.

Findings:

• Within 6 of the libraries usage actually went down, this included one of the control libraries
•  97% confident that discovery tools have an impact on usage
• Journal usage at Summon and Primo institutions increased more than with other services
• Control group shows usage naturally increases over time

Next steps:

• Design and test for additional items including;
• Aggregator full text availability
• Size of publisher
• Journal subject
• Overall usage trends
• Configuration options in discovery services; Direct linking, altering algorithm etc.
• Expand pool of libraries
• Perhaps explore why

See http://www.slideshare.net/MichaelLevineClark/niso-dda-uksg-2014-33587193 for the full slides from the presentation.

Impact of Library discovery technology

Varlérie Spezi, LISU Loughbrough University

Presented the key findings of the UKSG/Jisc funded research project, 'Impact of library discovery technologies'. The study was UK focused. Work started in July 2013. Report released in Dec 2013.

Objectives:
Evaluation of the impact of a RDS (Research Discovery Systems) on usage

Methodology:

Phase 1: Survey of UK HE libraries

62 respondents

Phase 2: Case studies of libraries and publishers

Collected JR1, BR2 and DB1 COUNTER reports

Phase 3: Interviews with stakeholders

Findings:

• 77% of UK HE libraries who responded are already using an RDS
• Further 11% in process of implementing one
• Summon, Primo and EDS accounted for 76% of systems in use
• Libraries reported high levels of satisfactory;  liked one stop shop experience  (Google like experience), felt they were getting better use of subscriptions – no silos
• Undergraduates are seen as primary users of services

Overall a RDS may positively influence content usage especially for ebooks, although this usage varies across institutions. For ebooks the impact was positive, for journals it was a mixed picture and for databases the results were inconclusive.Many other factor can influence usage e.g. changes in titles and more data is needed to pick up trends and isolate these other  factors.

Problems/challenges:

•  Libraries unable to see how libraries subscriptions match coverage by RDS systems
• Interoperability between systems difficult - There are problems if don't use the RDS system from the vendor you already work with for other systems within the library
• Incomplete metadata
• Publisher's concern about dilution of publisher brand when content is made available via a RDS system
• Lack of feedback/communication from the RDS vendor
• Lack of clarity about how the results are displayed e.g. the relevance ranking

Recommendations:

Collaboration key to success

Libraries

• Work with bodies such as UKSG, Jisc to drive development in RDS community
• Consider issues of interoperability between products from different vendors
• Engage in cross-sectional talks to understand how minor changes in settings can affect usage

RDS supplier

• Work towards open communication
• User testing for publisher and content providers
• Provide clearer information on relevance ranking

Publishers

• Engage with library and RDS to optimise discovery
• Voice need for more communication and feedback

Other stakeholders

•  Inclusion of RDS usage in JUSP and/or KB+
• Development of COUNTER code for RDS usage

The full study is available from http://www.uksg.org/researchstudy

Libraries in the cloud, on the ground and in between

Birte Christensen-Dalsgaard, Royal Library Denmark

At the start of her presentation Birte emaphasised that her presentation had been strongly influenced by other talks at the conference.

Libraries live within RDS system, Students/researcher live in different worlds
Things have changed, but still closer to the cards (library cards) than the bots

Google results are much richer and provide a better user experience than library catalogue

Google using semantics (schema.org), but libraries can do the same AND can evaluate quality control through author profile (ORCID's) and FoF (Friend of Friend) networks.
Authoritative information needs to be created locally or obtained through mining

Search needs to be more than just a word-match. More services are taking user behaviour into account for example bX from ExLibris, but due to privacy rules libraries can’t use a lot of data e.g. what user has borrowed.

Libraries need to add value, this can include supporting users to judge value, big data, semantics, collaborations, supporting different delivery channels and local tailoring.

Thinking the unthinkable - doing away with the library catalogue

Simone Kortekaas, Utrecht University

In 2012 Utrecht University built their own discovery tools, called Omega. Since then have found that more users reaching content from Scopus, Google Scholar etc. Library catologue usage dropping, but usage of content increasing suggesting that users are finding other routes into content. Patrons switching faster than libraries; a 2010 survey revealed that 83% students start research from a search engine. In 2012 Utrecht University concluded they could shut down their own discovery service and not replace it; needed to rethink role of library.

A survey of users within the University revealed that most started research in Google Scholar. Before switched off of the library catalogue they had to prepare the students. Needed to offer them an alternative, so in 2013 started communicating better ways of finding content, much of which has been done via social media. As part of the changes they removed the search box, changed instruction/support, added libguides on effective use of search engines.  When the system was switched off there were no major complaints, but there was an enormous increase in searches from Google Scholar from their Proxy Server, so much so that Google thought there was crawler activity!

To ensure their users are finding the correct content Utrecht University has added holding information to discovery tools e.g. Dutch national catolugue and WorldCat, shared SFX knowledge bases with Google Scholar and Scopuus, opened the repository for harvesting and support easy authentication for off campus access; developed a bookmarklet (using Javascript) to browser to allow login via University.  Kortekaas said there is a lot of other work to be done by libraries rather than worrying about discovery systems.

Discussion

The idea of a library doing away with the library catalogue and relying on Google Scholar was quite a controversial one with many librarians feeling very uncomfortable with this idea. Kortekaas responded to say that you have to look outside library and see what is going on and how users finding are the content; the library didn't choose to go with Google, the users did! This change is responding to real user behaviour rather than what they think their users want; it is silly to try and pull them back into library discovery system, better to educate them on platforms already using.

Joe Waas from CrossRef asked what would happen if Google Scholar just gave a month’s notice and shut down (as they have done with other services). From my experience Google Scholar actually only drives a small percentage of traffic to content compared to Google Web Search, but it offers an easy way to find scholarly content, allows libraries to flag holdings information to their users and most importantly Google Scholar provides publishers and libraries with a communication channel with Google.

There was also a lot of discussion about what libraries can add to bring the users back and how experimentation and really understanding what your users want is key.