Data-driven recommendations: supporting user search

Alison Brock, Open University, on the JISC RISE project to improve the library user's search experience

What is activity data?

Activity data is data captured about customers' activities. Big retailers use complex algorithms to analyse this data and create new revenue opportunities / increase customer retention. Meanwhile, publishers are trying to raise awareness of content and increase readership. Across the library sector, we're trying to become more business-like and making better use of our activity data is one aspect of that.

What have the OU done?

The OU has a wealth of resources from which data can be harvested: SAMS single sign on, EZProxy, SFX and EBSCO's Discovery system. They developed programs to extract data from these systems that could create recommendations; Basic data can extracted from the logs to give information about the user's subject area / course and the level at which they are studying - this enables recommendations about "other people on your course looked at". Bibliographic data can be cross-referenced (eg via CrossRef) to retrieve more metadata. Usage of resources increases the value of a recommendation, and users are given the opportunity to rate recommendations, which in turn adds further data to the database. Search terms are also added, so there's a complex web of data from which recommendations are generated - course recommendations, relationship recommendations, search recommendations. Users were given the opportunity to opt out (of having their data used to inform recommendations).

How have users responded?

User surveys have shown that the recommendations are perceived as useful by 2/3 of respondents. Focus groups have explored these perceptions further: students are particularly interested to know whether the sources of the recommendation were getting good marks! Postgrads wanted to save time - which search terms would deliver best results? Generally, there was support for recommendations but caution around provenance - users seemed to trust Google's recommendations / algorithms more than OU's, because they knew and didn't respect their peers. [this is a fascinating inversion of the commonly held view that people trust recommendations from people they know]. Google Analytics showed that course and search recommendations were used more than relationship recommendations. [intresting that OU restricted project's ability to get feedback from students - had to go through a panel run survey etc]

Lessons learnt

• Users like recommendations in principle, but would like more detail about how they are being generated.
• Activity data needs to be combined with other sources eg bibliographic data, student data to increase the value of recommendations,
• The plan to release an open data set of anonymised data for other institutions to implement didn't come to fruition - lots of complexity around privacy

What next?

The data and code will be expanded for use in other projects eg OU Learning Analytics - data warehouse that enables OU to look at this data in a wider context. More info: http://www.open.ac.uk/blogs/RISE

Discussion: what do we all think about recommendation services? Sense tht the jury is still out; people don't necessarily have time to follow recommendations, and typical user behaviour is driven by search not browse.