Plenary: ‘Towards the next Research Excellence Framework’

Presenter: Steven Hill (HEFCE)
Blogger: David Walters (King’s College London)

After extensive consultation, HEFCE and the other three UK funding bodies have published details of a new policy for open access relating to future research assessments after the last REF (submitted in 2013). Steven presented on aspects of the new policy and the motivations which are driving this change.

Steven was quick to point out that the policy is still in the early stages. HEFCE are framing the next REF by looking forward to expected changes in research methods and practices. In particular, they are looking at questions on how this will be assessed. The ‘juicy details’ will be forthcoming.

Details of the announcement can be found below, but in brief there is a focus on open access full-text deposit and metadata discovery for article submissions. This will require significant engagement by authors in terms of open access if they want to submit papers for assessment.
http://www.hefce.ac.uk/news/newsarchive/2014/news86805.html
------------

Open

Steven noted that, in the changing research landscape, alongside open access there are other ‘open’ terminologies emerging like ‘open research’ and ‘open science’.

The Budapest initiative and subsequent definition of open access encapsulated the social revolution underway in how we perceive ownership of information. Their inspirational opening paragraph really sets the scene for the changes to come.

Steven quoted from the book ‘Reinventing Discovery’, where Michael Nielsen argues that we are living at the dawn of the most dramatic change in science in more than 300 years. Steven discussed the importance of moving information out of people's heads, and out of siloed laboratories, to be accessible on the network as a fundamental imperative on this road to change.
------------

Funders response

Steven commented on funder’s response to this issue, which have served to drive and incentivise the issue of open access.

The new HEFCE policy will work alongside this by removing those perceived barriers, whilst protecting the elements of dissemination that should be retained.
------------

The Post 2014 REF

• Deposited
• Discoverable
• Accessible

Steven pointed out that ‘The Post 2014 REF’ is really the correct terminology for the forthcoming assessment. Phrases like ‘REF 2020’ are misleading as we don’t yet know when the assessment will take place. Assessments usually take place every 5-8 years.

There is a focus on open access full-text deposit and metadata discovery. This applies to Journal articles and conference proceedings accepted for publication after 1 April 2016.

The requirements state that peer-reviewed manuscripts must be deposited in an institutional or subject repository on acceptance for publication. The title and author of these deposits, and other descriptive information must be discoverable straight away by anyone with a search engine. The manuscripts must then be accessible for anyone to read and download once any embargo period has elapsed.

Steven highlighted the fact that deposits are not limited to institutional repositories. However, he expects that universities will require this for their researchers in order that they have better control over the assessment.
The aims are to make papers discoverable as early as possible and accessible through whatever open access route is available. They are green/gold neutral, but expect required embargoes for publically accessible open access to be 12 and 24 months depending on the discipline.

There is a feeling that the open access monograph landscape is not yet developed enough to make this an assessment criteria of the panel. This is especially in terms of business models, but the board does recognise emerging opportunities and associated risks. Consequently, the policy does not apply to long-form outputs. However, they are discussing the possibilities of additional credits for authors who do make their monographs and book chapter’s available open access. They expect this will be a criteria for this in future REF assessments.

They are also discussing the availability of additional credits for reuse rights and text mining. Text mining is expected to be available under the new government copyright legislations outlined by the Hargreaves review.
Whilst there are exceptions for submission, they don't think these will be widely used.
Based on the results of the REF 2013 assessment, they expect that 96% of papers submitted will be able to comply with these requirements without changing their choice of publication venues.
------------

Open data

Open data could be rewarded in the next REF. However, it's a complex and diverse issue. Sometimes it's not possible to make your research data openly available, for example when dealing with issues of confidentiality. Sometimes research data is very large. Steven gave the example of the square kilometre array, which is the world’s biggest radio telescope. In terms of data, this project annually collects the equivalent of 50000 DVDs.

The culture surrounding open data is still being developed. It is most important that all key stakeholders play a role in supporting researchers as they adapt and react to this new imperative. For HEFCE this is a key consideration at the forefront of their thinking.
------------

Metrics

HEFCE have been performing a metrics research review. They have been considering some key questions. Primarily, what kinds of metrics for research performance are out there? What do they measure? Are they fair? What are the behavioural impacts of using metrics?

Steven commented on example of gender bias in citations. Overall, men are cited more than women.
------------

Open research assessment

Steven took us through an example of the classical research cycle. This has been described and thought of in its current form for a very long time. He commented that open access serves to take part of this cycle and make it openly available. The same can be said for open data.

However, the changing landscape makes it possible for this entire cycle to be revolutionised. Resources like figshare enable ‘micro publishing’. This allows for little chunks of data and small experiments to be instantly accessible potentially providing a much broader picture.

Steven also mentioned ‘Open notebooks’ as a scientific method. In these research cycles, the whole process becomes open at all stages.

Steven also commented that Post-publication peer review is becoming more important than pre-publication peer review, as it provides impact and analysis in real-time.

All these networks and linkages across institutions, resources – across large and small data and research projects – present a real challenge for the community as researchers and assessors.

New methods and standards are required in order support these new evolving research practices and to ensure fair and accurate assessment.