Final Reports from UKOLN’s Evidence, Impact, Metrics Work

During 2010-11 I led UKOLN’s Evidence, Impact, Metrics work. The aim of this work was to identify best practices for gathering quantitive evidence and supporting metrics on the use of networked services to support institutional and project activities.

An Evidence, Impact, Metrics blog was set up on the UKOLN Web site, but the usage statistics for the first few blog posts provided evidence of a lack of use of the blog. This evidence led to a decision to set up an Evidence category on the UK Web Focus blog which was used with related posts which were published on this blog. The aim of the blog posts was to raise awareness of the importance of metrics, explore ways of gathering and interpretting such metrics and encourage discussion on the advantages and disadvantages of using metrics, leading to recommendations on how metrics can be used.

As described in a report on Blogs Posts about Evidence of Impact by 13 December 2011 there had been 28,5907 views of the 35 relevant posts published in this category. In addition there had been 275 comments, although the numbers for the comments include trackbacks and may also contain automatically-generated links from other WordPress blogs which may subsequently be deleted.

This example provides an illustration of how metrics can be used. It should be noted that this does not say anything about the quality or relevance of the posts. It also summarises ways in which the metrics may be misleading (and note it was only when updating the figures on the numbers of comments posted on the blog that I became aware that automatically generated links to posts on this blog may subsequently be deleted.

The final report on this work has been published on the Evidence, Impact, Metrics blog. The report has been produced as a series of self-contained documents which are suitable for printing as well as being published in HTML format.

The following sections of the report are available:

• Why the Need for this Work?:  This document provides the background to the work.
  [HTML] – [MS Word]
• Summary of Events:  This document provides a summary of the three one-day workshops and talks given at other events.
  [HTML] – [MS Word]
• Summary of Blog Posts:  This document provides a summary of the blog pots published related to this work.
  [HTML] – [MS Word]
• Feedback from the Second Workshop: This document provides a summary of the feedback received at the second one-day workshop.
  [HTML] – [MS Word]
• Summary of the Final Workshop:  This document provides a report on the third and final one-day workshop.
  [HTML] – [MS Word]
• A Framework For Metrics:  This document provides a summary of the lightweight framework developed for gathering quantitative evidence.
  [HTML] – [MS Word]
• Metrics FAQ:  This document provides an FAQ about the metrics work.
  [HTML] – [MS Word]

Note that the MS Word files are intended for printing in A5 format on a printer which supports double-side printing. For a number of the reports the content is duplicated to enable A5 summaries to be printed. The HTML format contain the same information in a more universal format.

As can be seen from the altmetrics manifesto the research community has strong interests in developing metrics which can help to identify evidence of value related to various aspects of research activities. The manifesto highlights the changes in ways in which research activities is being carried out and points out that “as many as a third of scholars are on Twitter, and a growing number tend scholarly blogs“.

The Evidence, Impact, Metrics work has sought to engage in a related area of work for those involved in both project and service who wish to make use of new approaches for which metrics can help to identify the value (or not) or new ways of working and share examples of appropriate best practices. Feedback on this work is welcomed.

---

Twitter conversation from Topsy:  [View]