If a paper is deposited in an institutional repository and nobody notices, can the associated work be seen to have any relevance? I wondered about this recently after looking at the download statistics for my papers hosted in Opus, the University of Bath repository. Normally I’m interested in the reasons for popular downloads (such as the evidence that this might suggest that the large numbers of downloads are due to the ‘Google juice’ provided by links from popular Web site). However as part of the preparation for a talk on “Open Practices for the Connected Researcher” I’m giving at the University of Exeter during Open Access Week I was interested in lessons to be learnt from papers which hardly anyone downloads.
In my case the papers nobody cares about are an article published in LA Record in 1997, a paper on Collection Level Description also published in 1999 which I had forgotten about until I rediscovered it a few years ago and uploaded to the repository, the final report for the QA Focus project and a peer-reviewed paper on Using Context to Support Effective Application of Web Content Accessibility Guidelines.
It was the peer-reviewed paper I was most interested in. This paper, written by myself, David Sloan, Helen Petrie, Fraser Hamilton and Lawrie Phipps and published in the Journal of Web Engineering (JWE), has only been downloaded twice. Clearly nobody is being deafened by the impact of this paper challenging the status quo!
Given that a total of 13,104 papers of mine have been downloaded from the repository what are the reasons for the lack of interest in this paper?
The obvious starting point would be the content. But this paper was a follow-up from previous papers on Web accessibility which have been well-read and widely-cited and the interest in our papers in this area has continued.
Looking at the email folder about this paper it seems that the first version of the paper was submitted to the publishers in July 2005. I seem to recall that we were invited to submit a paper based on an updated version of a paper on Forcing Standardization or Accommodating Diversity? A Framework for Applying the WCAG in the Real World by the same authors which had been presented at the W4A 2005 conference.
We received positive comments from the reviewers in August 2005 and responded with appropriate updates to the paper. But then everything went quiet. It wasn’t until August 2006 when we received the final proofs of the paper and September 2006 when we received confirmation that the paper had been accepted and the paper had published in the Journal of Web Engineering, Vol. 5 No. 4 in December 2006. This was 17 months after we had submitted the first version of the paper!
By this time myself and my co-authors had forgotten about the paper, and the ideas we described had been superceded by a paper on Contextual Web Accessibility – Maximizing the Benefit of Accessibility Guidelines presented at the W4A 2006 conference in May 2006.
Looking at the download statistics for my papers it seems that I began depositing items in the Opus repository in October 2008. My first set of papers were deposited by repository staff based on the links available from the UKOLN Web site. However it would appear that the JWE paper had not uploaded, probably because I had failed to include it in my list of publications due to its long gestation period. A few months ago I noticed that the paper had not been uploaded to the repository so on 17 May 2012 I uploaded the paper.
The reason for the lack of downloads is now clear: the paper wasn’t available until recently! And by the time the paper was available the ideas were no longer current.
What are the lessons which can be learnt which I can share in my talk on “Open Practices for the Connected Researcher“? I would suggest:
Repository items need to be made publicly available when the ideas are current. Depositing old papers may be useful for preserving the content and for record-keeping purposes, but not if the aim is to maximise the impact of the ideas.
Of course there is a bigger question about the value of peer-reviewed papers. In his 1,000th blog post Tony Hirst gave his reflections on The Un-academic. Tony pointed out that “Formal academic publications are a matter of record, and as such need to be self-standing, as well as embedded in a particular tradition” and contrasted this with blog posts which are “deliberately conversational: the grounding often coming from the current conversational context – recent previous posts, linked to sources, comments – as well as discussions ongoing in the community that the blog author inhabits and is known to contribute to“.
Tony argued the value of blogs in the support of the research process by point out blog posts can provide:
“a contribution to a daily ongoing communication with a community that often mediates its interests through the sharing of links (that is, references); in part it’s a contribution of ideas at a finer resolution than a formal academic reference, and in completely different style to them, to the free flow of ideas that can be found through the searchable and sharable world wide web.“
Since 2005 myself and my colleagues have had peer-reviewed papers published at the W4A conferences in 2005, 2006, 2007, 2008, 2010 and 2012. This is part of the “annual ongoing communication with a community that often mediates its interests through the sharing of links (that is, references)”. However sometimes this process goes wrong, as has been described in this post. Although the problems associated with the long time frames it can take for research work to be published this doesn’t mean that the process of research publications is fundamentally flawed. However I think this example does illustrate the need for researchers to make “contribution to a daily ongoing communication with a community that often mediates its interests through the sharing of links“.
Tony’s blog post concludes by referencing a number of recent posts by Alan Levine (@CogDog) in which he has shared his thinking on blogging: The question should be: why are you NOT blogging?; Every box you type in can be a doorway to creativity, and in a roundabout way, Gotta know when to walk. Alan’s first post provides his reflections on his blogging activities since he started 0n 19 April 2003. This long post is worth reading, but can be summarised very succinctly:
So here is why I blog. It is foolish and informationally selfish, not to.
Perhaps that should be the key message I give in my talk in Exeter during Open Access Week. Oh, having reflected on the paper which nobody reads I have decided that if a peer-reviewed paper is not read, this is a failure. My time and the time spent by my co-authors in writing the paper could have been more productively spent on other work. And no, unlike blog posts in which writing ideas may be a useful process in itself, peer-reviewed papers aren’t intend to assist in self-reflective.
Twitter conversation from Topsy: [View]