How Can We Assess the Impact and ROI of Contributions to Wikipedia?
Posted by Brian Kelly on 27 Sep 2010
On Friday Andy Powell tweeted about a sentence he had written. He had written:
303 See Other is one way of responding to a request for a URI that identifies a real-world object according to Semantic Web practice (the other being the use of hash URIs)[1].
I responded to Andy suggesting that “this might have been the biggest impact you’ve made!”
The contribution Andy had made was to the Wikipedia entry for the HTTP 303 status code. Andy’s contribution to this brief entry was to add a “note about Semantic Web usage of 303 response to indicate real-world object being identified”.
My comment to Andy was based on the usage statistics for this entry – in August 2010 there had been 3,515 views of the page and over the past year there have been a total of 35,489 views as illustrated.
Now although the contribution appears modest it does amount to about a quarter of the full article:
The HTTP response status code 303 See Other is the correct manner in which to redirect web applications to a new URI, particularly after an HTTP POST has been performed.
This response indicates that the correct response can be found under a different URI and should be retrieved using a GET method. The specified URI is not a substitute reference for the original resource.
This status code should be used with the location header.
303 See Other is one way of responding to a request for a URI that identifies a real-world object according to Semantic Web practice (the other being the use of hash URIs)[1].
The addition makes it clear that the HTTP status code has an important role to play in Semantic Web usage – something that wasn’t mentioned in the original version. So if there are a further 35,000+ views in the next 12 months they may benefit from this additional information. And although there are much more detailed articles about use of the HTTP 303 status code in this context, such as “How to Publish Linked Data on the Web” the addition to the Wikipedia article has the advantage of brevity and the little effort needed to add the sentence.
In a recent post on Having An Impact Through Wikipedia I suggested that it would be useful if JISC-funded project work used Wikipedia as a means of disseminating their knowledge and went on to provide examples of how well-read technical articles in Wikipedia can be. But how would we assess the impact of such work and identify the return on investment?
In the case of the HTTP 303 article it appears that Andy created the first version of his update at 09.52 on Friday 26 September with the final version being published at 10.13. This suggests that the update took about 20 minutes to produce – although it should be noted that Andy pointed out that he “contribute[s] to wikipedia so rarely, it always takes me ages when i do“.
So can we speculate that 20 minutes work may provide a significant part of an article which will be read by over 35,000 people, based on current trends? And how does this compare with other ways in which 20 minutes of work? Is a blog post likely to have a similar number of readers?
I can’t help but feel that contributions to Wikipedia (by which I mean ‘sticky’ contributions which are not removed) may have a more significant contribution in certain areas that many other dissemination channels. Unfortunately there doesn’t seem to be any incentives for such contributions to be made apart from the ‘Big Society’ approach of doing good on a voluntary basis in order to provide benefits to others. This approach, I feel, won’t scale. So shouldn’t we encourage contributions to Wikipedia as a dissemination activity which should be formally recognised?
My question, therefore. is should JISC programme managers encourage projects to contribute to Wikipedia and encourage the projects to report on successes they have doing this? And if you want an example of the outreach which can be gained through use of Wikipedia have a look at the August 2010 usage statistics for the Scientology article (158,845 visits in the month) – an article which Martin Poulter (a well-established contributor to Wikipedia who is ICT Manager at the ILRT, University of Bristol) has contributed to.
Twitter conversation from Topsy: [View]
UK Web Focus (Brian Kelly) 
Andy Powell said
20 minutes for one sentence and a footnote sounds a bit OTT doesn’t it! The time is partly to do with having to remember Wikimedia mark-up syntax and so on… but it also relates to your point about impact. When you add something to a Wikipedia page you know that it is potentially going to be seen by a lot of people and you want it to be correct – getting the wordsmithing right is an important part of that. It took me 3 or 4 attempts to get that one sentence into a state that I was happy with and that was factually correct. I was writing about the pedantic web after all!
Sceptic said
“Unfortunately there doesn’t seem to be any incentives for such contributions to be made apart from the ‘Big Society’ approach of doing good on a voluntary basis in order to provide benefits to others. This approach, I feel, won’t scale.”
Isn’t the very existence of Wikipedia, with 3 million English articles easily the largest repository of human knowledge ever created [and in under a decade, for free, at that], a direct refutation of this point?
Or did you mean it won’t scale with technical/professional types with more lucrative or status-increasing ways to spend their comparatively valuable time?
Brian Kelly (UK Web Focus) said
Good point – clearly Wikipedia 3 million English articles does show how well the voluntary approach scales :-)
The point I meant to get across is that an environment which is critical of use of Wikipedia (which can sometimes be found in higher education and library circles) can be a barrier to contributions. When I’ve run Web 2.0 workshops I normally ask for a show of hands of people who have edited articles in Wikipedia and been disappointed by the small numbers. So I think there are barriers to contributing from technical/professional types – and I would like to see such barriers removed.
David Gerard said
The usual way we fix this from the Wikipedia end is the existence proof: i.e., Wikipedia exists, is highly popular and therefore useful, so if you come to Wikipedia instead of expecting it to come to you then things are better for you and your field.
Wikipedia is highly imperfect with flaws shallow and deep. It’s still there, however.
A lot of people think shouting at Wikipedia will make it go away. This is, IMO, unlikely.
Chris Rusbridge said
When I was at DCC we flirted with Wikipedia as a message channel. Two problems worried us, perhaps more than they should have. The first person who tried it uploaded text that was publicly available on the DCC pages (after all, we were happy with that text). It was immediately rejected as replicating copyright material from elsewhere; presumably Wikipedia does a search of the web! It was presumably possible to get round this one, but it didn’t happen for various reasons, including pressure of other work.
The second problem is that we had to prove impact. While you make a good point about the impact of Andy’s sentence, we were concerned that it would prove difficult to demonstrate impact if we put our efforts into Wikipedia rather than our own web site. In retrospect, I doubt we got this right; it seems clear that people are much more likely to look in Wikipedia than a particular service web site. Meanwhile you have demonstrated that impact can be measured.
Newsletter for September 2010 « UKOLN Update said
[…] How Can We Assess the Impact and ROI of Contributions to Wikipedia? […]
Microattributions, Wikipedia and Dissemination « UK Web Focus said
[…] microattributions. Looking at an example which I am familiar with, a year ago in a post entitled How Can We Assess the Impact and ROI of Contributions to Wikipedia? I commented on the potential value of entries in Wikipedia with the example of Andy Powell’s […]
The Blog as a Narrative or the Post as a Self-Contained Item « UK Web Focus said
[…] Amber some examples of use of Wikipedia in a research context, based on posts I’d written on How Can We Assess the Impact and ROI of Contributions to Wikipedia? (published on 27 September 2010) and How Well-Read Are Technical […]
Supporting Use of Wikimedia Across the UK Higher Education Sector « UK Web Focus said
[…] A follow-up post in September 2010 which asked How Can We Assess the Impact and ROI of Contributions to Wikipedia? […]
How Are You Using Wikipedia? « UK Web Focus said
[…] as Having An Impact Through Wikipedia, How Well-Read Are Technical Wikipedia Articles? and How Can We Assess the Impact and ROI of Contributions to Wikipedia?) I felt that such work was not appreciated as having value in promoting innovative use of […]