UK Web Focus (Brian Kelly)

Innovation and best practices for the Web

Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges

Posted by Brian Kelly on 20 Sep 2010

In addition to the paper  on Approaches To Archiving Professional Blogs Hosted In The Cloud” which I mentioned last week a second paper was accepted by the programme committee of the iPres 2010 conference which is taking place in Vienna this week.  The paper, “Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges“, was co-authored by myself, Martin Hawksey, John O’Brien, Marieke Guy and Matthew Rowe. The paper is based on the JISC-funded developments to the Twapper Keeper service.  As summarised in the abstract:

This paper describes development work to the Twapper Keeper Twitter archiving service to support use of Twitter in education and research. The reasons for funding developments to an existing commercial service are described and the approaches for addressing the sustainability of such developments are provided. The paper reviews the challenges this work has addressed including the technical challenges in processing large volumes of traffic and the policy issues related, in particular, to ownership and copyright.

As described on the JISC Beginner’s Guide to Digital Preservation blog my colleague Marieke Guy will be presenting a poster of the paper and will also give a lightning presentation at the conference.

A copy of the poster is available on Scribd and is also illustrated.

Note that the paper states that “The software developments which have been funded will be made available under an open source licence“. Since the paper was submitted this development has now been done, as described n a post entitled “Twapper Keeper Goes Open Source“, and the final stage of the work will include working with the JISC OSS Watch service to ensure that best practices for releasing open source software are adopted.

One particular aspect of this work which pleases me is the use of two additional services which have been built on top of the Twapper Keeper developments: Andy Powell’s Summarizr service, which provides various statistics on hashtag usage and Martin Hawksey’s iTitle Twitter captioning service.  Increasingly it seems to me that Twapper Keeper is becoming an established component in the provision of an amplified event,with the Summarizr service seeming to provide a common way of providing statistics on Twitter usage at such events.

Such (unfunded) developments are interesting in terms of identifying the return on the JISC’s investment in funding this work. If these services had been included in the formal project plan the costs would have been  much higher.  It seems to me that the rapid innovation we are seeing across a number of JISC development activities could be regarded as the ‘Big Society’ in operation – rather than requiring funding to ‘do good’ the community is demonstrably willing to do good without prompting. I wonder if policy makers and politicians are aware of the added value which is being provided within the high education sector which is unlikely to be formally audited?  Hmm – I wonder if this means that unfunded development work should be accompanied by formal project reports :-)

Advertisements

2 Responses to “Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges”

  1. Les Carr said

    Can I just flag up that the JISC Open Impact project has created a twitter archiving function for EPrints (available in http://files.eprints.org/563/ ).

  2. […] Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

 
%d bloggers like this: