UK Web Focus (Brian Kelly)

Innovation and best practices for the Web

  • Email Subscription (Feedburner)

  • Twitter

    Posts on this blog cover ideas often discussed on Twitter. Feel free to follow @briankelly.

    Brian Kelly on Twitter Counter

  • Syndicate This Page

    RSS Feed for this page


    Creative Commons License
    This work is licensed under a Creative Commons Attribution 2.0 UK: England & Wales License. As described in a blog post this licence applies to textual content published by the author and (unless stated otherwise) guest bloggers. Also note that on 24 October 2011 the licence was changed from CC-BY-SA to CC-BY. Comments posted on this blog will also be deemed to have been published with this licence. Please note though, that images and other resources embedded in the blog may not be covered by this licence.

    Contact Details

    Brian's email address is You can also follow him on Twitter using the ID briankelly. Also note that the @ukwebfocus Twitter ID provides automated alerts of new blog posts.

  • Contact Details

    My LinkedIn profile provides details of my professional activities.

    View Brian Kelly's profile on LinkedIn

    Also see my profile.

  • Top Posts & Pages

  • Privacy


    This blog is hosted by which uses Google Analytics (which makes use of 'cookie' technologies) to provide the blog owner with information on usage of this blog.

    Other Privacy Issues

    If you wish to make a comment on this blog you must provide an email address. This is required in order to minimise comment spamming. The email address will not be made public.

Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges

Posted by Brian Kelly on 20 Sep 2010

In addition to the paper  on Approaches To Archiving Professional Blogs Hosted In The Cloud” which I mentioned last week a second paper was accepted by the programme committee of the iPres 2010 conference which is taking place in Vienna this week.  The paper, “Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges“, was co-authored by myself, Martin Hawksey, John O’Brien, Marieke Guy and Matthew Rowe. The paper is based on the JISC-funded developments to the Twapper Keeper service.  As summarised in the abstract:

This paper describes development work to the Twapper Keeper Twitter archiving service to support use of Twitter in education and research. The reasons for funding developments to an existing commercial service are described and the approaches for addressing the sustainability of such developments are provided. The paper reviews the challenges this work has addressed including the technical challenges in processing large volumes of traffic and the policy issues related, in particular, to ownership and copyright.

As described on the JISC Beginner’s Guide to Digital Preservation blog my colleague Marieke Guy will be presenting a poster of the paper and will also give a lightning presentation at the conference.

A copy of the poster is available on Scribd and is also illustrated.

Note that the paper states that “The software developments which have been funded will be made available under an open source licence“. Since the paper was submitted this development has now been done, as described n a post entitled “Twapper Keeper Goes Open Source“, and the final stage of the work will include working with the JISC OSS Watch service to ensure that best practices for releasing open source software are adopted.

One particular aspect of this work which pleases me is the use of two additional services which have been built on top of the Twapper Keeper developments: Andy Powell’s Summarizr service, which provides various statistics on hashtag usage and Martin Hawksey’s iTitle Twitter captioning service.  Increasingly it seems to me that Twapper Keeper is becoming an established component in the provision of an amplified event,with the Summarizr service seeming to provide a common way of providing statistics on Twitter usage at such events.

Such (unfunded) developments are interesting in terms of identifying the return on the JISC’s investment in funding this work. If these services had been included in the formal project plan the costs would have been  much higher.  It seems to me that the rapid innovation we are seeing across a number of JISC development activities could be regarded as the ‘Big Society’ in operation – rather than requiring funding to ‘do good’ the community is demonstrably willing to do good without prompting. I wonder if policy makers and politicians are aware of the added value which is being provided within the high education sector which is unlikely to be formally audited?  Hmm – I wonder if this means that unfunded development work should be accompanied by formal project reports :-)

2 Responses to “Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges”

  1. Les Carr said

    Can I just flag up that the JISC Open Impact project has created a twitter archiving function for EPrints (available in ).

  2. […] Twitter Archiving Using Twapper Keeper: Technical And Policy Challenges […]

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: