Twitter Captioned Videos Gets Even Better
Posted by Brian Kelly on 28 Jul 2010
A recent post described the Captioned Videos of IWMW 2010 Talks which made use of Martin Hawksey’s iTitle service to synchonise a video with an accompanying Twitter stream.
I was not along in being impressed by this service – but since it made use of HTML 5 and the videos were encoded in MP4 format the video display would only work in a limited number of browsers, including Google Chrome. Many users who do not have access to such browsers will not be able to see how this service works and try out for themselves features such as searching the Twitter stream and having the video jump directly to the appropriate point.
However Martin has updated the service to provide a Flash-based solution for viewing the captioned video, thus enhancing access to a much wider audience.
So if you use Opera or Internet Explorer you can, for example, visit the page about Paul Boag’s talk and search for what he had to say about ‘legacy’ Web sites.
The rapid development we have seen with Martin’s service illustrates the benefits of a ‘just-in-time’ approach to accessibility which myself, Sarah Lewthwaite and David Sloan described in a paper on “Developing countries; developing experiences: approaches to accessibility for the Real World“. If the videos had not been made available due to concerns regarding the costs of providing captioning in order to conform with WCAG accessibility guidelines we would not have been in a position to exploit the rapid developments we are currently seeing across the Web development community, including this example of exploiting the Twitter stream – which, again, we needed to archive in order to provide the content for this just-in-time solution.
UK Web Focus (Brian Kelly) 
Tweets that mention Twitter Captioned Videos Gets Even Better « UK Web Focus -- Topsy.com said
[…] This post was mentioned on Twitter by Brian Kelly, Arata Kojima. Arata Kojima said: RT @briankelly: Twitter Captioned Videos Gets Even Better: iTitle supports mainstream browsers http://bit.ly/a3OMh6 More #a11y benefits […]
Anthony Leonard said
These are genuinely impressive approaches to what is an important and difficult are for video capture of academic lectures and seminars – i.e. subtitling and searchability. It’s perfect for a live streamed event where Twitter is in its element anyway. It’s fast, workable, cheap and crowdsourced for extra value. But there are dangers here.
Firstly, has any user testing been done with Twitter captioned videos? I wonder whether the inevitable time lag between the speakers words and a tweeted caption is off putting, in the way that badly synced dubbing is. Also if the tweets are actually a back channel or news feed this should be presented differently to genuine subtitles, e.g. by using a scrolling news ticker like on 24 news channels, instead of what looks like (but may not be) a faithful written expression of someone else’s words.
Secondly, I see a lot of development effort going into customised players that can do ever more fancy things, but over time these become difficult to maintain, get over burdened which features and buttons detracting from the actual content, and are difficult to integrate to existing systems. We ought to be working together to build on standards and open-source frameworks for media players where possible. Of those the two I can think of are the Kaltura Dynamic Player (heavily commercially focused) and the Opencast Matterhorn Player (still in early development stages), both of which have their pros and cons. Standards standards standards!
Brian Kelly (UK Web Focus) said
Thanks for the comments.
I agree with you that there is the potential for confusion between general discussion on an event’s back channel and an official summary of the talks. This was a reason why at #iwmw10 we had an official Twitterer who used the @iwmwlive twitter account. With the iTitle service you can choose whether to select all tweets of follow a particular Twitter account. The initial work, described in the post, made use of all tweets but follow-up work will use the official channel to provide the sub-titles. We’ve just done this for the video of Chris Sexton’s talk – you can compare the captioning using the full Twitter stream with the captioning using the @iwmwlive Twitter stream.
These developments could then enable user evaluations to be carried around. Note that we have no plans to carry out such evaluations ourselves, but there is nothing to stop others doing so.
Your comments about standards are interesting. The initial post on Captioned Videos of IWMW 2010 Talks described use of an open standard (HTML5) – but access was limited due to the prevalence of browsers which don’t support HTML5 (old versions of IE) and the use of a popular video format (MP4) which is encumbered by patents and isn’t supported by other browsers. In the future use of open formats for video may help to overcome such barriers but at present there is limited support in tools and browsers which most users actually. The iTitle software also allows one to create SMIL files – another open standard which has failed to take off. So when you say “Standards standards standards!” I would respond with “Standards that work, standards that users can use, standards that can achieve users’ goals!”
Anthony Leonard said
I’m certainly not pro-standards for their own sake. Personally I like “standards we trust”. After all, like anything that needs voluntary take up, it’s a brand thing.
Actually in this case I was more emphasising that we shouldn’t be re-inventing the wheel. That I’m pretty convinced of. Media players are hard widgety-type applications that should be embeddable into static web sites. They sometimes need to show secured content using authorisation drawn from the parent site’s access controls. They have to be easily customisable by shape and size and reflow their contents accordingly, potentially to zero display height for audio players. They also have to be accessible. In an academic context they sometimes need to show two or even three video images side by side. That plus pull-down playlists, search interfaces, thumbnail indexes, share/embed buttons, view me in fullscreen and/or HD options, closed captions, audio descriptions, metadata and transcript links, and now twitter backchannels, it all gets tricky.
We ought to be building players that make use of what native HTML5 offers where possible, falling back to Flash where this isn’t possible, and wrapping the whole lot in future proof javascript that we develop together as a community. Failing that we go with what’s out there. It’s the Big Society in turbulent times after all ;-)
Tony Hirst said
In the uTitle version of the app, which allows viewers to twitter annotate videos post hoc, using a time stamp relative to the time at which they start a recorded video playing, I bounced around a couple of ideas with Martin that would allow the time stamp to be recorded when a user starts to type a tweet, rather than when they send it.
A similar client could possibly be developed for tweeting at live events, that maybe adds an offset between the time a user started to type a tweet and the time at which it is sent? In short term this could be passed in the tweet body, but this time offset would also be a good candidate for a twitter annotation.
tony
Martin Hawksey said
Here’s an idea: Once I include Vimeo support for uTitle you could upload a blank video which is say 90 minutes long. Load up the video in uTitle and when the session starts hit play. Then use the uTitle interface for making tweets (the Tony Hirst patent pending auto-mark feature creating timestamps when the tweeter starts typing).
Once you are done uses Vimeo’s feature of being able to replace existing videos to swap the blank video with the actual video of the event which will have the same video ID.
At this point you could either parse out the results into iTitle using the csv import feature or even cooler distribute a link to the uTitle clip so that other people can add their own comments.
[I know what I’m going to be doing tonight ;-)]
Martin
Martin Hawksey said
I feel like I’m taking residency at UK Web Focus ;-)
Standards are an interesting one. When I started developing iTitle on the back of Tony’s earlier work I knew nothing about subtitling standards. It quickly became apparent that like a lot of the web on the rare occasions that standards existed they were tweaked and extended to fit what people were trying to do on a day to day basis. This post on Video: the track element and webM codec (particularly the comments) tickled me recently as the community tries to work out what the next standard will be.
It was only recently that it occurred to me that I should have in fact called iTitle a ‘Timed Track’ service to avoid confusion with the Accessibility sectors definition of subtitles/captions (although arguably iTitle provides broader accessibility through improved navigation of video clips)
Whilst the iTitle tool generates subtitle files in SMIL 3.0 (smilText), SubRip and W3C Timed Text Markup Language (TTML), at the end of the day it is just some glue sticking two external resources together (video and tweets), as new standards emerge all I need to do is change the formula of the glue.
[Re user testing I would love to say that this is in the project plan but, well this is me just developing something in my spare time. There’s no funding, no big master plan, just me drawing on the suggestions of Tony, Brian and any other users adding new features as and when people need them. Matterhorn looks interesting]
Martin
captioning video with its twitter stream — cafedave.net said
[…] UK Web Focus ] Twitter Captioned Videos Gets Even Better – has anyone worked out how to do this with Australian TV? I think being able to playback a […]
What influenced me this week… at Scenario Girl said
[…] Twitter captioned video gets even better […]