UK Web Focus (Brian Kelly)

Innovation and best practices for the Web

Posts Tagged ‘DERT12’

Personal Perspectives on How Metrics Can Influence Practice

Posted by Brian Kelly on 9 Apr 2012

A few days ago I favourited the following tweets from @lesleywprice:

RT @LnDDave: Too many people focus on the metric instead of the impact < and its impact that matters otherwise what is the point#trainchat [source]

RT @LnDDave: Too much of ROI is sterile data; use the data to help tell a better story < I like story telling makes it real #trainchat  [source]

Interestingly, Lesley noticed that I had favourited her tweets and provided some additional contextual information:

@briankelly…tks for faving tweets. I have taken part in a couple of really good tweetchats this week worth a look #trainchat swchat

This made me realise how the simple act of favouring a tweet is an action which can provide an identification of interest and lead to a subsequent dialogue – a good example, I feel, of frictionless sharing in action. Having been given the context to those two tweets I was able to find a post on “#TrainChat: Recap of our Twitter Chat with David Kelly“. This summary, incidentally described how “For the sake of clarity, we’ve condensed Kelly’s various tweets into a single response, and in places cleaned up a little bit of Twitter grammar” – an interesting example of an emerging practice for the curation of tweets.

Although this online Twitter chat covered “Three Essential Tips for New Online Trainers” the discussions about metrics for learning resonated with me in another context – our rapper sword dancing team’s performance in the annual DERT 2012 competition. We had a great time dancing in pubs in Soho and felt that our dances reflected the dedication we had shown in practices coming up to the competition, although we were conscious of the mistake we made in one of the competition spots. However it wasn’t until we saw how the other teams did that we realised how disappointed we felt – we were in bottom place :-( Even worse, not in the bottom place in the Premier Class, but the Olympic Class, the second division. I’ll not comment on the fact that in this traditional male dance we were beaten by three women’s team and two mixed teams, but being beaten by a morris team was embarrassing!

In the debriefing which took place on the first practice evening after the competition we agreed that our standard of dance had failed to keep up with our peers in recent years. We acknowledged that the judges’ comments were fair and that even though the marking system had its flaws, the scores were a valid reflection of the standard of our dance compared with the other teams (we should add that with scores for the comic characters from one judge of 10 and 12 out of a maximum of 10, the aberrations in the marking sometimes worked to our advantage!)

At the debriefing we agree that we should set a goal of being in the top three at next year’s event, for the dance team, musicians and characters. We also agreed that we needed to develop a new dance, in keeping with the current expectations and standards which have been raised over recent years. We also agreed that to achieve these new team goals we needed to have a more coherent approach to our weekly dance practices and regular dances in pubs around the area and at folk festivals.

How does this relate to the comments about ROI and learning analytics? For me it is clear that in this context:

  • It is easy to be self-deluded about the quality of one’s performance.
  • It is also easy to be self-deluded when reading scores and interpretting the feedback.
  • Being able to have evidence on one’s ranking with one’s peers can provide a better understanding of the perceived value of the performance.
  • Such evidence can inform subsequent goals for improvement.

But the interesting aspect was that the stories, the comments from the judges which said, for example “Good strong beat and nice changes” could be misleading. In our case the value was to be found in the numeric scores, but the scores in comparison with others and not in isolation.

Of course, whilst such considerations may be important in the context of rapper sword dancing, it would be inappropriate to apply such views to, say, learning analytics. After all, rapper dancing is important, whereas learning is supposed to be about fun, isn’t it.

Finally, we were pleased with the performance after our first dance. The judges gave us the following scores:

Stepping  Sword
Presentation Buzz
Tommy Betty
Scores out of 15 Scores out of 10
Judge 1  11 10 11  6  6
Judge 2  9  9  10  7  4  4

What do you think – perhaps of our dance, but also on the value of metrics for such cultural activities? A video of the dance is available on YouTube and embedded below.

Posted in Evidence | Tagged: | 1 Comment »