Metrics For Measuring The Effectiveness Of Blogs
Posted by Brian Kelly on 29 January 2007
Do We Need Blog Metrics?
In light of such surveys we might therefore expect cost-benefit analyses of services such as blogs to be included in similar surveys in the future. And such approaches might also be applied more widely, perhaps with institutions being expected to justify the costs of providing blog services. Indeed institutions would probably be wise in asking themselves such questions before committing themselves to significant expenditure.
What is the Purpose of the Blog?
If you wish to measure the effectiveness of a blog, you’ll need to be clear of the purpose of the blog. A blog may have several purposes, and in a teaching and learning context this might include self-reflection and collaborative working. However I will leave such issues for colleagues at CETIS to address. Instead I’ll focus on use of blogs by the research community and service departments (such as IT services and libraries), which are intended to provide dissemination to and engagement with the user community. Metrics for measuring dissemination and engagement are probably easier to identity than the complex range of activities associated with the learning process – so I’ll start off with the this.
I’m A C-List Blogger
Scott Wilson’s post about the C-List blogger tool was pointed out to me recently so I thought I’d explore this further. I discovered that, like Scott, I am a C-List blogger. So I am entitled to add the tasteful badge to my blog.
OK, I know, the badge is tacky, as is the methodology they use (it’s simply based on the ranking in Technorati). Not only that, but entering the URL of a non-existent blog and you’ll find that it’s classed a a D-list celebrity blog! Proof, I concluded, of the flawed nature of this service – although I then wondered whether this might be a subtle post-modern reflection of a Big Brother celebrity culture in which even Mr 404 can be a minor celebrity!
A Portfolio of Metrics
If a single metric can be flawed, can there be a portfolio of quantifiable statistics which can help identify the effectiveness of a blog? Some possibilities include:
Numbers of visitors: If nobody visits your blog, then it has failed has a dissemination channel. If it has lots of visitors, then it may be working effectively. As with conventional Web usage statistics, this metric isn’t without flaws, but it will have a role to play.
Numbers of posts: If a blog is meant to provide a dissemination function, then the number of posts will provide a measure of this (although clearly this metric is open be abuse!)
Numbers of comments: If a blog is meant to provide a means of engaging with the user community, receiving feedback, etc. then the number of comments can provide an indication of the effectiveness of this aspects of the blog’s role.
Numbers of inward links: As with conventional Web sites, the number of links to the blog can provide an indication of the quality of the content. (It should be noted that this is one of the main features of the rating scheme used by Technorati).
Geographical distribution of users: If a blog is meant to have an impact at a national or international level, then the geographical distribution of the readers will be an indicator to be recorded.
Numbers of feed readers: Unlike conventional Web sites, blogs normally allow their content to be syndicated using RSS or Atom. Since users of RSS readers will not normally be included in the blog site usage statistics, then will be a need to record the numbers of access to the blog’s feed.
Numbers of aggregators: Since a blog’s feed may be harvested once and cached, with subsequent reads accessing the cached content, such usage statistics will not be available to the provider of the original blog. In such circumstances, an analysis of the aggregator services may provide an indication of secondary accesses to the blog.
This is Flawed!
I am aware that these metrics have their limitations. However it is still true that blogs may be successful or be failures, and may provide a valuable return on investment or may be a waste of money. And may organisations will be asking how the effectiveness of blogs can be measured. Indeed I recently came across the International Museum Blog Survey 2007 in which the responses to the question “How do you measure the success of your blog?” were:
- Number of visitors (33 out of 54)
- Geographical spread of blog visitors (15 out of 54)
- Numbers of comments posted to blog per day/week/month (20 out of 54)
- Quality/relevance of comments (27 out of 54)
- Number of links to blog from other sites (25 out of 54)
- Number of media mentions (10 out of 54)
We will need metrics, I suggest. We will therefore need to collect the statistics and the qualitative data. But we will also need to ensure that the limitations of such approaches and well-understood and the potential dangers of wasting time and effort in collecting flawed data. Perhaps most importantly, though, will be the need to develop more sophisticated approaches for measuring the effectiveness of blog services.