Whilst preparing a follow-up post on institutional repositories I started to explore the data which has been collected by the JISC-funded CORE project. The CORE (COnnecting REpositories) project aims to “facilitate free access to scholarly publications distributed across many systems“. The CORE Web site, which was developed at the Open University, provides access to four applications including:

Repository Analytics - A tool that enables to monitor the ingestion of metadata and content from repositories and provides a wide range of statistics.

I wanted to use this service to find information about the repositories provided by the 24 Russell Group universities. However, as can be seen from the accompanying screenshot, it was not easy to associate a repository with its host institution.

The first four examples illustrate the difficulties I had in using the information. The first entry, for the Aberdeen University Research Archive, gives a clear indication of the host institution. The second example, Abertay Research Collections, is somewhat more obscure, unless you know that Abertay is the name of a Scottish university. However the next two examples, Access to Research Resources for Teachers and Advanced Knowledge Technologies EPrints Archive, give no clue as to the host institution.

This meant that browsing the list was not an effective way of finding the repositories for the Russell Group universities. In addition the search interface was misleading: a search for “Southampton” enabled me to find eCrystals – Southampton and Electronics & Computer Science EPrints Service – University of Southampton - but not the main repository which has the name e-Prints Soton.

Using CORE to Search for Russell Group University Repositories

Despite the limitations caused by the lack of institutional identifiers I felt it would be useful to discover information held about Russell Group university repositories, based on a search of the CORE system using the obvious name for the host institution. The following table summarises the findings for a survey carried out on 21 February 2013 using the search term given in the second column.

Note that the Repository Analytics page does not appear to provide a formal definition of the data collected. However from hovering over the accompanying icon for the entries it appears that the Metadata Download column gives the number of metadata records, the Metadata Readable column gives the number of links extracted from the metadata and the PDF Download column the number of PDFs which were downloaded.

Discussion

It is difficult to interpret the data given in the table: the entry for the UCL Discovery repository, for example, tells us that there are 0 metadata records, with 245407 links having been extracted from these records and 2 PDFs downloaded!

However the table does suggest patterns of naming conventions for institutional repositories, such as the institutional name being provided at the beginning (“University of Birmingham Research Archive, E-prints Repository“, “University of Liverpool Research Archive” and “LSE Research Online”) or end of the repository name (“EPrints at the Centre for Scientific Computing, University of Warwick“, “Electronics & Computer Science EPrints Service - University of Southampton” and “Computer Laboratory Technical Reports - Cambridge University“) together with a large number of examples which use a partial form of the institution’s name (e.g. “Edinburgh Research Archive”, “Glasgow DSpace Service” and “Manchester eScholar Services“).

But of greater interest are the institutional repositories which have been harvested by CORE but are missing from this search such as “e-Prints Soton” and the “White Rose E-theses Online” and “White Rose Research Online” repositories which are used by the universities of Leeds, York and Sheffield.

Whilst the ownership of a repository will be apparent to the end user who access the service via the main entry point (perhaps from the institution’s Library Web site) in a number of cases such information is not apparent when the repository has been harvested and accessed using other systems such as, in this case, the interface developed by the CORE project.

In light of the findings from a survey of Russell group Universities, I would make the following simple recommendation:

Institutional repositories should contain the name of the host institution.

In order to illustrate the need for such a recommendation, here are a list of repositories which have been harvested by CORE:

Access to Research Resources for Teachers - Department of Computer Science E-Repository - Enlighten - Modern Languages Publications Archive - Online Publications Store - Open Research Online - Pharmacy Eprints

If you are unfamiliar with these repositories, would you to able to guess who owns them?

Or, to put it another way, meaningful metadata is important for repositories!

View Twitter conversation from: [Topsy] | View Twitter statistics from: [TweetReach] – [Bit.ly]

How significant is use of third-party repository services?

In a recent post I explained Why I’m Evaluating ResearchGate. In the post I summarised the reasons why I felt that researchgate.net could provide an additional service for depositing research papers which would complement Opus, the University of Bath institutional repository. But what others services might also be relevant? And which services are hosting the largest numbers of research papers?

In order to seek answers to these questions, I used Google to provide a measure of the size of a number of hosting services for PDFs and the number of PDFs they host. The services I analysed were:

• Researchgate.net: This site is described in Wikipedia as “a social networking site for scientists and researchers to share papers, ask and answer questions, and find collaborators. The site has been described as a mash-up of “Facebook, Twitter and LinkedIn” that includes “profile pages, comments, groups, job listings, and ‘like’ and ‘follow’ buttons”. Members are encouraged to share raw data and failed experiment results as well as successes, in order to avoid repeating their peers’ scientific research mistakes.“
• Academia.edu: This site is described in Wikipedia as “a platform for academics to share research papers. It was launched in September 2008. Currently the site is approaching 2 million registered users.^[2] The platform can be used to share papers, monitor their impact, and follow the research in a particular field.“
• Mendeley.com: Thus site is described in Wikipedia as “a desktop and web program for managing and sharing research papers,^[2] discovering research data and collaborating online. It combines Mendeley Desktop, a PDF and reference management application (available for Windows, Mac and Linux) with Mendeley Web, an online social network for researchers.^[3][4][5] Mendeley requires the user to store all basic citation data on its servers – storing copies of documents is at the user’s discretion“.
• Citeulike.org: This site is described in Wikipedia as “based on the principle of social bookmarking [the service] is aimed to promote and to develop the sharing of scientific references amongst researchers. In the same way that it is possible to catalog web pages (with Furl and del.icio.us) or photographs (with Flickr), scientists can share information on academic papers with specific tools (like CiteULike) developed for that purpose“.
• Scribd.com: This site is described in Wikipedia as “a document-sharing website that allows users to post documents of various formats, and embed them into a web page using its iPaper format“.

Many researchers will probably be familiar with the first four services listed. The fifth service, scribd.com, is included in order to explore whether a general-purpose PDF repository service could have a role to play in supporting the sharing of research publications.

Findings for the Coverage of the Services

Google was used in order to provide an estimate of the coverage of the services, including the total number of resources which have been indexed by Google and the number of PDF files. The findings are given in the following table. Note that the figures were initially collected on 6 February 2013. In order to check the volatility of the findings the searches were repeated on 11 February.

It seems that Scribd hosts a very large number of resources (although a finding of 3 PDF resources originally found was discarded as the results seemed to be unreliable).

However since Scribd is a general purpose repository service, it was felt that ResearchGate provides a repository of a large number of PDFs resources which are more relevant for researchers. In light of this confirmation of the popularity of Researchgate an additional survey was carried out which reported on use of the service across Russell Group universities.

Findings for Institutional Use of Academic.edu and Researchgate

On 1 August 2012 a Survey of Use of Researcher Profiling Services Across the 24 Russell Group Universities was published on this blog. This survey has been repeated in order to detect changes in the use of ResearchGate. Since the original survey also provided an analysis of Academia.edu, this was also included in the current survey. The results are given in the following table. Note that the data is also available in Google Spreadsheets.

Note: *  As described in the previous survey the numbers of Academia.edu members is obtained by entering the name of the institution in the search box.

Conclusions

Nos. of items deposited in Researchgate in Aug 2012 (blue) & Feb 2013 (red)

Nos. of Researchgate Members in Aug 2012 (blue) & Feb 2013 (red)

As illustrated in the accompanying diagrams it seems that the numbers of researchers who have signed up for a ResearchGate account has grown significantly over the past six months, and now stands at over 33,000 users, a growth of 82.7%. The numbers of papers which have been deposited by researchers at Russell Group universities has also grown to a total of over 477, 000 items. However since this represents a growth of 11.9% over six months it suggests that new members are providing metadata records only and not depositing the full text.

I therefore conclude that the conclusions I reached in my post which explained Why I’m Evaluating ResearchGate were correct and ResearchGate is a service which I should use not only to provide a presence about my research activities but also to host my research papers. I do wonder, though, whether the large numbers of items which have been deposited in ResearchGate is due to promotion of the service with the Russell Group universities or represents a bottom-up approach, in which researchers have recognised the benefits of the service and recommended it to their peers?

View Twitter conversation from: [Topsy] | View Twitter statistics from: [TweetReach] – [Bit.ly]

In a recent post which explained Why I’m Now Embedding ORCID Metadata in PDFs I described my intentions to ensure that my research papers contains rich embedded metadata to held enhance the discoverability of the publications, ensure that authorship is asserted (by embedding the ORCID ID of the authors of the papers) and ensure that embedded images contain descriptions which help ensure that the content can be understood by visually impaired readers. In addition I wish to ensure that the PDF is stored in PDF/A format which provides a more preservable format.

In light of discussions on the blog and on email I have decided to embed the ORCID IDs for co-authors of my peer-reviewed papers although, as suggested by Geoffery Bilder, I will be embedding the HTTP URI version of the ORCID IDs (e.g. http://orcid.org/0000-0001-5875-8744) rather than just the ORCID ID itself (0000-0001-5875-8744). In addition I will also be embedding the DOI for papers which have been assigned a DOI.

But I am now faced with the problem of where the paper should be hosted. This post summarises the processes I am using in the selection of an appropriate repository service to complement my institutional repository.

Selection Processes

As described previously workflow processes used in the creation of cover sheets for items hosted in our repository means that metadata embedded in PDFs is lost. Although we’re having discussions with repository staff about this, it occurred to me that I now have an ideal opportunity to make use of a third-party repository service.

In the past I have normally deposited papers in my institutional repository and used third-party services (such as ResearchGate and academia.edu) to host the metadata, with links being provided to the full-text of the papers hosted in the institutional repository. The main reason for doing this was to ensure that usage statistics for accesses of the full-text was available in a single location rather than being fragmented across a range of services. There was a need to minimise the effort in collating such statistics for the product of evidence reports of our work which our funders have required in the past. However in light of the recent announcement of the cessation of core-funding for UKOLN, this is no longer a priority! Indeed it is now important to ensure that ideas described in peer-reviewed papers are widely disseminated.

Using ResearchGate

Having recognised the value of hosting PDF copies of my papers on a third-party repository service the question then was which one to select. The key criteria used in the selection were:

• Easy to upload files.
• Popular with readers.
• Resource is easily found using Google.
• PDF files preserved intact.
• Service appears to be viable.

On 25 December 2012 I received an automated email from ResearchGate which informed me that “28 of your colleagues from University of Bath have joined ResearchGate in the last month“. On 24 January 2013 an automated message announced “44 of your colleagues recently joined ResearchGate“. As illustrated the University of Bath”s entry of ResearchGate shows that there are currently researchers from 26 departments who have uploaded a total of 7,263 publications. It seems ResearchGate is growing in popularity, at least at the University of Bath.

On 20 December 2012 I was notified of the numbers of views of my papers (or, more accurately, the numbers of views of the metadata for my papers): “Your published research was viewed 1,678 times in 2012” so perhaps ResearchGate is popular beyond the University of Bath!

In light of the apparent popularity of the service I decided to upload one of my papers to the service: the PDF copy of the paper on “Developing A Holistic Approach For E-Learning Accessibility“.

It was trivial to upload the paper, especially as the associated metadata had been created previously. I then downloaded the PDF and was able to confirm that the metadata was still embedded in the PDF resource.

The paper can be accessed from ResearchGate and the user interface is shown below. I’ll leave others to judge the usability of the service.

Page on ResearchGate for one of my papers

But in addition to users who are linked directly to the paper or access resources on the ResearchGate service using the Web site’s browse and search functionality, what of the discoverability of resources using Google.

ResearchGate, Google and Embedded Metadata

The PDF version of the paper now contains content which will not be widely used elsewhere: a combination of the authors’ names and their ORCID ID. A Google search for “Brian Kelly ORCID: 0000-0001-5875-8744“, “Lawrie Phipps ORCID: 0000-0002-0834-273X” or Elaine Swift ORCID: 0000-0002-6101-6861“ should initially find information about the paper hosted on the UKOLN Web site, the UK Web Focus blog and other services which may be used by the co-authors, although not the institutional repository as this does not currently provide ORCID information (understandably, as ORCID is so new).

I have therefore provided links to the following Google searches which I will monitor to see when Google has indexed the PDFs hosted on ResearchGate:

It appears that over a period of a week the ORCID metadata is being found from citation records hosted on the UKOLN Web site together with the citation records already indexed on the ORCID Web site and this blog, but not yet the PDF files hosted on ResearchGate. Might this be due to Google not indexing the researchgate.net site? In order to answer this question Google was used to provide information on the total number of resources on the service and the total number of PDF files. The results are given below.

* The numbers of search results have fluctuated from 24,100,000 – 55,300,000 during the last few days.

It seems that a large number of PDF files hosted on Researchgate have been indexed by Google, but it takes longer than a week for new resources to be indexed and the results found using a Google search.

Sustainability of the Service

What Does The Evidence Say?

The home page for the service displays a graphic (to users who are not logged in) of the numbers of the service. It seems that 2.4 million users have subscribed. Since there are likely to be researchers, this does appear to be a significant number.

But what else do we know about the service and the company which provides the service? TechCrunch provides a handful of posts about the company together with the following summary:

ResearchGate is the leading social network for scientists. It offers tools and applications for researchers to interact and collaborate. ResearchGate offers a social, crowdsourced platform designed for researchers. The platform provides a global scientific web-based environment in which scientists can interact, exchange knowledge and collaborate with researchers of different fields.

The results of ResearchGate’s new search engine, called ReFind, are not merely based on keywords, but selected in an intelligent way based on semantic, contextual correlations.

In addition the article also provides a graph showing the numbers of users over the past year, based on figures provided by Compete.

As can be seen, the numbers of unique visitors seem to be growing significantly, from 61,640K in December 2011 to 236,170K in December 2012.

I also used MajesticSEO to report on the SEO characteristics of the service (note free subscription required in order to view findings). As can be seen there are 7,459 domains which have links to Researchgate.net and a total of 177,945 backlinks. Although such figures need to be regarded with caution (for example, they can be skewed significantly by link spam) the number of links from educational domains (3,241) and the numbers of educational domains (551) may be more appropriate to measure, due to the difficulties in creating educations domains to host link farms. This snapshot may therefore provide a useful baseline for measuring changes in the link popularity in the service.

Terms and Conditions

It should be noted that looking at the ResearchGate terms and conditions I found no suggestions that the company claims rights to sell my data or my attention data to others (although I haven’t studied the terms and conditions in great detail). Although some may welcome this, others may wonder what the business model for the company is. An article entitled ResearchGate Wants To Be Facebook For Scientists published by Forbes in March 2012 described how:

“ResearchGate will also be looking into ways to monetize its platform. The “no-brainer” way to do that, in Madisch’s words, is to provide job boards for scientists looking for jobs. Universities and companies would pay the site to place listings. The company is also looking for ways to partner with other companies that manufacture and sell biotech lab equipment, as well as several other different programs.”

 Perhaps this is an appropriate business model which will accepted by researchers who normally shy away from free services on the grounds that “If You’re Not Paying for It; You’re the Product“.

Interest in UK HE Sector

Although ResearchGate seems to be growing in popularity globally (and in the University of Bath) is there any evidence of interest with the UK’s higher education community? For me this is not necessarily a significant issue (it can be fine to be an early adopter) but it would be interesting to see what others in my community are saying about the service.

Using a Google search for “researchgate terms and conditions“ I found that the DCC have provided a summary of ResearchGate in its list of resources of digital curators with a similar resource being provided by the University of Edinburgh’s College of Humanities and Social Science. A Google search for “researchgate UK“ finds a number of additional resources from the sector including pages provided by the University of Leeds (PDF format), the University of Leicester, the University of Liverpool (PDF format) and the University of Gloucester together with blog posts at the University of Loughborough and the University of Warwick.

My Decision

In light of these figures and my experiences in using the service I am happy to use the service to provide additional exposure to my research papers which complements the master copy of papers which are hosted on my institutional repository. Are other researchers making similar decisions or are alternative services felt to provide better options?

View Twitter conversation from: [Topsy] | View Twitter statistics from: [TweetReach] – [Bit.ly]

The day after announcing a post on Reflections on the Discussion on the Quality of Embedded Metadata in PDFs I received a tweet from @community which alerted me to a blog post on SEO Action for PDF files on the Adobe blog. The post describes an extension for use in Acrobat X Pro which automates the settings of the properties of the PDF file in accordance with guidelines which can enhance the discoverability of PDF files by Google. The guidelines, which had been published way back in August 2009, were based on experiments which demonstrated improvements in Google’s indexing of PDF files. The article’s main conclusion was that “Every PDF needs a title“:

In terms of PDF files, the blue underlined text in Google’s search results comes from one of two places. First, Google looks in the “Title” document information field. If it finds nothing, Google’s indexer tries to guess the document’s title by scanning the text on the first few pages. This usually doesn’t work, producing incorrect and improperly formatted results.

In addition to this advice, the article also suggested use of other metadata fields including author, subjects and keywords.

Metadata For Peer-Reviewed Papers

Although I ensure that I provide the correct title for my peer-reviewed papers when I create them in MS Word I was unsure whether I included the names of the co-authors or made use of other metadata fields.

On Friday 25 January 2013 I decided to update the metadata for one of my papers, ”Developing A Holistic Approach For E-Learning Accessibility” which was the first paper myself, Lawrie Phipps and Elaine Swift wrote back in 2004

I added a number of tags to the paper and used the Comments field to provide the abstract. In addition the publication details were added to the Status field.

Whilst updating the metadata it occurred to me that it would be useful to include the ORCID ID for the authors as this will be less volatile than the author’s email address (one of the co-authors was based at the University of Bath when the paper was published but subsequently moved to Nottingham Trent University).

In addition to the resource discovery metadata for the paper I also remembered that I should ensure that images in the paper contained appropriate alt text so that image descriptions are available to those who may make use of a screen reader. Fortunately we had done this for the paper, but I have to admit that this isn’t necessarily done for all of my research papers.

Having updated the metadata for the paper and embedded images I then created the PDF from MS Word. I noticed that the Save As PDF option in MS Word enabled a number of options to be specified, including Save As ISO-19005 (PDF/A).

As described in Wikipedia PDF/A is “an ISO-standardized version of the Portable Document Format (PDF) specialized for the digital preservation of electronic documents“. The articles goes on to explain that “PDF/A differs from PDF by omitting features ill-suited to long-term archiving, such as font linking (as opposed to font embedding)“.

Since the digital preservation of peer-reviewed publications is important I ensured that I saved the paper in PDF/A format, using the Save As option illustrated.

Approaches to Embedded Metadata Embedded in PDFs

What practices should be used in providing the metadata to be created in the original authoring tool (MS Word, in my case) which will then be available in the PDF version of the paper? Here’s a summary of the approaches I have used:

Title: The title of the paper

Tags: My preferred tags about the content and my organisation.

Comments: The abstract of the paper, normally taken from the abstract provided in the paper.

Author: First Name Surname (ORCID: ORCID ID) e.g. Brian Kelly (ORCID: 0000-0001-5875-8744)

The title field will be obvious. The tags will reflect keywords which I feel will enhance access to the document (and I choose less than five). I am using the comments field to host the abstract for the paper. Finally the author field contains the full name followed by ORCID: ORCID ID (in brackets). I feel that this is a pragmatic approach to ensuring that the significant information which will be indexed by Google is found in the metadata fields which are available through my authoring tool (MS Word).

But could this cause problems? Might Google think my name is Mr Orcid or Mr 0000-0001-5875-8744? Might other indexing and aggregation tools have problems as I am misusing the semantics of these metadata tools? My feeling is that Google will be capable of understanding the content and it is better to have such quality metadata (which I have chosen) rather than no metadata. But are other researchers embedding ORCID IDs in PDFs? In order to answer this question I have using Google’s advanced search capability to search for “ORCID” in PDF resources across a number of domains, as summarised below.

These numbers are low – and when you realise that the results include PDFs which contain the string “ORCID” in the text of the pages (as illustrated) it seems clear that there is little evidence that ORCID IDs are being embedded in PDFs yet.

So before I embed ORCID IDs in my other papers I would welcome feedback on this proposal. Is it desirable to include the ORCID IDs of authors in the PDF versions of papers? If so, is the approach I have taken to be recommended to others? Or might it be desirable to provided richer structured metadata in PDF files, using the XMP (Extensible Metadata Platform) standard? But if this is felt to be desirable, how would it fit into the workflow, given that it appears difficult to persuade authors to provide metadata for their papers in any case?

View Twitter conversation from: [Topsy] | View Twitter statistics from: [TweetRearch] – [Bit.ly]

Aaron Swartz, a celebrated internet freedom activist and early developer of the website Reddit, has died at 26.

The activist and programmer took his life in his New York apartment, a relative and the state medical examiner said. His body was found on Friday.

A sad day, especially for those who share Aaron Swartz’s commitment to openness and admire his commitment to the development of tools, services and standards, such as RSS, which have helped to make open access to resources accessible on a global basis.

Earlier today I came across a tweet which encouraged academics to show their support for Aaron’s work:

Please share: Academics posting their papers online in tribute to Aaron Swartz using hashtag #pdftribute.

I would like to endorse this proposal. I have created a Storify summary of the #pdftribute tweets, which contains over 500 posts since the call was made just over 3 hours ago.

Although we have see that initial tweet being widely retweets, as @neuroconscience (Micah Allen) has suggested:

Folks as exciting as #pdftribute is we need less links talking about it and more actual paper posting.

But what could be said in 140 characters?

Within my Twitter stream I have already seen tweets from those involved in supporting their institutional repository including @SarahNicholas:

Cardiff academics! Post your articles to @CardiffOrca. #openaccess#pdftribute

and @glamlaflib (Sue House):

Glamorgan academics can deposit their articles & papers here (if you retained the copyright) http://dspace1.isd.glam.ac.uk/dspace/ #pdftribute

I have also seen @openscience endorsing @jambina’s reminder of the role which can be played by librarians:

Librarians: always friends in #openaccess, #openscience MT @jambina: librarians can help you free your work. we are on your side #pdftribute

Meanwhile @MrGunn describes services which can be used:

@opendna @venturejessica @Aine Mendeley can push into to local repository via Symplectics Elements, other routes can be made with Open API.

Of course many researchers are demonstrating their commitment to providing open access to their research papers:

whilst others provide a more political view:

@MarietjeD66 @mikebutcher Let this be the start of the end of the ridiculous copyright laws. #pdftribute #AaronSwarz

Would you like to join in by giving your views or ensuring that your Twitter community is aware of how you have made your research papers openly available?

Note archives of the #pdftribute tweets are available at http://pdftribute.net and http://twubs.com/pdftribute

View Twitter conversation from: [Topsy] | View Twitter statistics from: [TweetReach] – [Bit.ly]

The recent post on Embedded Metadata in PDFs Hosted in Institutional Repositories: An Inside-Out & Outside-In View generated a fair amount of discussion, with ~17 comments on the post itself but perhaps more significantly, a more interactive discussion on Twitter, with relevant contributions being made by @mrnick, @neilstewart, @rmounce, @carusb, @pj_webster, @emmatonkin, @MikeTaylor and @wrap_ed, with other Twitter users sharing links to the posts to their communities.

Whilst some people may still feel that discussions should take place on one centralised system (e.g. a mailing list) in reality this is an unrealistic expectation. In the real world discussions based on ideas which may have originated online will be dispersed across office and common rooms in institutions around the world, to say nothing of other discussions which may take place in pubs and coffee rooms as well as whilst travelling. Conversations about interesting ideas will be distributed; we have to accept that. However it can be helpful to aggregate valuable comments which may be fragmented across a variety of communication channels. Since I felt that the Twitter discussions about the post were particularly interesting I have created a Storify summary entitled The Quality of Embedded Metadata in PDFs (Jan 2013). Note that this complements the Topsy summary which gives the tweets which contains links to the blog post.

Note that in the comments on the blog post Nick Sheppard suggested that a forthcoming UK RepNet event might provide an opportunity to discuss the issues which were raised in more depth::

I wonder if some of these issues might be relevant within the context of the UK RepNet project which is holding a meeting in London on 21st Jan –http://www.rsp.ac.uk/events/supporting-and-enhancing-your-repository/

I will therefore provide a summary of the main issues which were discussed on the blog and on Twitter.

The Context

The initial post was written in response to a post by Ross Mounce in which he asked PDF metadata – why so poor? and a follow-up post a week late on PDF metadata: different tool, same story. Ross’s post was based on an analysis of the metadata embedded in PDFs hosted by scholarly publishers. Ross’s second post succinctly summarised his work:

So a week ago, I investigated publisher-produced Version of Record PDFs with pdfinfo and the results were very disappointing. Lots of missing metadata was found and one could not reliably identify most of these PDFs from metadata alone, let alone extract particular fields of interest.

I wondered whether PDFs hosted in institutional repositories also suffered from poor quality or missing embedded metadata. I examined some papers I had deposited in the University of Bath repository and found that metadata which was contained in the original PDF file I uploaded to the repository was missing from the PDF which users can download. I surmised that the metadata had been lost in the workflow when a cover sheet was added to the paper.

My post referenced a post by Lorcan Dempsey entitled Discovery vs discoverability … in which he explored the idea of the “inside-out and outside-in library“. This seemed very relevant to this scenario as both Ross and myself were concerned primarily by the implications is missing metadata for systems which may be used outside of the repository context: in Ross’s case this related to text mining of large collections of PDFs whereas my interest focussed on reuse of PDFs in other repositories.

The Discussion

The initial comment on the blog post by Ingmar Koch illustrated how embedded PDF metadata can be (mis-)used by external systems. Ingmar descried how “the company that designed the document templates for most of the government agencies added a title and author in the template-file. The result is that thousands of online government documents (.pdf and .doc) are titled “at opinio facillime sumitur” and are written bij M. Hes.” This example provides a vivid illustration of how metadata embedded in PDFs is being used by Google. However this example might also be used to demonstrate the poor quality of embedded metadata.

In light of such examples Neil Stewart therefore asked “does it matter if the rare and patchy instances of author-created metadata gets over-written or otherwise distorted?” since “the structured metadata provided at Eprint/DSpace/other repository software record level does the job here (as opposed to metadata embedded within the PDF itself).“

But surely we cannot argue that since some resources may contain poor quality metadata we should delete all metadata! I would argue that there is a need to educate authors on the importance of appropriate metadata, which includes showing how such metadata can be used by services outside of the host institution. Neil recognises the validity of this point when he acknowledged that “not every service will use OAI-PMH or web crawling, some might parse the objects themselves“.

The discussion then moved on Twitter and initially addressed the relevance of cover sheets, since these appear to cause problems in work flows which take place outside of the institutional repository.

Ross Mounce asked:

why do IRs need 2 slap on cover page anyway? Perhaps they should just embed additional provenance metadata @briankelly @mrnick @neilstewart

Neil Stewart provided one use case for cover sheets:

@rmounce @briankelly @mrnick viewed as a way of advertising provenance (proper citation), branding as from home inst but agreed!

However Ross re-iterated his criticisms of cover sheets:

Cover-pages from a user-POV r just plain annoying. If provenance must be visibly embedded why not overlay? @neilstewart @briankelly @mrnick

Others, such as Chris Rusbridge, agreed with this view:

@mrnick @ukcorr @rmounce @briankelly @stevehit I agree with Ross that it’s BAD practice, from my POV

The discussion then moved on to problems which may occur if a paper is to be downloaded, with Nick Sheppard provided a good example of how PDFs may end up containing multiple cover sheets if they are taken from one repository and deposited (by, for example, a co-author) in another repository:

@neilstewart Um, can also lead to cover page disasters like this (scroll down) eprints.port.ac.uk/2278/1/A…@rmounce @briankelly

I then highlighted a paper by my colleague Emma Tonkin which showed that that problems with poor quality metadata went beyond the individual examples provided on Twitter:

@carusb @mrnick @rmounce My colleague @emmatonkin analysed PDF metadata a few years ago: opus.bath.ac.uk/24958/

The paper (PDF format) described how:

Many repositories … have developed or identified a means of adding a cover sheet to each document within the repository. This has potential for positive impact, for example, as a means of clearly indicating the provenance of an item (Puplett, 2008). As can be seen in Fig. 7, Google Scholar does not necessarily recognise the cover sheet for what it is, and this has negative implications for effective indexing and retrieval.

and went on to conclude:

However, the addition of a cover sheet has caused a number of issues beyond those that are usually encountered with the PDF format (ie. font problems, file corruption, etc). This limits the ability for automated processes to make use of this information, and could therefore be said on the level of automated indexing and other software access (such as conversion) to be a retrograde step. If this becomes common practice it may be necessary to review both the assumptions under which automated systems are developed, and perhaps the rationale that lead us to make use of cover sheets in this context.

Conclusions

The paper on Supporting PDF accessibility evaluation: early results from the FixRep project was written in 2010 by my colleagues Emma Tonkin and Andy Hewitt and presented at the 2nd Qualitative and Quantitative Methods in Libraries International Conference (QQML2010).The concluding sentence in the paper highlighted work which needs to be addressed:

it may be necessary to review both the assumptions under which automated systems are developed, and perhaps the rationale that lead us to make use of cover sheets in this context

The paper identified the benefits of cover sheets but also the problems they can cause for automated activities which may take place outside of the institutional repository environment.

But should repository managers and developers necessarily devote resources to addressing potential problems which may arise downstream of the repository environment? In a comment on Ross Mounce’s blog the point was made that publishers will need there to be a sound business case to be made:

“Why would publishers add metadata? Because their customers – libraries, governments, research funders (in the case of Open Access PDFs ) should demand it.” I’m not seeing a compelling business case here. High-quality metadata would be nice, but can anybody argue that their research is being hampered by a lack of such metadata? Could someone working in publishing make a case to their boss that adding such metadata would generate more revenue, web traffic, manuscript submissions (insert whatever metric matters)?

In the context of institutional repositories perhaps the approach to be taken would be to ensure that embedded metadata is preserved and that the training and advice provided by repository support staff ensures that authors are made aware of the ways in why embedded metadata can be used, even if such reuse takes place outside of the institutional repository.

The discussion also highlighted the need for enhanced workflow practices for merging cover pages with the original content and also for enabling users (and automated tools) to be able to access the original source paper in addition to the version contained provenance information designed for consumption by users.

Do any institutional repositories currently provide solutions to these requirements? In addition, I am interested in how many institutional repositories provide cover pages and whether those that do use a repository plugin technology to do this, some other automated technologies or by manual processes. Two polls on these questions are embedded in this post but if the situation is more complex than can be summarised in the poll, feel free to add a comment.

Footnote (added 12 January 2012): A tweet from @community alerted me to the blog post on SEO Action for PDF files on the Adobe blog. This describes an “Action” for use in Acrobat X Pro that will automate setting the properties of the PDF file in accordance with guidelines which can enhance the discoverability of PDF files by Google.

View Twitter conversation from: [Topsy]  |  View Twitter statistics from: [TweetReach] – [bit.ly]

PDF metadata – why so poor? asked Ross Mounce in a blog post published on New Year’s eve.

In the post Ross expressed surprise that although ”with published MP3 files of audio you get rather good metadata … the results from a little preliminary survey of academic publisher PDF metadata” were poor: “Out of the 70 PDFs I’ve published (meta)data on over at Figshare, only 8 of them had Keywords metadata embedded in them“.

This made we wonder about the quality of the metadata for papers I have uploaded to Opus, the University of Bath repository.

I looked at a paper on A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First which is available in Opus in PDF and MS Word formats.

I first used Adobe Acrobat in order to display the metadata for the original source PDF file, prior to uploading to the repository. As can be seen from the accompanying screen shot the metadata included the title, the author details (with the email address for one of the authors) and two keywords.

However looking at the display for the PDF downloaded form the repository we find that no metadata is available!

This PDF differs from the original source in that a cover page is added dynamically by the repository in order to provide appropriate institutional branding. It would appear that in the creation of the new PDF resource, the original metadata is lost.

Looking at the metadata created in the original source document – an MS Word file – we can see how the authors’ names which were subsequently concatenated into a single field. We can also see that although the title of the paper was given correctly, poor keywords had been included, which did not reflect the keywords which were included in the paper itself (Web accessibility, disabled people, policy, user experience, social inclusion, guidelines, development lifecycle, procurement).

I suspect that I am not alone in not spending much time in ensuring that appropriate metadata is embedded in the master source of a peer-reviewed paper. I have also previously not considered how such metadata might be lost in the workflow processes when uploading to an institutional repository: after all, surely the important metadata is added when the paper is deposited into the repository?

Ross’s blog post made me check the embedded metadata – and I discovered that the correct metadata is still included in the MS Word file which was uploaded to the repository along with the PDF copy.

Does the loss of the metadata embedded in the PDF matter? After all, surely people will use the search facilities provided in the repository in order to find papers of interest?

But people will not necessarily visit a repository to find papers of interest. A post which described A Survey of Use of Researcher Profiling Services Across the 24 Russell Group Universities showed that on 1 August 2012 there were over 18,000 users of ResearchGate in the 24 Russell Group universities and judging by the messages along the lines of “28 of your colleagues from University of Bath have joined ResearchGate in the last month. Why not follow them today?” which I am currently receiving, use of this service is growing.

As can be seen from the screenshot of my ResearchGate profile, the service provides access to PDF copies of my papers. I normally simply provide a link to the PDF hosted in the repository but the example illustrated contains a copy of original PDF which was uploaded to the service by one of the co-authors.

In the case of most of my papers it is clear from the thumbnail of the PDF that the paper contains the coversheet provided by the repository.

Discussion

We can see that the PDF copy of a paper hosted in a repository should not be regarded as a final destination; rather the PDF may be surfaced in other environments.

It will therefore be important to ensure that workflow processes do not degrade the quality of the PDF. It will also be important to ensure that authors are made aware of how embedded metadata may be used by services beyond the institutional repository. But to what extend do repository managers feel they have a responsibility to advise on practices which will enhance the discoverability of content on services hosted outside the institution?

In a paper which asked “Can LinkedIn and Academia.edu Enhance Access to Open Repositories?” myself and Jenny Delasalle commented on how “commercial publishers are encouraging authors to use social media to drive traffic to papers hosted on publishers’ web sites” and provided examples of such approaches from Taylor and Francis, Springer, Sage and Oxford Journals. As an example, Taylor and Francis describe how they are “committed to promoting and increasing the visibility of your article and would like to work with you to promote your paper to potential readers” and go on to document services which can help achieve this goal.

In a blog post which discussed the ideas describe din the paper I described how we had failed to find significant evidence of similar approaches being employed by repository managers:

It was interesting that in Jenny’s research she found that a number of commercial publishers encourage their authors to use services such as LinkedIn and Academia.edu to link to their papers hosted behind the publishers paywalls – and yet we are not seeing institutional views of the benefits of coordinated use of such services by their researchers. Institutional repository managers, research support staff and librarians could be prompting their institutions to make the most of these externally provided services, to enhance the visibility of their researchers’ work in institutional repositories.

But that paper was limited to use of third-party services to provide access routes to research papers. What of the bigger picture in which institutional work flow processes should be designed to enhance discoverability?

The ‘inside-out and outside-in library’

On Wednesday in a post entitled Discovery vs discoverability … Lorcan Dempsey explored the idea of the “inside-out and outside-in library“. In the post Lorcan described how:

Throughout much of their existence, libraries have managed an outside-in range of resources: they have acquired books, journals, databases, and other materials from external sources and provided discovery systems for their local constituency over what they own or license.

However in a digital and network world, there have been two major changes, which shift the focus towards inside-out:

First access and discovery have now scaled to the level of the network: they are web scale. If I want to know if a particular book exists I may look in Google Book Search or in Amazon, or in a social reading site, in a library aggregation like Worldcat, and so on. … Secondly the institution is also a producer of a range of information resources: digitized images or special collections, learning and research materials, research data, administrative records (website, prospectuses, etc.), faculty expertise and profile data, and so on.

Lorcan goes on to describe the challenge facing libraries:

How effectively to disclose this material is of growing interest across libraries or across the institutions of which the library is a part. This presents an inside-out challenge, as here the library wants the material to be discovered by their own constituency but usually also by a general web population.

I would suggest that institutional repositories could usefully adopt the approach taken by Taylor and Francis:

 ”[The institution is] committed to promoting and increasing the visibility of your article and would like to work with you to promote your paper to potential readers“

But rather than simply encourage researchers to simply add links to papers deposited in the repository from popular services such as LinkedIn and ResearchGate might the institutional goal be enhanced by encouraging researchers to make the content of their papers available in such third party services (subject to copyright considerations) – with the institutional repository providing both a destination and a component in a workflow, with papers being surfaced in services such as ResearchGate, as I have illustrated above.

If such an approach were to be embraced there would be a need to ensure that embedded metadata was not corrupted through repository workflow processes. If, however, the repository is regarded as the sole access point, there would be little motivation to address such limitations in the work flow.

Or to put it another way, repository managers will have a need to manage content hosted within the institution, including management to support the use of the content by services they have no control over.

To a certain extent, this has already been accepted: repositories were designed to have “cool URIs” which can help resources to be discovered by Google. I am suggesting that there is a need to observe usage patterns which indicate emerging ways in which users are finding content. The growing numbers of email alerts from ResearchGate suggest that it may be a service to monitor – with Ross Mounce’s recent post of on the quality of metadata embedded in PDFs suggesting one area in which there will be a need to revisit existing workflow processes.

PS. Ross Mounce described “a little preliminary survey of academic publisher PDF metadata” and has published the data on Figshare. Has anyone harvested the metadata embedded in PDFs hosted on repositories and published the findings?

View Twitter conversation from: [Topsy]

Yesterday I received an email which informed me that contribution to the Jisc Inform online newsletter (issue 35, December 2012) had been published. The article on Top 10 Tips on How to Make Your Open Access Research Visible Online is based on a blog post originally published on the Networked Researcher blog which was tweaked slightly and republished on the Jisc blog. The version published in the Jisc Inform newsletter includes a series of images to accompany each of the ten tips.

The tips were originally developed to accompany a series of presentations given at the universities of Exeter, Salford and Bath during Open Access Week. These presentations were based on the experiences gained in use of social media to help maximise access to peer-reviewed publications. In particular the tips documented the experiences of use of social media services such as blogs, Twitter and Slideshare to help maximise the readership of a paper entitled “A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First“.

The Complexities Behind the Tips

It is interesting to see how the advice initially given in a one-hour seminar can be distilled into a series on top tips. The sceptic may be dismissive of the value of reducing the complexities of open practices for researchers to a series of top tips. However at the recent SpotOn 2012 conference in sessions such as How to do Smart Journalism on Complex Science the value of science writers in being able to communicate complex scientific ideas in ways which can be understood by the general public was emphasised. The challenges, however, was to ensure that those with a deeper interest in the complexities can be able to access resources which provide more in-depth discussions.

In the case of the Top 10 Tips on How to Make Your Open Access Research Visible Online more detailed information was provided in the slides of the original talk. In addition, as illustrated the slides also contain links to further information. In the example shown evidence that being proactive in ensuring that the co-authors of the paper provided links to the presentation on their blog posts and Twitter channels can be seen from the large numbers of views of the slides during the week of the conference.

The limitations of Slideshare statistics was mentioned, but the slide also contain a link to the usage statistics which showed how the accompanying paper was, at the time, the most downloaded of my peer-reviewed papers which had been deposited in the University of Bath repository this year.

In addition to the more detailed information provided in the slides during the presentation itself I expanded on a number of issues, including responding to questions raised during the talk. A post has been published on the JISC-funded Open Exeter blog about the Open Access Week @ Exeter which includes a series of videos of the invited presentations. The video of my talk is available on YouTube and embedded below. I hope this additional information complements the top 10 tips published in Jisc Inform.

View Twitter conversation from: [Topsy]

As a follow-up to those posts, which were motivated by a commitment to openness and sharing which is prevalent in the repository community, this post by Catherine Dockerty (Web and Data Services Manager, Radcliffe Science Library) and Juliet Ralph (Bodleian Libraries Life Sciences Librarian) provides a summary of the activities behind the Open Access Week event at the University of Oxford.

Open Access Week at Oxford

Open Access Week 2012 saw a determined effort from the Bodleian Libraries of Oxford University to shine a light on developments in Open Access with a full week-long programme of events. This was prompted by the need to assess the state of play in Open Access (OA) which, for major research institutions such as Oxford, is particularly urgent in the wake of the publication of the Finch Report. It was the second year we have participated in Open Access Week – last year we held a single event and we wanted to do a lot more this time round.

What We Were Trying To Do

We had a number of specific things we wanted to achieve though our programme:

• Increasing the knowledge of library staff. All reader-facing staff will potentially deal with enquiries relating to Open Access.
• Assembling and showcasing the expertise of Bodleian Libraries staff in Open Access. Readers need to know what we can do for them.
• Raising awareness of publishing options to academic researchers.
• Promoting submission to Oxford’s institutional repository ORA (Oxford Research Archive). Oxford currently has mandatory deposit for doctoral theses, but not for research papers.
• Highlighting Oxford’s progress in the field of Open Data.

What We Did

We put together a programme of talks and other activities, most of which were lunchtime sessions and took place at the Radcliffe Science Library, one of the Bodleian Libraries and Oxford University’s main library for the sciences and engineering. The majority of speakers were library staff. The focus was on science, but events covering law and medicine were included and there were attendees from the humanities and social sciences.

An evening session, “Bodley’s ‘Republic of [Open] Letters” was hosted by the Oxford Open Science Group and highlighted the DaMaRO Project, which is developing a research data management policy and data archiving infrastructure for Oxford

The presentations are available online.

Wikipedia Editathon

Ada Lovelace by Margaret Carpenter, 1836

The final event of the Open Access Week programme was a Wikipedia “Editathon” on the theme Women in Science. The event was organised as a collaboration between the Bodleian Libraries and Oxford University’s IT Services, and was a follow-up to the Ada Lovelace Day event at the Royal Society the week earlier. This tied in neatly with Open Access Week as we were able to highlight open access sources for use in updating articles. Our event was publicised at the Royal Society one and on Ada Lovelace Day Wikipedia page.

Having an Oxford-based Wikipedia event was also an opportunity to encourage academics and students to get involved in editing Wikipedia, which is reliant on expert contributors to add high quality articles and improve existing ones. Wikipedia has a readership vastly exceeding that of any academic journal, and presents an opportunity for academics to have an impact on a wider audience.

Juliet Ralph (Bodleian Libraries Life Sciences Librarian) kicked off the proceedings with an introductory talk to introduce Wikipedia and outline the format of the session. Online resources for editing articles were suggested, focusing on open access. The fact that the Royal Society was providing free access to all its publications until 29^th November 2012 was highlighted. A collection of printed reference materials from the RSL’s collection was also provided.

A list of articles for adding/updating was provided as guidance to participants, but this was not intended to be prescriptive. The list was the same one as used at the Royal Society event, updated to reflect all the work done that day.

We were very pleased that Oxford-based Wikipedians James and Harry Burt were able to attend and assist the assembled editors. They also treated us to an impromptu presentation on their work as long-time Wikipedia editors.

Online participation via Twitter was encouraged using the hashtag #WomenSciWP (the same as for the Royal Society event). Note that a Twubs archive of the tweets is available. The event was also live-tweeted from the RSL’s Twitter feed (@radcliffescilib).

By the end of the session two new articles were created and 12 updated. Attendees were mainly research staff and postgraduate students from the fields of science and medicine. Also present were two archivists from the Saving Oxford Medicine project who posted a blog post about the work.

Special thanks to:

• James and Harry Burt for presenting and for help they gave to other participants.
• Izzie McMann and Karen Langdon (Radcliffe Science Library staff) for assisting participants on the day.
• Janet McKnight (IT Services) and Alison Prince (Bodleian Libraries Web Manager) for help in organising and publicising the event.
• Andrew Gray (British Library Wikipedian in Residence) and Daria Cybulska (Wikimedia UK) for publicising the Editathon and supplying learning materials for the session.

Reflections

We certainly achieved the aim of increasing the knowledge of OA issues in Library staff within the sciences, several of whom attended more than one event. In future we will aim to actively promote the staff development benefits from participating to all Bodleian Libraries staff, not just those in the sciences. Our collaborations with the Open Science Group and IT Services were successful, and we hope to work together with them on future events.

We fulfilled all our original intentions to some extent, but some events were not well attended in spite of being publicised widely although were positively received by those who did.

The timing of Open Access Week is a problem for Oxford as the start of the academic year is later than for most UK universities, which means the new term is just getting underway in earnest and there are many other events to compete with. Staff time in planning events is also in short supply as reader-facing staff will have been prioritising inductions for new students over the previous weeks.

The Wikipedia event was a success (well attended with positive feedback) and we would certainly hold a similar event in the future, although not necessarily as part of Open Access Week. The fact that it was a hands-on session went down well, and the Women in Science theme attracted interest.

Next Time

Holding events at lunchtime was evidently not popular and we may decide to move them to an afternoon slot (colleagues who run user education programmes had a higher take-up when they did this). We may also move the sessions out of the library into academic departments or colleges, and hold events at other times of year.

We will be making a concerted effort to involve well-known speakers, rather than relying heavily on library staff.

We will be looking to encourage other OA events in Oxford and elsewhere, and we will also think about using online chat as well as Twitter for online participation. The planning starts now!

View Twitter conversation from: [Topsy]

Catherine Dockerty is the Web and Data Services Manager at the Radcliffe Science Library at Oxford University where her role is managing online content, social media and communications, and to support colleagues in serving the University’s teaching and research in the sciences. She has spent 13 years working in various reader services roles at Oxford University, and has also worked in the civil engineering industry and the book trade.

Juliet Ralph is the Subject Librarian for Life Sciences and Medicine in the Bodleian Libraries at Oxford, where she has worked for over 15 years. She is one of many librarians involved in providing support for research at Oxford, including Open Access.

Open Access Week, which took place last week, was a busy period for me. Not only did I give talks on how social media can enhance access to research papers hosted in institutional repositories at the universities of Exeter, Salford and Bath, I also wrote accompanying posts which were published on the Networked Researcher and JISC blogs. But perhaps more importantly last week I coordinated the publication of three guest posts on this blog: SEO Analysis of WRAP, the Warwick University Repository, SEO Analysis of LSE Research Online and SEO Analysis of Enlighten, the University of Glasgow Institutional Repository.

Sharing of Repository Practices and Experiences

The background to this work were the two papers I co-authored for the Open Repositories OR 2012 conference. In the paper on “Open Metrics for Open Repositories” (available in PDF and MS Word formats) myself, Nick Sheppard, Jenny Delasalle, Mark Dewey, Owen Stephens,Gareth Johnson and Stephanie Taylor conclude with a call for repository managers, developers and policy makers to be pro-active in providing open access to metrics for open repositories. In the paper which asked “Can LinkedIn and Academia.edu Enhance Access to Open Repositories?“, also available in PDF and MS Word formats, Jenny Delasalle and myself described how popular social media services which are widely use by researchers can have a role to play in  enhancing the visibility of papers hosted in repositories. However although LinkedIn and Academia.edu appeared to be widely used we  concluded by described how “further work is planned to investigate whether such links are responsible for enhancing SEO rankings of resources hosted in institutional repositories“.

This work began with a post which described the findings of a MajesticSEO Analysis of Russell Group University Repositories. This post made use of the MajesticSEO service which can report on SEO ranking factors for Web sites. The survey provided initial findings of a survey of institutional repositories hosted by the 24 Russell Group universities.

This initial post was intended to explore the capabilities of the tool and gauge the level of interest in further work.  In response to the post the question was asked “Are [the findings] correlated with amount of content, amount of full-text (or other non-metadata-only) content, breadth or depth of subject matter, what?” These were valid questions and were addressed in the more detailed follow-up surveys, which were provided by repository managers at the universities of Warwick, Glasgow and LSE who have the contextual knowledge needed to provide answers to such questions.

In this initial series of guest blog posts, William Nixon concluded with the remarks:

This has been an interesting, challenging and thought-provoking exercise with the opportunity to look at the results and experiences of Warwick and the LSE who, like us reflect the use of Google Analytics to provide measures of traffic and usage.

The overall results from this work provide some interesting counterpoints and data to the results which we get from both Google Analytics and IRStats. These will need further analysis as we explore how Majestic SEO could be part of the repository altmetrics toolbox and how we can leverage its data to enhance access our research.

I feel the exercise has been valuable for the three contributors. But I also feel that the descriptions of the experiences in using the MajesticSEO tool, the findings and the interpretation of the findings in an open fashion will be of valuable to the wider repository community, who may also have an interest in gaining a better understanding of the ways in which repository resources are found by users of popular search engines, such as Google.  There will also be a need to have a better understanding of the tools used to carry out such analyses. How, for example, will SEO analysis tools address link farms and other ‘black hat’ SEO techniques which may provide significant volumes of links to resources which may, in reality, be ignored by Google?

William Nixon’s post concluded by pointing out the need for:

further analysis as we explore how Majestic SEO could be part of the repository altmetrics toolbox and how we can leverage its data to enhance access our research.

I suspect the University of Glasgow will not be alone in wishing to explore the potential of SEO analysis tools which can help in understanding current patterns of traffic to repositories and in shaping practices to enhance such traffic. I hope the work which has been described by Yvonne Budeden, Natalia Madjarevic and William Nixon has been useful to the repository community in summarising their initial experiences.

I should also add that Jenny Delasaale and I are giving a talk at the ILI 2012 conference which will ask “What Does The Evidence Tell Us About Institutional Repositories?” We are currently finalising the slides for the talk, which are available on Slideshare and embedded below. There is still an opportunity for us to update the slides, which might include a summary of plans for future work in this area. So we would very much welcome your feedback and suggestions. Perhaps you might be willing to publish a guest post on this blog which builds on last week’s posts?

The second in the series of guest blog posts which gives a summary of an SEO analysis of a repository hosted at a Russell Group university is provided by Natalia Madjarevic, the LSE Research Online Manager. As described in the initial post, the aim of this work is to enable repository managers to openly share their experiences in use of MajesticSEO, a freely-available SEO analysis tool to analyse their institutional repositories.

SEO Analysis of LSE Research Online

This post takes an in-depth look at a search engine optimisation (SEO) analysis of LSE Research Online, the institutional repository of LSE research outputs. This builds on Brian Kelly’s post published on this blog in August 2012 on MajesticSEO Analysis of Russell Group University Repositories.

The London School of Economics and Political Science

Background

LSE is a specialist university with an international intake and a global reach. Its research and teaching span the full breadth of the social sciences, from economics, politics and law to sociology, anthropology, accounting and finance. Founded in 1895 by Beatrice and Sidney Webb, the School has a reputation for academic excellence. The School has around 9,300 full time students from 145 countries and a staff of just under 3,000, with about 45 per cent drawn from countries outside the UK. In 2008, the RAE found that LSE has the highest percentage of world-leading research of any university in the country, topping or coming close to the top of a number of rankings of research excellence. LSE came top nationally by grade point average in Economics, Law, Social Policy and European Studies and 68% of the submitted research outputs were ranked 3* or 4*.

LSE Research Online – a short history

LSE Research Online (LSERO) was set up in 2005 as part of the SHERPA-LEAP project. The aim of the project was to create EPrints repositories for each of the seven partner institutions, of which LSE was one, and to populate those repositories with full-text research papers. In June 2008 the LSE Academic Board agreed that records for all LSE research outputs would be entered into LSE Research Online. We have no full-text mandate but authors are encouraged to provide full-text deposits of journal articles in pre-publication form, clearly labelled as such, alongside references to publications. Research outputs included in LSE Research Online appear in LSE Experts profiles automatically, thereby reusing data collected by LSE Research Online.

LSE Research Online is to be the main source of bibliographic information for the Research Excellence Framework (REF) in 2014. This has served to further increase the impetus for deposit and visibility of the repository in the School and we have various repository champions throughout the School across departments.

LSE Research Online size and a brief look at usage statistics

As of September 2012, LSE Research Online contains around 33,696 records, with 7,050 full-text items. We include a variety of item types such as articles, book chapters, working papers, data sets, blogs and conference proceedings. We most recently began collecting LSE blogs to create a permanent home for this important content. We began tracking LSERO site usage with Google Analytics in 2007 and the site has received 2,268,135 visits since this date. According to Google Analytics, 76.55% (1,748,725 total visits) of traffic to LSE Research Online comes from searches. Only 16.13% of traffic is from referrals and 7.14% from direct traffic. We also use analog server statistics to monitor downloads and total downloads May 2007-Sept 2012 was 5,266,871.

Expectations of the survey

Before running the Majestic SEO report, I expected we would see plenty of traffic from Google and backlinks (i.e. incoming links) from lse.ac.uk as, understandably, these are key sources of traffic to LSERO and are indicated as such on Google Analytics. Google Analytics also points to referrals from Wikipedia and Google Scholar, and most recently, our Summon implementation which includes LSERO content. However, I was intrigued as to how LSERO would fare in an SEO analysis.

Majestic SEO survey results

The data was generated from Majestic SEO using a free account on 24^th September 2012 using the ‘fresh’ index option. A summary of the results is shown below: there are 1,285 referring domains and 8,856 external backlinks. Note that the current findings can be viewed if you have a MajesticSEO account (which is free to obtain).

Figure 1: Majestic SEO analysis summary for eprints.lse.ac.uk

This includes 408 educational referring backlinks. If we look at backlinks in more detail, patterns begin to unravel:

Figure 2: Top 5 Backlinks

This illustrates a distinct majority of Wikipedia pages linking to LSERO content and yet this is only ranked as the sixth most popular source of traffic in Google Analytics.

Top referring domains, sorted by matched links, can be found in the table shown below:

Table 1: Top 5 Referring Domains

Flickr makes a surprise appearance, with WordPress and Blogger dominating the top of the table.

Top 5 items sorted by Majestic’s flow metrics can be found here:

Figure 3: Top 5 Resources in Repository (sorted by flow metrics)

Perhaps more indicative, the Top 5 linked resources sorted by number of backlinks can be found in the table shown below:

Table 2: Top 5 Linked Resources in Repository (sorted by no. of links)

These pages are:

1. The LSE Research Online homepage.
2. A PDF of a research paper on climate policy.
3. The record for a paper on teenager’s use of social networking sites.
4. The record for a paper on climate policy.
5. The record for a paper on open source software.

Summary

Looking in more detail at the top backlinks to the repository, as listed in Figure 2, we can see that Wikipedia represents four out of five top pages. This includes the Wikipedia page on Free Software, which links back to a Government report on the cost of ownership of open source software. The Wikipedia pages on the European Commission and Proportional Representation are ranked second and third respectively. The Proportional Representation page links back to the full-text of a 2010 workshop paper: Review of paradoxes afflicting various voting procedures where one out of m candidates (m ≥ 2) must be elected. The fifth and only backlink not be Wikipedia is avert.org, an AIDS Education site which links back to the record of an early LSERO paper: Peer education, gender and the development of critical consciousness : participatory HIV prevention by South African youth.

In Table 1, the Top 5 Referring Domains to LSE Research Online are WordPress, Blogspot, Wikipedia, Flickr and Google. We can see the dominance of international social platforms here with WordPress (14,502 links) and Blogspot (11239 links), followed by Wikipedia (349 links), Flickr (272 links) and, finally a search engine, google.com (225).

In Figure 3, Top 5 Resources in Repository (sorted by flow metrics), we can see several links to LSERO information pages including the home page and the feed of latest additions. There are, however, several direct links to full-text papers including an Economic History Working Paper on A dreadful heritage: interpreting epidemic disease at Eyam, 1666-2000. Sorting this data by number of backlinks, as shown in Table 2, the top item is the LSERO homepage with 501 backlinks. The second item is the PDF of one of our most downloaded papers of all time: The Hartwell Paper.

Discussion

So what can I draw from the results of the Majestic SEO report of LSE Research Online? Analysing the top referring domains according to the Majestic report, it seems reasonable to suggest that adding links to repository content on blogging platforms such as WordPress and Blogspot may result in an increased SEO ranking. We often link to LSERO content in various LSE Library blogs hosted on Blogspot, including New Research Selected by LSE Library. Flickr is also listed as a top referring domain according to the Majestic SEO but running a Google search for site:flickr.com “eprints.lse.ac.uk” retrieves zero results. It’s difficult to ascertain how MajesticSEO gets this result when Google does not confirm the findings – perhaps it uses very different algorithms to Google? The MajesticSEO top referring domains indicate that blogging platforms are the main referring domains to LSERO content. However, according to our Google Analytics stats, 76.55% of traffic to LSERO is from searches. Furthermore, the Majestic report indicates that there are 349 matched links to LSERO content on Wikipedia. “Running the search site:wikipedia.org “eprints.lse.ac.uk” in http://www.google.co.uk/ you get (on 11 October 2012) “About 92 results”. From the last page of the results, by repeating the search to include omitted results, Google ends up with 80 hits.” Searching for eprints.lse.ac.uk in http://en.wikipedia.org/wiki/Main_Page retrieves 83 hits. How does MajesticSEO retrieve such varying results?

Looking at backlinks, it’s important to note that the majority of top backlinks refer to papers that have the full-text attached and often link directly to the full-text PDF, of course resulting in a direct download. In addition, the Top 5 Resources in Repository (sorted by external backlinks) as seen in Table 2 tallies with our consistently popular papers according to Google Analytics and our analog statistics.

It is apparent that the inclusion of repository links on domains such as Wikipedia and blogging platforms appears to have a positive impact in helping the relevancy ranking weighting for LSERO content in web pages. This is not to mention direct hits on the links themselves, adding directly to the site’s visitors, and thus the dissemination of LSE research outputs. However, whether we can draw firm conclusions from the Majestic report remains to be seen, particularly with such differing results to those found on Google.

Thanks to my colleague Peter Spring for his advice when writing this post.

About the Author

Natalia Madjarevic is the manager of LSE Research Online, LSE Theses Online and LSE Learning Resources Online, the repositories of The London School of Economics and Political Science.

Natalia is also the Academic Support Librarian for the Department of Economics and LSE Research Lab. Joining LSE in 2011, prior to that Natalia worked at libraries including UCL, The Guardian and Queen Mary, University of London. Her professional interests include Open Access, research support, REF, bibliometrics and digital developments in libraries.

A post published in August 2012 on an MajesticSEO Analysis of Russell Group University Repositories highlighted the importance of search engine optimisation (SEO) for enhancing access to research papers and is part of a series of articles on different repositories and provided summary statistics of the SEO rankings for 24 Russell Group University repositories.

This work adopted an open practice approach in which the initial findings were published at an early stage in order to solicit feedback on the value of such work and the methodology used. There was much interest in this initial work, especially on Twitter. Subsequent email discussions led to a number of repository managers at Russell group universities agreeing to publish more detailed findings for their repository, together with contextual information about the institutional and the repository which I, as a remote observer, would not be privy too.

We agreed to publish these findings on this blog during Open Access Week. I am very grateful to the contributors for finding time to carry out the analysis and publish the findings during the start of the academic year – a very busy period for those working in higher education.

The initial post was written by Yvonne Budden, the repository manager for WRAP, the Warwick Research Archives Project. It is appropriate that this selection of guest blog post begins with a contribution about the Warwick repository as Jenny Delasalle, a colleague of Yvonne’s at the University of Warwick and myself will be giving a talk on “What Does The Evidence Tell Us About Institutional Repositories?” at the ILI 2012 conference to be held in London next week.

SEO Analysis of the University of Warwick’s Research Repositories

The following summary of a MajesticSEO survey of the University of Warwick’s research repositories, together with background information about the university and the repository environment has been provided by Yvonne Budden.

A Little Background on Warwick

The University of Warwick is one of the UK’s leading universities with an acknowledged reputation for excellence in research and teaching, for innovation and for links with business and industry. Founded in 1965 with an initial intake of 450 undergraduates, Warwick now has in excess of 22,000 students and employs close to 5,000 staff. Of those staff just fewer than 1,400 are academic or research staff. Warwick is a research intensive institution and our departments cover a wide range of disciplines, including medicine and WMG, a specialist centre dedicated to innovation and business engagement. In the 2008 RAE nineteen of our departments were ranked in the top ten for their unit of assessment and 65% of the submitted research outputs were ranked 3* or 4*.

University of Warwick’s Research Repositories

Warwick’s research repositories began in the summer of 2008 with the Warwick Research Archives Project (WRAP), a JISC funded project that created a full text, open access archive for the University. WRAP funding was taken by the Library and in April 2011 we launched the University of Warwick Publications service, which was designed to ‘fill the gaps’ around the WRAP content with a comprehensive collection of work produced by Warwick researchers. The services work on the same technical infrastructure but WRAP remains distinct and exposes only the full text open access material held. The system runs on the most recent version of the EPrints repository software, using a number of plugins for export, statistics monitoring and most recently to assist in the management of the REF2014 submission. To date we do not have a full text mandate for WRAP and engagement with both WRAP and the Publications service varies across the departments. Deposit to the services is highly mediated through the repository team and so engagement is not necessarily reflected in the number of papers available per department, especially as some departments benefit more from the service’s policy of pro-active acquisition of new material where licenses allow. I would judge that our best engagement in terms of full text deposit comes from Social Science researchers but we also have some strong champions in the Medical School, History, Life Sciences and Psychology.

Size and Usage Statistics

At the end of August 2012 WRAP contained 6,554 full text items covering a range of item types, journal articles, theses, conference papers, working papers and more. The Publications service contained a further 40,753 records. In terms of usage since its launch the system has seen 900,997 visits according to Google Analytics, an average of just over 18,000 a month in the 50 months active. To track downloads we use the EPrints plugin, IR Stats, this counts file downloads either directly or through the repository interface. IR Stats will only count one download per twenty-four hours from each source, but will count multiple downloads if an item has multiple files attached. Over the life of WRAP the files held have been downloaded a grand total of 730,304 times with 49.08% of downloads coming from Google or Google Scholar.

Expectations of the Survey

Going into the survey using the MajesticSEO system wasn’t sure what to expect from the results, the majority of the work we’ve done so far with the statistics is with the Google Analytics and the IR Stats package. Looking at the referral sources in the our Google output I can indicate a number of sources I might expect to see back links into the system, including our Business School (wbs.ac.uk) and the Bielefeld Academic Search Engine(BASE) as well as a number of smaller sources. The Warwick Blogs service seems to have fallen out of favour over the past few years with the number of hits from there dropping as people move to other platforms. Above all I’m most curious to see if the SEO analysis can help with the work I am doing in promoting the use of WRAP and the material within it. If this work can assist me in creating the kinds of ‘interest stories’ that help to persuade researchers to deposit it could become another valuable source of information. We are also looking at expanding the range of metrics we have access to, looking at the IRUS project as well as the forthcoming updated version of IR Stats, recently demonstrated at Open Repositories 2012.

Our Survey Results

The data for this survey was generated on the 10^th September 2012 using the ‘fresh index’ option, although the images were captured on 19 October. The current results can be found if you have a MajesticSEO account (which is free to obtain). The summary for the site is given below showing 413 referring domains and 2,523 backlinks.

Figure 1: MajesticSEO analysis summary for wrap.warwick.ac.uk

On first glance this seems to be rather low in terms of backlinks, it also shows a fairly low number of educational domains linking to us. The top five backlinks in to the system can be seen below, ranked as standard by the system by a combination of citation and trust flow:

Figure 2: Top 5 Backlinks

Interestingly this lists some of the popular referrers we see in Google Analytics driving traffic to us, but not some others I might have expected to see. The top referring domains are shown below:

Figure 3: Top Referring Domains

This is the only place in the results where Google features at all. The top five pages, as ranked by the flow metrics show a fairly distinct anomaly, as two of the pages are not listing any flow metric information despite this supposedly being the method by which they are ranked:

Figure 4: Findings Ranked by Flow Metrics

The top five pages as sorted by number of backlinks can be seen in the table below:

Table 1: Top 5 Pages, Sorted By Number of Links

These five items are as follows:

1. A research paper on the impact of cotton in poor rural households in India.
2. The WRAP homepage.
3. A PDF of an economics working paper on currency area theory.
4. A PDF of an economics working paper on happiness and productivity.
5. The record for a PhD thesis on Women poets.

Summary

The top ten backlinks into the WRAP system include a range of sources, from this blog, two Wikipedia pages and two referrals from the PhilPapersrepository, which monitors journals, personal pages and repositories for Philosophy content. We also see a two of pages that collect literature on health topics who are linking back to us, a Maths blog and the newsletter of the British Centre of Science Education.

Interestingly in Figure 3 there is no mention of the University of Warwick or any of its related domains (wbs.ac.uk for the Business School, for instance). I assume this is because MajesticSEO are excluding ‘self’ links, so as WRAP is a Warwick subdomain they are excluding a lot of the links I am aware of. This may also take into account the lack of any backlinks from the Warwick Blogs service. Many of the domains listed here are blog platforms of one form or another, which may be because of the database driven architecture of these platforms and the way the MajesticSEO system are reading those links. For example, if a researcher puts a link to his most recent paper in WRAP on the frame of the blog and this propagates onto every post in the blog, does this count as a single link or as many? We are also seeing links from sources such as the BBC and Microsoft, where, again, it would be nice to be able to see who was linking to what and from where in these domains.

The top pages, as listed by number of backlinks in Table 1, show a trend for linking directly to the file of the full text material we hold in WRAP. This information would tie in nicely with the fact that item three is the most downloaded paper in WRAP over the lifetime of the repository, with 9,162 downloads to the end of August 2012. So in this case we can draw a tentative line between the number of downloads and the number of backlinks. However we can’t follow this theory through, especially as the top paper linked to externally, Paper 1 as listed in Table 1, has been downloaded only a fraction of the number of times compared to the currency working paper. When listed by the flow metrics, as in Figure 4 the pages largely follow the results as seen for the Opus repository at Bath and link to pages about the repository. This is apart from the two anomalous results where despite having no citation or trust flow scores they are ranked second and third, when ranked on flow metrics.

Discussion

I think when looking at metrics the most important thing for a repository manager to do is to be able to build stories around the metrics, as these help the researchers to engage with the figures. Was this spike in downloads because of featuring in a conference, or an author moving to a new institution, or for some other reason? What can I show my users that are going to help them to make the decision to use us over other options and to expend scare time resources maintain a blog or Twitter account? Here the issue, I have with the data we have discovered is that the number of backlinks into a repository will never conclusively prove that a paper will get more downloads, as ably illustrated by the example above. Many researchers are not interested in the fuzzy conclusions we can draw at this point; they want to see clear, conclusive proof that links = downloads = citations.

I also think that search engine performance is an increasingly difficult area to be really conclusive about, especially now users can ‘train’ their Google results to prefer the links they click on most often. This was recently a cause of concern for us as it was reported that our Department of Computer Science (DCS)’s EPrints repository was overtaking our Google ranking and that WRAP didn’t feature until page two of the results now. This wasn’t the case, but because the user reporting this to us was heavily involved in the area of computer science his Google rankings had preferred the DCS repository to the WRAP one as the results were more relevant to his interests. In the same was as when I search for ‘RSP’ my top result is now the Repositories Support Project and not, RSP the Engineering Company or the Peterborough Health and Safety firm as it was initially

We need to always be conscious of what the researcher want from metrics and whether it is possible for us to give it to them. As with any metrics we need to be aware that we have to be explicit in what it is that we are saying and what can be inferred by it. If we are users of metrics don’t understand how the metrics are being developed or how the search engines ranking algorithms work, we won’t be able to confidently predict what we can do to improve them. It may also come down to the way researchers are using these services and for what purpose, which may be why we are not seeing any evidence of the use of services like Academia.edu and LinkedIn. I would imagine if researchers are using services to showcase their work to prospective employers and other researchers they may prefer to link to the publisher’s version of their work rather than the repository versions. I suspect the interest story from the SEO data may be more about ‘who’ is linking to their work rather than where they are linking from, which is detail we cannot and possibly should not be able to provide.

About the Author

Yvonne Budden (@wrap_ed), the University of Warwick’s E-Repositories Manager is responsible for WRAP, the Warwick Research Archive Portal and is the current Chair of the UK Council for Research Repositories (UKCoRR).

Email: Y.C.Budden@warwick.ac.uk

Open Access Week, a global event now entering its sixth year, is an opportunity for the academic and research community to continue to learn about the potential benefits of Open Access, to share what they’ve learned with colleagues, and to help inspire wider participation in helping to make Open Access a new norm in scholarship and research.

I am participating in Open Access Week by sharing my experiences of making use of the Social Web to maximise access to papers hosted in institutional repositories. Tomorrow (Tuesday 23 October 2012) I am giving a talk on “Open Practices for the Connected Researcher” in a seminar which is part of a series of Open Access Week events which are taking place at the University of Exeter.

On Thursday, as described in a news item published by the University of Salford, I am the invited guest speaker for an Open Access event which will take place at the  Old Fire Station at the University of Salford where I will give a talk on “Open Practices and Social Media for the Connected Researcher“.

The following day I will be giving a talk on “Open Access and Open Practices For Researchers” at the University of Bath. This event, which marks the launch of a Social Media programme for Researchers, will include a presentation from Ross Mounce, a PhD student and Open Knowledge Foundation Panton Fellow at the University of Bath, who will talk about the need for true Open Access (as originally defined), why it matters and the plethora of options we have for OA publishing in addition to my talk.

In addition to such ‘real-world’ activities in support of Open Access Week I am also taking part in the Networked Researcher Blogging Unconference and earlier today published the launch post for the unconference.

My slides for tomorrow’s talk are available on Slideshare and are embedded below.

In my case the papers nobody cares about are an article published in LA Record in 1997, a paper on Collection Level Description also published in 1999 which I had forgotten about until I rediscovered it a few years ago and uploaded to the repository, the final report for the QA Focus project and a peer-reviewed paper on Using Context to Support Effective Application of Web Content Accessibility Guidelines.

It was the peer-reviewed paper I was most interested in. This paper, written by myself, David Sloan, Helen Petrie, Fraser Hamilton and Lawrie Phipps and published in the Journal of Web Engineering (JWE), has only been downloaded twice. Clearly nobody is being deafened by the impact of this paper challenging the status quo!

Given that a total of 13,104 papers of mine have been downloaded from the repository what are the reasons for the lack of interest in this paper?

The obvious starting point would be the content. But this paper was a follow-up from previous papers on Web accessibility which have been well-read and widely-cited and the interest in our papers in this area has continued.

Looking at the email folder about this paper it seems that the first version of the paper was submitted to the publishers in July 2005. I seem to recall that we were invited to submit a paper based on an updated version of a paper on Forcing Standardization or Accommodating Diversity? A Framework for Applying the WCAG in the Real World by the same authors which had been presented at the W4A 2005 conference.

We received positive comments from the reviewers in August 2005 and responded with appropriate updates to the paper. But then everything went quiet. It wasn’t until August 2006 when we received the final proofs of the paper and September 2006 when we received confirmation that the paper had been accepted and the paper had published in the Journal of Web Engineering, Vol. 5 No. 4 in December 2006. This was 17 months after we had submitted the first version of the paper!

By this time myself and my co-authors had forgotten about the paper, and the ideas we described had been superceded by a paper on Contextual Web Accessibility – Maximizing the Benefit of Accessibility Guidelines presented at the W4A 2006 conference in May 2006.

Looking at the download statistics for my papers it seems that I began depositing items in the Opus repository in October 2008. My first set of papers were deposited by repository staff based on the links available from the UKOLN Web site. However it would appear that the JWE paper had not uploaded, probably because I had failed to include it in my list of publications due to its long gestation period. A few months ago I noticed that the paper had not been uploaded to the repository so on 17 May 2012 I uploaded the paper.

The reason for the lack of downloads is now clear: the paper wasn’t available until recently! And by the time the paper was available the ideas were no longer current.

What are the lessons which can be learnt which I can share in my talk on “Open Practices for the Connected Researcher“? I would suggest:

Repository items need to be made publicly available when the ideas are current. Depositing old papers may be useful for preserving the content and for record-keeping purposes, but not if the aim is to maximise the impact of the ideas.

Of course there is a bigger question about the value of peer-reviewed papers. In his 1,000th blog post Tony Hirst gave his reflections on The Un-academic. Tony pointed out that “Formal academic publications are a matter of record, and as such need to be self-standing, as well as embedded in a particular tradition“ and contrasted this with blog posts which are “deliberately conversational: the grounding often coming from the current conversational context – recent previous posts, linked to sources, comments – as well as discussions ongoing in the community that the blog author inhabits and is known to contribute to“.

Tony argued the value of blogs in the support of the research process by point out blog posts can provide:

“a contribution to a daily ongoing communication with a community that often mediates its interests through the sharing of links (that is, references); in part it’s a contribution of ideas at a finer resolution than a formal academic reference, and in completely different style to them, to the free flow of ideas that can be found through the searchable and sharable world wide web.“

Since 2005 myself and my colleagues have had peer-reviewed papers published at the W4A conferences in 2005, 2006, 2007, 2008, 2010 and 2012. This is part of the “annual ongoing communication with a community that often mediates its interests through the sharing of links (that is, references)”. However sometimes this process goes wrong, as has been described in this post. Although the problems associated with the long time frames it can take for research work to be published this doesn’t mean that the process of research publications is fundamentally flawed. However I think this example does illustrate the need for researchers to make “contribution to a daily ongoing communication with a community that often mediates its interests through the sharing of links“.

Tony’s blog post concludes by referencing a number of recent posts by Alan Levine (@CogDog) in which he has shared his thinking on blogging: The question should be: why are you NOT blogging?; Every box you type in can be a doorway to creativity, and in a roundabout way, Gotta know when to walk. Alan’s first post provides his reflections on his blogging activities since he started 0n 19 April 2003. This long post is worth reading, but can be summarised very succinctly:

So here is why I blog. It is foolish and informationally selfish, not to.

Perhaps that should be the key message I give in my talk in Exeter during Open Access Week. Oh, having reflected on the paper which nobody reads I have decided that if a peer-reviewed paper is not read, this is a failure. My time and the time spent by my co-authors in writing the paper could have been more productively spent on other work. And no, unlike blog posts in which writing ideas may be a useful process in itself, peer-reviewed papers aren’t intend to assist in self-reflective.

Twitter conversation from Topsy:  [View]

There is a need “to investigate whether links [from popular social media services] are responsible for enhancing SEO rankings of resources hosted in institutional repositories” concluded the paper by myself and Jenny Delasalle which asked “Can LinkedIn and Academia.edu Enhance Access to Open Repositories?“.

The importance of SEO rankings for surfacing content hosted in institutional repositories can be gauged from the responses to the query I asked on the JISC-Repositories JISCMail list: “Does anyone have any statistics on the proportion of traffic which arrives at institutional repositories from Google?”. I asked a similar question on Twitter and found that mature research repositories seem to get about from 50-80% of their traffic from Google. This aligns with the findings reported by Les Carr for the University of Southampton back in 2006: “the majority of repository use, if I can equate eprint downloads with repository use, is due to external web search engines (64%)“. Indeed since it has been reported that direct downloads of PDFs hosted in repositories may not be reported unless Google Analytics has been configured appropriately such figures may be an underestimate!

In light of the importance of Google in supporting repositories in their mission of making research papers easily accessible to others it will be useful to gain a better understanding of the factors which contribute to supporting the discoverability of the content hosted in institutional repositories.

The survey described in this post reports on summary SEO findings for the 24 Russell Group universities. The aims of the survey are to provide a benchmark for comparisons with surveys which may be carried out in the future, to attempt to identify any interesting usage patterns which may help to enhance the effectiveness of institutional repositories and to identify the highest ranked domains which provide links to institutional repositories.

Survey Using MajesticSEO

The data was collected on 27-28 August 2012 using the MajesticSEO service. Note that the current finding can be obtained by following the link in the final column. The findings can be viewed if you have signed up to the free service.

NOTE:

1. The list of repositories is taken from OpenDoar.
2. The ERIC repository at the University of Exeter is hosted at https://eric.exeter.ac.uk/repository/ Since the repository home page is a redirect from https://eric.exeter.ac.uk/ it was possible to analyse the SEO rankings and get appropriate results.
3. The eScholar repository at the University of Manchester is hosted at http://www.manchester.ac.uk/escholar/  Figures for this home page are given but since the domains with incoming links may refer to pages hosted on the manchester.ac.uk domain, these figures are not given in order to avoid skewing the findings.
4. The Queen’s University Belfast repository is hosted at http://www.qub.ac.uk/schools/SchoolofPoliticsInternationalStudiesandPhilosophy/Research/PaperSeries/ConWEBPapers/ Figures which are available for this home page are given but since the domains with incoming links may refer to pages hosted on the qub.ac.uk domain, these figures are not given in order to avoid skewing the findings.
5. The DCS repository at the University of Sheffield is hosted at http://www.shef.ac.uk/dcs/research/publications Figures which are available for this home page are given but since the domains with incoming links may refer to pages hosted on the shef.ac.uk domain, these figures are not given in order to avoid skewing the findings.
6. The YODL repository of the University of York is hosted at http://dlib.york.ac.uk/yodl/app/home/index Figures which are available for this home page are given but since the domains with incoming links may refer to pages hosted on the dlib.york.ac.uk domain, these figures are not given in order to avoid skewing the findings.

Table 2 gives the total number of links to the high-ranking domains which are listed in the survey, together with the Alexa ranking for these domains. Note Google.com has the highest Alexa ranking and is listed at number 1. Figure 1 shows the significance of links from blog platforms compared with the other most highly-ranked domains.

Figure 1: Histogram of number of incoming links from top domains

Discussion

In a previous post I suggested that since LinkedIn.com is so widely used across Russell Group Universities, encouraging researchers to provide links to their papers hosted in their institutional repository would enhance the visibility of papers to Google, especially since LinkedIn has such a high Alexa ranking (it currently is listed at number 13 in the global ranking order).

However it appears that LinkedIn does not appear to have a significant presence according to the findings provided in MajesticSEO (although the free version does only list the top five domains).

Based on the information obtained in the survey it would appear that two blog platforms, WordPress.com and Blogspot.com, are primarily responsible for driving traffic to institutional repositories, having both high Alexa rankings together with large numbers of links to the repositories.

Following these two platforms, but a long way behind, we find Wikipedia and the BBC and then, perhaps somewhat confusingly, Google itself (perhaps links from Google Scholar). The presence of media sites such as the BBC, CNN and the Guardian suggest that researchers (or their media advisers) are doing a good job in ensuring that these organisations provide links to original research papers when stories about university research are being covered in the media.

But perhaps the most noticeable findings is that only one University Web site – Oxford’s – is included in the list of the top 5 domains across all of the Russell Group Universities. The low Alexa ranking (6,764) for the Oxford University Web site in comparison with the other sites listed (which have an Alexa ranking ranging from 1 to 259) suggests that links from university Web sites, even prestigious universities such as Oxford, will not have a significant impact on Google search results. It should also be noted that links from the University of Oxford Web site will not provide SEO benefits to the University of Oxford’s repository, which is hosted in the same domain (ox.ac.uk).

Limitations of this Survey

It should be noted that these conclusions are based on just one SEO tool and only a small selection of the findings are available. A more comprehensive survey would make use of the licensed version of the service, and make use of other SEO tools to compare the findings.

In addition Google do not publish the algorithms on which their search results are ranked so there can be no guarantee that the findings provided by SEO tools will relate directly to users experiences of using Google.

In order to relate these findings to the ways users access resources hosted on a repository there will be a need to examine usage statistics for repositories. It would be interesting to see if the downloads for the most popular items show any correlation with links from the services listed above.

Survey Paradata: The findings given in Table 1 were collected on 27-28 August 2012 using the free version of MajesticSEO. The Alexa rankings listed in Table 2 were obtained from the Alexa survey and collected on 28 August 2012. Where the findings from MajesticSEO were incomplete, due to the repository not being hosted on the root of a repository sub-domain this information was recorded and any data collected was not included in further analysis.

Twitter conversation from: [Topsy] – [SocialMention] – [WhosTalkin]

Where do visitors to this blog go do when they click on a link which published in a blog post? When I looked at the click statistics for the past year I was surprised that the top ten pages, with just one exception, were to the home page of a number of UK Universities: Abertay, Aston, Cambridge, Bangor, Buckingham, Glasgow, ECA, Exeter and Falmouth. I subsequently found that these were the nine of the 26 institutions which had been hyperlinked in a post on Best UK University Web Sites – According to Sixth Formers published in 2010.

Apart from the links followed from this single post the other top web sites visited in the past year are:

twitter.com, scholar.google.co.uk, opus.bath.ac.uk, iwmw.ukoln.ac.uk, computerweekly.com and ukoln.ac.uk

What does this evidence tells us? Suggestions for the popularity of these Web sites are given below:

twitter.com: I normally provide a link to tweets which I cite. This enables me to find the original source if I wish to make use of it in the future. In addition it will help people reading the post to see the source, see the context and find out more about the Twitter user. It would appear that my decision to do this has proved useful as people do seem to be clicking on links to tweets.

scholar.google.co.uk: This initially appeared to be an anomaly. However I subsequently realised that a post giving Thoughts on Google Scholar Citations published a few days after Google’s announcement that Google Scholar Citations Open To All had proven very popular after I had left a comment linking to the post on Google’s blog post. Scholar

opus.bath.ac.uk: It is pleasing to see that links to Opus, the University of bath’s institutional repository, features so highly. These are primarily to copies of my peer-reviewed papers. Interestingly a recent paper by myself and Jenny Delasalle asked Can LinkedIn and Academic.edu Enhance Access to Open Repositories? Although we feel the answer is “yes” it would appear that this blog also has a significant role to play in enhancing access to such papers.

iwmw.ukoln.ac.uk: As might be expected there are significant numbers of visits to the Web site for UKOLN’s annual Institutional Web management Workshop, IWMW, since the event is featured on this blog when we issues the call for submissions, when we open the event for bookings, and when we publish reflections on the event.

computerweekly.com: The reason for the significant number of visits to the Computer Weekly Web site is simple: they will have read the post in which I announced that this blog had been short-listed for the Computer Weekly’s IT Blogger of the Year award. Since I was the runner-up I know that large numbers must have followed the link and voted for this blog:-)

ukoln.ac.uk: As might be expected there are significant numbers of visits to the UKOLN Web site which hosts many of the resources for work which I write about on this blog.

Redesign of the Blog’s Sidebar

It should be noted that visitors do not only follow links provided in blog posts; the blog’s sidebars and navigation bar also provide addition content and links to resources.

Sometime ago I came across Markosweb which provides information about Web sites including the UK Web Focus blog. I was particularly interested in the heat map for the blog. As described on the Web site:

Heatmap – An F-shaped principle of how web-pages are read: two horizontal strips and one vertical. Using this principle we’ve suggested where your visitors’ eyes will first be directed to on the main page.

The left hand sidebar provides information about the blog which I feel is important information. However, as shown in the accompanying image of the heat map for a previous design of the blog, although the blog’s search box is likely to be used by people which wish to search for additional posts, the email subscription sign-up area was a waste of space, as this is something people will only do once, if at all.

In light of the suggestion that the heat map can help be to locate important content I updated the design of the sidebar in March 2012. The blog now has a Featured Paper area beneath the search box (as illustrated) which summarises a paper and provides links to the paper. The featured paper is updated every couple of weeks.

It was not clear to me whether the redesign had any effect on users’ behaviour. Having for the first time analysed the statistics for users clicks it would appear that this redesign has helped to raise the visibility of my papers (it should be noted that the clicks may also have come from links provided in blog posts) .

What Does the Evidence Tell Us?

Myself and Jenny are presenting a talk at the Internet Librarian International (ILI 2012) conference to be held in London on 30-31 October which will try to provide an answer to the question: What does the evidence tell us about institutional repositories? The evidence from analysis of the blog’s statistics tells us that the blog delivers significant traffic to the University of Bath’s repository. Given the significant relationship between this blog and the Opus repository it will be interesting to see if the links from this blog have any impact on the repository’s search engine rankings and the visibility of the repository itself, as well as my papers, for researchers who make use of Google to search for relevant information.

Perhaps my post which asked Can LinkedIn and Academia.edu enhance access to open repositories? which was republished yesterday on the LSE Impact of Social Sciences blog gave an incomplete view of the importance of social media for researchers seeking to maximise the impact of their work. Maybe it would be a mistake to ignore the importance of researcher’s blog, not just as an open notebook for sharing ideas at an early stage and inviting feedback, but to support the dissemination of existing published work?

Catching up with overnight tweets on a wet morning at the bus stop

At 7.30 am I was waiting in the rain for my bus to work. As normal I was catching up with the tweets I’d received overnight and had downloaded to my iPod Touch before leaving home. One of the tweets which was particularly interesting was from @KeitaBando. I met Keita, Digital Repository Librarian and Coordinator for Scholarly Communication for the My Open Archive service, at the Open Repositories OR 2012 conference recently, following his poster presentation on Current and Future Effects of Social Media-Based Metrics on Open Access and IRs. Keitatweet announced news of relevance to many who attended the OR12 conference:

http://Academia.edu Blog: Announcing @academia Analytics http://blog.academia.edu/post/29490656413/announcing-academia-edu-analytics

Since one of the papers I had submitted to the OR 2012 conference asked “Can LinkedIn and Academic.edu Enhance Access to Open Repositories?” this announcement was of particular interest to me.

The blog post Announcing Academia.edu Analytics described how:

Today we are announcing the release of Academia.edu’s Analytics Dashboard [which] allows academics to view the real-time impact of their research.

The development is based on the changing environment provided by the Web:

Increasingly, the primary consumption experience for scientific content is the web, and yet scientists have not generally been aware of the metrics around this consumption. If you ask a Harvard biology professor with 200 publications how many downloads she experienced in the last 30 days, typically she will not know.

Academia.edu’s Analytics Dashboard is changing this. It allows an academic to understand in sophisticated detail how their research is being used by the academic community. It shows them countries that are sending them the most traffic, search engines and other sites that are sending them the most traffic, and overall profile views and document views. 

What does the new service tell me about my papers? It seems that on 11 August 2012 there were 5 views of my items available on Academia.edu and over the last 230 days there had been a total of 9 views of information about my papers and 11 views of my profile on Academia.edu.

Since the analytics service “allows academics to view the real-time impact of their research” we can explore the individual visits:

• On 14 August at 04.10 someone from the US followed a link on Academia.edu to information about the paper on Can LinkedIn and Academia.edu Enhance Access to Open Repositories?
• On 14 August at 021.39 someone in Australia used Google and found information about the paper on Accessibility 2.0: Next Steps For Web Accessibility
• On 14 August at 08.19 someone in the UK viewed my profile page.

and then no other activities until 22.00 on 11 August when someone from Argentina read information about the paper on Open Metrics for Open Repositories.

Clearly such numbers are underwhelming! This would therefore seem to provide evidence which suggests that the question Jenny Delasalle and myself posed in our paper “Can LinkedIn and Academic.edu Enhance Access to Open Repositories?” would be “No” in the case of Academia.edu.

Since the metadata I have uploaded to Academia.edu provides a link to papers hosted on Opus, the University of Bath repository, it will be interested to make comparisons with the numbers of downloads of papers hosted on Opus over a similar period.

Since the Opus service provides statistics on a monthly basis it was not possible to make a direct comparison. However looking for the download statistics for my papers during July 2012 I found that there had been a total of 679 downloads with the top two downloads which, as might be expected, were of my most recent two papers, having been downloaded a total of 184 times.

From these personal experiences we might conclude that Academia.edu is not a significant driver of traffic to my papers and it might therefore be questionable as to whether it is worth creating a profile in the service and adding links to one’s papers. I think it would be a mistake to draw such conclusions, for the following reasons:

• These experiences may not be replicated by others.
• I have chosen to replicate my research profile across a number of services, including Mendeley, LinkedIn and ResearcherGate as well as Academia.edu. I would expect some of these services to be widely used, while others are less-well used.
• Using a variety of researcher profiling services with links to my papers will enhance the ‘Google juice’ for the papers (and the repository). Use of these services can therefore enhance the discoverability of the papers for people who use Google – and this is likely to be the majority of people!

I’d be interested to hear about other people’s experiences of Academia.edu. Is anybody finding that their pages on the service are being well-used?

Twitter conversation from Topsy: [View]

The University of Bath recently announced that “the University’s new Current Research Information System, Pure is now available to all academic and research staff“.  The announcement went on to describe how:

Pure provides a single location for staff to store information about their research, such as publications, collaborations, research projects and grants etc and the associations between them. Having been entered into Pure once, data can be used for a variety of purposes, including creating CVs and bibliographies and later this year automatic population of personal web pages. Pure is designed to make it as easy as possible to keep information about research up-to-date, providing ongoing visibility of research activities at the University.

As soon as I saw this announcement I logged on to the service and viewed my papers, which had been deposited using the University of Bath’s ePrint’s service, Opus.

I have to admit that I was impressed with the interface, which provides a much cleaner interface that the Opus interface I have had to use previously.  In addition to the listing of my papers, illustrated, the editing interface was also much easier to use and, as illustrated, I am able to update the metadata from a single page – a simple task which was  cumbersome when I had to use the ePrints service.

In addition to the simple list display of my papers as illustrated there is also an option to view a graph of connections with co-authors.

From my initial use of Pure I felt that the service provided a valuable development to the University’s ePrints service, with improved editing and display features.

I was very pleased with the service and was glad that I had chosen to use it as soon as I saw the announcement that the service had been launched.

Impure Thoughts

Quality Validated Metadata or Instantly Updated Access to Content?

Further use of Pure, however, revealed a number of limitations. The reverse date order display of items within years is a minor glitch (as shown in the initial image, my first paper published in 2012 is displayed after two other papers published this year. However  I have to admit that I was annoyed when I found that  items I had edited were deleted from Opus, with a 404 Item not found error message being displayed. It seems that items are deleted if they are edited and are not available until edits have been validated. In my case, there was a delay or several days before the items were retrieved due to a combination of annual leave and sickness.  However this seems t0 me to be an inappropriate policy decision especially, as in my case, the items were new and, during this period, are more likely to be read.  I’m pleased that my concerns have been acknowledged by Bath repository staff who have agreed to revisit this policy. I am highlighting this issue here as it appears likely that others may well encounter the tension between the repository managers’ desire to ensure that they possess high quality validated metadata (especially in the run-up to the REF) and the desire for researchers to be able to maximise access to their research. Such concerns were highlighted in a recent post on the JISC-Repositories JISCMail list when Stephan Harnard argued that:

What OA IRs need today, urgently, is not cataloguers to monitor quality, nor IP specialists to monitor rights, etc. etc. No intermediary is needed between the author and the IR “monitor”, retard, block or otherwise impede deposits (though help is always welcome to encourage depositors and facilitate and speed their deposits!).

What OA IRs need urgently today instead of needless, costly and counterproductive monitoring and mediation is e*ffective Green OA **mandates (ID/OA)*. That is what will generate deposits (and further minimize the negligible cost per paper deposited).

…

The problem of IRs today is not fraudulent researchers depositing bogus content, it is legitimate researchers failing to deposit OA’s target content (refereed research publications).

Private or Public Content?

The issue of rapidly updated versus validated content is one topic for discussion with colleagues at Bath. However the feature which surprised me most was that the information about my papers is only available to me.  In retrospect I should have realised that the prime function of a CRIS) (Current Research Information System) is for internal management and reporting purposes. As the Pure web site describes:

[Pure] covers Grant applications, Research Income, Projects, Research Outputs, Research staff, Organisational units, External collaborations, and more. It is achieved by integrating Pure with local systems while also capturing data by work processes that ensure quality and completeness.

This makes Pure a single authoritative source of quality-assured information about an institution’s research affairs. Information is available at the desired level of granularity in real-time.

This is the main role which is envisaged locally:

Jane Millar, Pro-Vice-Chancellor for Research, and Project Sponsor, said that the implementation of Pure was a major step forward in how we handle information about our research here at the University: “In an increasingly competitive environment it is essential that we have up to date and accurate information about the excellent research undertaken here at the University. Pure will provide this data and help us to comply with external reporting requirements such as REF2014“.

However the Pure web site goes on to add that “Pure is also a tool for researchers, PIs, and departmental managers - clear and recognised value is provided for these users, which furthers user-acceptance and -uptake; mission-critical factors in any CRIS project“.

This is, however, were I have my reservation. Although the ability to publish a CV of one’s research publications is provided by Pure and, as illustrated, I can currently create a CV in PDF or MS Word formats,  this functionality does not appear to provide the social function in establishing connections with one’s peers that is provided in services such as LinkedIn or Academia. edu. In addition, it seems unlikely that a researcher profiling service which is co-located on the same institutional domain as the institutional repository will provide the ‘Google juice’ to one’s research papers which LinkedIn and Academia.edu appear to pr0vide. It should also be noted that, as described in posts on What I Like and Don’t Like About IamResearcher.com, Thoughts on Google Scholar Citations and Will the Real Scott Wilson Please Stand Up, Please Stand Up services such as LinkedIn, Academia,edu and IamResearcher appear to provide richer interfaces and visualisations than Pure provides.

Conclusions

It would, however, be inappropriate to criticize Pure for not providing the same quality of visualisation of one’s co-author network as, Microsoft Academic Search, for example, does.

Microsoft Academic Search’s visualisation of my co-authors is illustrated. However Microsoft Academic Search also thinks I am an expert in Psychiatry and Psychology! The service has confused me with B D Kelly who is an expert in these areas and, despite updating my profile, I have been unable to decouple my research publications from B D Kelly’s.

Pure aims to provide  an authoritative list of research publications by researchers within the institution which will be needed to support institutional reporting requirements.

However an individual researcher may have different requirements –  and if a key aim is to enhance access to one’s research papers I am still convinced that use of social media services such as LinkedIn and Academia.edu will provide benefits which aren’t provided by a Current Research Information System.

A recent post entitled IWMW 2012: The Feedback summarised the feedback we had received for the recent IWMW 2012 event. In addition to this summary more detailed information was sent to the individual speakers and workshop facilitators on their talks and workshop sessions. Such feedback can be valuable in either showing the value of the contribution made at the event or providing suggestions on how the talk could be improved in repeated in future.

We published the feedback two weeks after the event as it is important that such information is available while the event is fresh in people’s memories. But, of course, there can be other ways of getting feedback. At the UCISA User Support Services Conference which took place a few day’s ago at the impressive Crewe Hall Hotel I was pleased to receive feedback on Twitter on the talk I gave on “Social Media: For Ourselves and For Our Customers” which have been summarised on Storify. The feedback included:

• Excellent presentation, you gave me a lot of new ideas for how I can communicate with my staff and customers. Thanks!
• Brilliant presentation from @briankelly – good to have a push to tweet a bit more!
• Brilliant talk from @briankelly – typically informative, insightful, and full of #lolz…
• Also really enjoyed @briankelly talk about social media. Engaging. Had a chuckle. And I think he likes a real ale so is in my good books

together with an example of an action taken as a result of the talk:

• Inspired to send my first tweet

Beyond the tweets, a post entitled What a difference a day makes published on the Musings from the frontline blog described how

Today we sat and listened to people who had not only aspired to do things differently and better but, most importantly, had achieved it.

and went on to conclude:

So, thank you @heloukee, @maffrigby, @briankelly and #ussc12 for the inspiration. You have provided the relationship counselling that I needed and me and conferences are now blissfully happy together again (for now anyway…

It’s About Links; It’s About Connectedness!

The topic of my talk was the importance of social networks to facilitate more effective collaborative working by making use of the existing social networking infrastructure. Although this is a subject I have spoken about previously, as described recently in a post on It’s About Links; It’s About Connectedness! I was fortunate to see Cameron Neylon’s opening plenary talk at the Open Repositories 2012 conference. As described in the live blog of the  closing session for the conference given by Peter Burnhill:

we need to think about connectivity, as flagged by Cameron. And these places ie Twitter and Facebook… We don’t own them but we need to be I them, to make sure that citations come back to us from here.

The importance of use of such social media services to provide links to papers hosted in open repositories was also highlighted by Peter Burnhill in his observation that:

And there was talk of citation… LinkedIn, Academia.edu etc. is all about linking back to research to data

It was pleasing to see that the ideas described in a paper by myself and Jenny Delasalle which asked “Can LinkedIn and Academic.edu Enhance Access to Open Repositories?” had been highlighted in the conference conclusions. But these particular ideas were just a simple example of the bigger picture provide by Cameron Neylon on the importance of networks which, on a global scale, can enable researchers to address difficult research topics which cannot be achieved by the single researcher or research group.

The Video For Connecting, For Sharing

Cameron’s talk, which is available on YouTube and embedded below, makes the point about the importance of connectivity (the social web) and ease-of-use (the lack of ‘friction’ needed to embed social web tools in workflow practices) very eloquently and is well worth viewing (and I’d like to give my thanks to the OR 12 organisers for publishing this video recording so quickly – and also for making it available on YouTube so it can be embedded in this blog).

It would, however, be a mistake to regard social networks as being purely a tool for scientific researchers – just as some people mistakenly feel that social networks are just for young people or for purely ‘social’ purposes a confusion caused by the different meanings of the term ‘social’. As I described in my talk, for which a video recording is also available, social networks can also be valuable for those working in support services – and institutions should gain benefits in use of social networking services across teaching and learning, research, marketing and support areas if they are regarded as valuable tools rather than treated with suspicion as is current the case in some areas.

Another important point made by Cameron is the importance of openness for both facilitating connections and minimising the friction caused by licensing barriers. The videos of Cameron’s talk and my talk provide another example of the ways in which connections can be made and knowledge and ideas shared by facilitating access to videos of talks at conferences. As I have described in previous talks on amplified events, such approaches can help the ideas shared at conferences escape the constraints of space and time. Many thanks to the OR 2012 and UCISA conference organisers for providing the live videos streams (escaping the constraints of space) and providing rapid access with little access barriers to the recordings of the talks (escaping the constraints of time).  Long may this continue – and if you are considering organising an amplified event the recent “Event Amplification Report” may be of interest.

Twitter conversation via Topsy: [View]

This paper, which was written my myself, Nick, Jenny Delasalle, Mark Dewey, Owen Stephens, Gareth Johnson and Stephanie Taylor, describes the importance of metrics for institutional repositories for a number of stakeholders, including funders at a national level, developers of services which may aggregate repository content, librarians and research support unit within institutions as well as the individual researchers and their departmental colleagues.

In light of the diverse requirements for metrics across these stakeholder communities, in the paper we argue that such metrics should be provided as open data. This would appear to be particularly relevant in the context of open repositories – we are aware of the tensions regarding open access to research publications due to the complexities of quality assurance processes and business models for funding peer-reviewing, but such considerations should not act as a barrier for providing access to the variety of usage statistics and related data associated with repositories.

Our paper is available from the University of Bath institutional repository. In addition, as Nick has described in a post on the Repository News blog at Leeds Metropolitan University, Nick will be presenting the paper later today. The presentation will be given in the RF1: Pecha Kucha – Repository Tools and Approaches which starts at 15.30 today Tuesday 10 July. Note that the Twitter hashtag for the conference is #or2012 – so follow this tag in your Twitter client at around this time to follow the discussion about the paper.  The slides Nick will use in the presentation are available on the Slideshare repository and embedded below.

This paper, which is available from the University of Bath institutional repository, is based on work initially published on this blog.

A blog post entitled “How Researchers Can Use Inbound Linking Strategies to Enhance Access to Their Papers” published on 2 March 2012 described an Inbound linking strategy to get to the top listing on google fast. It occurred to me that my willingness to make use of researcher profiling services such as Academia.edu, ResearcherID, Scopus, Researchergate, Mendeley, Microsoft Academic Search and Google Scholar Citations may have helped to enhance the visibility of my research papers which are hosted in the University of Bath repository. The blog post went on to describe how I found that I was author of 15 of the most downloaded papers in the repository from my department.

More recent investigations reveal that, as illustrated, I have the largest number of downloads of any author at the University of Bath! This was recently brought to the attention of the PVC for Research who, in a departmental meeting, informed me that a University of Bath Research Group had discussed these figures and asked me to share the approaches with other researchers at Bath. In response I mentioned that the approaches I’d taken, the evidence I’d gathered, the hypothesis I had proposed for explaining the evidence, possible alternative hypotheses, the limitations of the approaches, the implications of the findings and areas for further work had been submitted to the Open Repositories 2012 conference – and if the paper was accepted the findings would be available to all, and not just researchers at my host institution.

The paper explores other possible reasons for the high visibility of these papers – and one possibility worthy of further investigation is the provision of many papers in HTML formats and not just PDF and MS Word. However the use of popular researcher profiling services such as LinkedIn and Academia.edu are felt to be worth recommending to researchers in order (a) to ensure that their research papers can be more easily found by their peers on these services and (b) so that links to the paper on their institutional repository can enhance the visibility to Google of the papers as well as enhancing the Google ranking of the repository itself.

Of course it probably needs to be said that that the number of downloads is not necessarily an indicator of quality. However the converse is also true: just because a paper in a repository is seldom viewed does not indicate that it must be a great paper! I am quite happy to promote the use of such approaches since increased numbers of views, especially for the target communities, can help to both embed the ideas given in the papers by practitioners and increase the likelihood that the papers will be cited by other researchers. In my case I’m pleased that, according to Google Scholar Citations, my most cited papers have been cited 87, 67, 54 and 40 times.

My co-author Jenny Delasalle has been investigating use of researcher profiling service at the University of Warwick, her host institution. It was interesting that in Jenny’s research she found that a number of commercial publishers encourage their authors to use services such as LinkedIn and Academia.edu to link to their papers hosted behind the publishers paywalls – and yet we are not seeing institutional views of the benefits of coordinated use of such services by their researchers. Institutional repository managers, research support staff and librarians could be prompting their institutions to make the most of these externally provided services, to enhance the visibility of their researchers’ work in institutional repositories.

Surely it is time for the research community to develop inbound linking strategies to their research work, especially as this can be done so simply. Indeed the OR12 conference organisers have invited us to summarise the ideas described in a poster and a one-minute presentation. The ideas have been summarised using the Pixton cartoon generation tool in four strips.

[link to source]

[link to source]

[link to source]

[link to source]

I’m not sure if it will be possible to use PowerPoint during the one-minute madness but I have prepared some slides which are available on Slideshare and embedded below.

NOTE: A one minute summary of this paper was given on the opening day of the OR 12 conference. A video recording of the summary is available on Vimeo and embedded below.

Also note that a slightly modified version of this post was published on the LSE Impact of Social Sciences blog on Thursday 23 August 2012. You can also view the bit.ly statistics for access to the post via the bit.ly URL.

Twitter conversation from Topsy: [View]

A recent post on this blog which described How Researchers Can Use Inbound Linking Strategies to Enhance Access to Their Papers reviewed personal experiences of the benefits of making use of third party services to provide inbound links to research publications.

In the post I suggested that the large numbers of downloads of my papers from the University of Bath institutional repository may be due to the enhanced Google juice provided by having links to my papers from such services. The purpose of the post was to suggest that researchers may benefit from increased access to their research publications if they are pro-active in using such services. My speculations may, of course, be incorrect; the downloads may be due to the quality of the papers rather than the numbers of in-bound links, for example :-). In addition, downloads themselves are not, of course an indication of quality. However since the papers, which have been through some form of peer-reviewing, will not have any influence if they are never read, I am happy to regard such approaches as helping to enhance the numbers of people reading the papers which may, or may not, lead to some form of subsequent ‘impact’.  Note that the slideshow on “Metrics: The New Black?” by Kristen Fisher Ratan which are available on Slideshare explores such considerations in more detail.

Profiling Institutional Use of Such Services

I recently came across the Libresearch blog which is provided by Jenny DelaSalle who, on her @JennyDelasalle Twitter profile describes herself as a “Research support Librarian: interested in bibliometrics, copyright, scholarly communications, and all sorts!”  I read her posts on topics including Webometrics and altmetrics: digital world measurements, Warwick people on external profile sites and 1,670 Warwick people on Academia.edu?. In the latter two posts she documented evidence of take-up of a number of third party services by researchers at the University of Warwick. Her post included a reference to one of my posts which profiled Russell Group university use of Google Scholar Citations. I am now able to build on Jenny’s work by using some of the survey methodology techniques she has helpfully documented in her blog to document evidence of take-up across the twenty Russell group university of popular third party service which provide links to research publications.

Having read the post on Warwick people on external profile sites it occurred to me that such institutional profiling work would benefit from being seen in a wider context. I therefore used the methodologies documented by Jenny in her blog post to gather similar information across the twenty Russell Group universities.

The findings are given in the following table. Note that the data for the Academia, LinkedIn and ResearcherID was collected on 1 March 2012 and the data for Google Scholar Citations on 3 March 2012.

Discussion

As described in an article on Using LinkedIn For SEO:

Your profile can be an excellent source of SEO friendly links because:

• LinkedIn has great authority in Google
• Your website links can be given unique anchor text with the dofollow attribute
• Your LinkedIn profile can have highly relevant content relative to the websites you own

It might be reasonable to assuming the use of the LinkedIn service comes mainly from staff and research students. In light of the popularity of the service might be find that encouraging researchers to provide links to copies of their papers hosted in their institutional repository will provide benefits not only for the individual researcher, but for the repository service itself, though the increased numbers of inbound links?

The DirectionsSEO site provides information on 5 Inbound Link Analysis Tools which may help to provide evidence of the value of inbound links. Initial experimentation with the Linkdiagnosis.com service suggests, however, that WordPress.com has the highest SEO ranking of domains linking to the University of Bath Opus repository service. But before concluding that researchers should be blogging about their research publications on the WordPress.com platform  I’d welcome feedback on the suggestion that the next stage for maximising access to research publication should be based on inbound linking strategies rather than further developments to institutional services.

Paradata:   As described in  a post on Paradata for Online Surveys blog posts which contain live links to data will include a summary of the survey environment in order to help ensure that survey findings are reproducible, with information on potentially misleading information being highlighted.

The data for the Academia, LinkedIn and ResearcherID was collected on 1 March 2012 and the data for Google Scholar Citations on 3 March 2012.

The values for Google Scholar Citation for the universities of Birmingham and Newcastle include ‘UK’ in the search field in order to avoid including information from US and Australian universities with the same name.

It should also be noted that I was logged into the services when I gathered the information.

It should also be noted that the low values for LinkedIn followers for King’s College London and Queen’s University Belfast are felt to be due to the apostrophe used in the institution’s names. For example of search (carried out on 6 March 2012) on LinkedIn for King’s College London gives 3,418 hits but a search for Kings College London gives 294 hits.

Via Smartr, the iPod Touch app I use to read articles which have been posted by Twitter followers, this morning I came across a link provided by a tweet which described an Inbound linking strategy to get to the top listing on google fast. The post described how the author, a web manager at Florida International University:

… developed  a strategy I would make inbound links to the FIU President’s Council site from places I can control a few of these places include FIU News,Alumni Association, FIU A to Z index, blogs that have comments open, etc.  and on all those I make links using the words FIU President’s Council that link directly to the sites homepage.

The importance of providing links to a resource in order to maximise access to the resource is well understood – particularly, it seems, by spammers.  But how could such well-established techniques be used in an ethical way by researchers?

The answer, it seems to me, is quite simple. Researchers do have access to a wide range of web services which can legitimately provide links to their research publications.   This is an approach I have been using for several years. A summary of the numbers of publications which are listed in the services I use is given in the following Table.

*  The Microsoft Academic Search automatically includes papers from people with the same name.  These need to be manually excluded and there is a delay before updates are validated.  The service currently lists 286 papers, including many from medical researchers of the same name.  However only 39 papers have been claimed as authored by me.

It should also be noted that a number of the services provide links to the research papers (which in my case and normally hosted on the University of Bath institutional repository) although other services only provide the metadata.

Evidence of Enhanced Access

There is a cost to registering for such services and uploading details of one’s papers. However in practice I have found that it does not take a significant amount of time to upload relevant information and the services can provide useful information, such as helping to visualise one’s professional network and, as illustrated (taken from Mendeley) growth in  the number of citations, downloads, followers, etc.

But although individual  may or may not find such information of interest or value, there remains a question as to whether there is any tangible evidence of growth in downloads due to a policy of enhancing the numbers of links to such resources.

A possible answer to that question may be found form an analysis of the download statistics for items stored on Opus, the University of Bath institutional repository.

In order to make comparisons an image is shown of the top 20 most downloaded items provided by staff at UKOLN.

From this list we can see that I am a co-author of 15 of the top 20 items.

There may be several explanations for this:

Quality of the papers: Although two of my papers are the highest ranked papers which have been published at the W4A conference series I am quite happy to say that I am convinced that my colleagues have produced papers of much greater research value.

Social media optimisation: The paper on  Library 2.0: balancing the risks and benefits to maximise the dividends is the second most downloaded single paper from the University of Bath repository. The popularity of this paper was due to the large numbers of downloads shortly after the availability of the paper had been announced on this blog.  Although I am convinced that use of social media can also enhance access to peer-reviewed papers, several of the other popular papers in the above list were published between 2004 and 2007, before Twitter and before I was making significant use of the blog.

To conclude, I believe that adding information about one’s research publications to services such as Academia.edu, ResearchGate, Microsoft Academic Search and Google Scholar citations can increase the visibility of the papers to Google, as well as to users of the services, which may then lead to increased numbers of downloads, citations and take-up of the ideas described in the papers.

Do you agree?

The quality assurance approaches or metadata we described in the papers was focussed primarily on the service providers. However, six years later, the importance of the quality of metadata for resource discovery is no longer just of relevance to service providers. In a Web 2.0 environment in which content providers can make their teaching and learning and research outputs available on a wide range of services without the mediation of information professionals there is a need to ensure that a wider range of content providers are aware of risks that poor quality metadata can lead to valuable content being difficult to find.

I became aware of such risks while Surveying Russell Group University Use of Google Scholar Citations which I described in a recent blog post.  As mentioned in the post I became aware of the dangers of over-counting the numbers of researchers who have claimed a profile by aggregating researchers from the University of Birmingham with those from the University of Birmingham at Alabama or those from Newcastle University with Newcastle University,  New South Wales.

 Of further investigation I discovered entries from researchers who had misspelt the name of their university by using “univeristy” – a common typo which I myself have made. Currently it seems there are only 33 such misspellings.

A recent post gave some “Thoughts on Google Scholar Citations“. I concluded by suggesting that researchers could find it useful to claim their account on Google Scholar Citations and  ensure that the details of their papers are accurate but speculated on whether there would be barriers to researchers doing this. In order to investigate the level of usage of Google Scholar Citations in the UK higher education sector a survey of its usage across the twenty Russell Group Universities has been carried out and the findings published in this post. The institution’s name, as listed in the first column, was used as a search term.  The number of entries gives the current number of researchers found, with a link provided to the current final page of results.  In addition in order to investigate whether the service is being used by new researchers, who are likely to have a low number of citations or well-established researchers with large numbers of citations, a summary of the top three researchers having the largest numbers of citations is give, with links to the researchers profile together with details of the numbers of citations for the three researchers having the lowest numbers of citations. The results are given in the following table.  The survey was carried out on Tuesday 22 November 2011

* It was noted that the first entry for a search for the University of Birmingham referred to Mary Vignolo Wheatley from the University of Alabama at Birmingham. The numbers of Google Scholar Citation entries is therefore overstated for the University of Birmingham and potentially for the other institutions which are listed. ** I was informed after publication of this post that of the 44 citations quoted for Newcastle, 11 are actually for the University of Newcastle, NSW, Australia. Such errors could creep in for other institutions for which there are name clashes (e.g. York University and New York University). This highlights the need for globally unique institutional identifiers – but such discussions are outs the scope of this post. It was also noticed that the third entry for the University of Cambridge referred to Alan Turing, the English mathematician, logician, cryptanalyst, and computer scientist who, as described in Wikipedia, lived from 1912-1954.  Unsurprisingly his Google Scholar Citation entry states that his email address has not been verified!

Discussion

In a recent discussion about Google Scholar Citations I have been told about the difficulties in claiming authorship of papers after one has left one’s host institution and no longer has an institutional email address.  A second discussion I heard from one person who claimed his Google Scholar  account shortly before leaving his host institution who provided an alternative email account which could be used one his institutional email account had been deleted. The first example highlights a potential difficulty in asserting authorship of papers after one has left the host institution and the second example describes one way in which such potential problems can be addressed.  It would therefore appear sensible for researchers to claim a Google Scholar account while they are in a position to associate it with papers published in their host institution. An interesting issue, therefore, will be who should take responsibility for advising researchers on best practices for using services such as Google Scholar Citations.  Should the library include such advice in its training courses for new researchers?

Conclusions

A recent post by Wouter Gerritsma, subject librarian and bibliometrician at Wageningen UR Library described “How Google Scholar Citations passes the competition left and right“. Wouter’s post concluded:

Google Scholar is only about five years old. Give them another five years and they will have changed the market for abstracting and indexing database totally. If only 20 percent of all scientists make their publication lists correct (also editing of the references which can be done to improve the mistakes Google has made) even without making them publically available, Google sits on a treasure trove of high quality metadata. Really interesting to see how this story will develop.

It will be interesting to see how this story develops.  And as the launch of Google Scholar Citations was only announced a week ago today, we do have an opportunity to observe its take-up within our institutions from its early days.  Monitoring the take-up of the service, the approaches taken in managing the information and understanding difficulties in such management activities will be valuable not only in developing plans for use with other services in this space. Hmm, I wonder if Google Scholar Citations has APIs which will enable such monitoring approaches to be implemented in a scalable way?

I am involved in work on looking at ways in evidence-based approaches can make use of metrics in order to understand best practices and demonstrate impact. A series of surveys have been carried out which have sought to gather quantitative evidence of use of a variety of services and, by publishing the findings on this blog, have encouraged discussions about the approaches.

This work complements the report on Splashes and Ripples: Synthesizing the Evidence on the Impacts of Digital Resources carried out by the Oxford Internet Institute and described on their blog which focussed on “synthesizing the evidence available under the JISC digitisation and eContent programmes to better understand the patterns of usage of digitised collections in research and teaching“.

Although my work has avoided addressing the complexities of metrics for research a recent survey entitled A Pilot Survey of the Numbers of Full-Text Items in Institutional Repositories has sought to profile the institutional repositories hosted by Russell Group Universities in order to have a better understanding of patterns of usage related to deposits of full-text items which would appear to be of importance in a repository is to have a role to play in the digital preservation of research papers.

Surveys of the numbers of downloads of papers from a repository is clearly a flawed approach if one is attempting to determine the quality, impact and value of research. But are there other insights to be gained from examining download statistics for an institutional repository? This latest survey, which is being carried out a few days before a workshop on “Metrics and Social Web Services: Quantitative Evidence for their Use and Impact“, will seek to understand whether new insights can be gained from a lightweight survey of the most popular downloads from the University of Bath’s Opus institutional repository.

Survey of Downloads

The University of Bath’s institutional repository, which I’ll refer to by the name “Opus”, is, like many UK University repositories, based on the ePrints software. A stats module, IRStats, seems to be provided as standard with ePrints although, as discussed in a previous post, the data which is gathered can by configured by the repository manager.

Opus currently has a total of 136,347 downloads since its launch in 2005. Looking at the histogram of monthly downloads we can see a slow growth for five months after the launch and then a plateau. Zooming in on the graph we can see growth in the numbers of downloads taking place in October/November 2009 and 2010 – and we might reasonably expect a similar pattern to be repeated when the next academic year begins.

But who are the authors of the most downloaded papers and might we be able to discover and techniques which can help to ensure that papers are downloaded ?

Looking at the top ten downloaded authors we find that the conferences proceedings of the 11th International Conference on Non-conventional Materials and Technologies, NOCMAT 2009 is in the top place with 28,449 downloads – an order of magnitude more than the item in second place.

The next most popular item is The use of QR codes in Education: A getting started guide for academics (2,514 downloads) by Andy Ramsden, former head of the e-Learning Unit who used to work in the office down the corridor from me. Andy has two other paper in the top ten, related to his elearning interest in QR codes (1,161 downloads) and Twitter (805 downloads). I am in third place, with my paper on Library 2.0: balancing the risks and benefits to maximise the dividends having 1,419 downloads. The other most popular downloaded papers seem to be PhD theses, with the exception of my UKOLN colleague Alex Ball whose project report on Review of the State of the Art of the Digital Curation of Research Data. is in tenth place (with 745 downloads).

Is there a pattern emerging, I wonder, or are these just one-off examples. It would be interesting to see what the evidence from a wider profile of downloads may indicate. Looking at the top ten authors pages we find A. Ramsden has had 7,760 items downloaded; B. Kelly (6,758); A.D Brown (3,323); A. Ball (2,267); S. J. Culley (2,250); S. Deneulin (1,900); S. Abdullah (1,535); E. Dekoninck (1,469); E. W. Elias (1,469); J. Millar (1,439) and L. Jordan (1,161). [Note that the items do not seem to total correctly in all cases so I will omit the links until I've tried to resolve this].

Discussion

As mentioned previously it is important to note that downloads have relevance to quality – it would probably be timely now to point our that the numbers of readers of the News of the World demonstrate that quite clearly! However if we also acknowledge that researchers do have a responsibility to get their message across, then researchers will (should) have an interest in maximising the numbers of (appropriate) readers of their papers – and it is important, I feel, to highlight the need to engage with appropriate readers.

From the survey it seems that the authors who have papers in the top ten institutional downloads are also successful in having other papers also being downloaded in significant numbers. Perhaps having an office on level 5 of the Wessex House building may be a reason for such popularity of the papers! On the other hand it may be that the three of us who shared the same corridor discussed dissemination strategies or perhaps, and more likely, are simply working in an area (related to digital libraries) in which potential readers of our papers are more likely to access digital repositories.

A recent post provided A Pilot Survey of the Numbers of Full-Text Items in Institutional Repositories. The survey made use of the advanced search functionality of ePrints repository software in order to gather data on the numbers of full-text items. Unfortunately it was found that most repositories had not configured the software to provide such information. Whilst exploring the advanced search features it was noticed that it was possible to provide searches based on file formats. This would appear to provide an answer to the question of formats used for depositing items in repositories and how, for example, this relates to preservation policies.

Survey Across Russell Group University Repositories

Testing Approach

In order to test the approach the advanced search facility for the ECS repository at the University of Southampton was used. The figure for the total number of items used the same search option as described in the previous post. Details of the number of HTML items, PDF or Postscript items, other formats and the total number of formats were obtained and links to the findings included so that the current status can be obtained (which also had the advantage of documenting the search parameters used). The findings are given in the following table.

It should be noted that there are differences between the total number of full-text items and the total of all formats. I am assuming that the number of full-text items will be equal to or less than the total number of items in a repository but the total number of items could be larger if there are multiple formats for a single item.

It should be noted that in this survey a link is provided to the policy statement for the repository which has been taken from the ROARMap summary of IR policies. In this example the implementation of the following policy statement might be demonstrated by the evidence presented:

It is our policy to maximise the visibility, usage and impact of our research output by maximising online access to it for all would-be users and researchers worldwide. 

Survey

Once again a survey of the institutional repositories for Russell Group Universities was carried out. The results are given in the following table, which this time includes a link to the IR policies. The table below gives the results of the findings. Note that the results were gathered using the public advanced search interface where this was available. If information on the numbers of full-text items becomes available I will update this post and annotate accordingly.

NOTE: The entry for the University of Manchester was updated on 15 June 2011, the day after the post was published, using information provided in a comment to the post. Since this information was gathered in a different way to the other findings (which used the ePrints advanced search function) the findings may not be directly comparable.

It should also be noted that:

• The total number of full-text items listed in column 3 is taken from the findings for the ‘Full Text Status’ search. If this option is not available the entry is left blank.
• The totals listed in columns 3-7 is taken from the findings for the “Format” search option with column 6 including both PDF and Postscript options.
• The ‘Other formats’ totals listed in column 8 is taken from the findings for the “Format” search option using the options not included in columns 3-7. Note that this includes MS PowerPoint and Excel and various image, video, audio, XML and archive formats.
• The totals listed in column 9 is taken from the findings for the “Format” search option with all format options selected.
• The link to the policy page in column 10 is taken from entries provided in the ROARMap summary of IR policies. If no information is provided the entry is listed as “Not available”.

Comments

It seems that information on the file formats of items stored in institutional repositories is not easily obtained from using the advanced search option in ePrints software. From the previous survey we found that only IRs hosted at Liverpool and LSE of the 20 Russell Group Universities provided information on the numbers of full-text items. From this survey we find that Liverpool also provides information on the file formats used, but LSE does not. However Cardiff, Glasgow and Southampton Universities do provide information on the file formats used, so a better picture across this sector can be obtained.

It is clear that PDF/PostScript is the most popular format used for depositing items with very little evidence that HTML items are deposited. This will be disappointing for those who feel that the structure and ease of reuse provided by HTML should outweigh the convenience provided by PDF. Similarly the low usage of MS Word will be of concern to those who feel that the master format of a resource should be deposited rather than a lossy format such as PDF – although since PDF has been standardised by ISO it could be argued that depositing items in an open standard format reflects best practices.

In some respects the details of the widely-used formats is lost in the ‘Other formats’ column. This includes various multimedia formats, but also archive formats: ZIP, TGZ and BZ2. Perhaps we may find that there are large numbers of ZIP archives containing MS Word, PDF and HTML versions of resources.

I feel there is a need for further data mining in order to understand the patterns of usage which are emerging across institutional repositories, how such patterns relate to policies (including access and preservation policies) and the implementation difficulties which depositers may experience in uploading items to their institutional repository. The danger is that we may develop an improved format (Scholarly HTML, perhaps) but fail to understand the current barriers to depositing full-text items.

Conclusions

A comment to my post which argued that Numbers Matter: Let’s Provide Open Access to Usage Data and Not Just Research Papers suggested that the term ‘Paradata’ would be appropriate to use. As described on Wikipedia “the paradata of a survey are data about the process by which the survey data were collected” or alternatively “administrative data about the survey“. The term has been used in a CETIS Wiki page on “Generating Paradata from MediaWiki” which refers to a NSDL Community Network page which proposes how the term can be applied in the context of educational resources and suggests that paradata can provide opportunities to “explicates usage patterns and inferred utility of resources“.

Repository managers have a clear need to understand usage patterns and how their resources can be reused. Since repositories are also closely linked with the open access agenda it would seem to be self evident that repository ‘paradata’ should be published openly - after all, if repository managers are promoting the benefits of open access to research publications to their researchers their arguments will be undermined  if they fail to publish data under their control, where there should be no complexities of copyright ownership claimed by publishers.

It seems that there repositories which use DSpace do not provide advanced search capabilities similar to those available in ePrints. Perhaps this might be a reason for the lack of data from such repositories. But for those who are lucky enough to be using ePrints, what reasons can there be for not providing a full range of statistics?

Twitter conversation from Topsy: [View]

A recent post on How Do We Measure the Effectiveness of Institutional Repositories? sought to address the question of “What makes a good repository?” which was raised on the JISC-Repositories JISCMail list. The post outlined possible metrics which could be used for identifying the effectiveness of institutional repositories based on the intended purposes of a repository. In the post I suggested that if the purpose of a repository was to ensure the long-term preservation of resources, then there was a need to measure the number of full-text items in the repository – after all if the full text of a paper is not available the repository won’t be doing a very good job in the preservation of such resources!

The interest in this topic was revisited yesterday in a Twitter discussion which began with the suggestion from @PaulWalk that “I’ve thought we should use RepUK to measure actual persistence in repositories“‘. But in order to measure the persistence of of the actual resource we need to be able to differentiate between the persistence of the full-text item and the resource itself and not just the persistency of the URI of the item. How might one do this?

Initial Experimentation

Following a discussion with Les Carr at the JISC 2011 conference I discovered that the ePrints advanced search interface can be used to retrieve information on both the numbers of items containing the full text and those that do not. In order to see if this approach could be used I looked at UKOLN’s items in Opus, the University of Bath’s institutional repository. From this I found that there were a total of 344 items, of which 146 full text items were available (including published and confidential items) and 198 are metadata-only items. We can see that 42% of the items contain the full-text.

In order to see if this this use of ePrint’s advanced search could be used in a similar fashion for another repository I looked at the ECS ePrint Repository at the University of Southampton. This time I found that out of a total of 974 15,532  items the departmental repository contained 861 8,429 items with the full text and 113 7.093 metadata-only items – this time 54.3% of items contain the full-text.

But are these initial findings typical across the sector?

Survey Across Russell Group University Repositories

We might expect the 20 research-intensive Russell Group Universities to be playing a leading role in use of institutional repositories, with either institutional mandates (in the case of Southampton University) or institutional research culture helping to ensure that significant numbers of full-text items are deposited. But is this really the case? In order to investigate whether the approach described could be applied more widely the survey was carried out across Russell Group Universities.

Using the list of repositories taken from the OpenDOAR directory I found that 3 of the Russell group Universities seem to use the DSpace repository software and the advanced search functional in DSpace does not appear to allow searching to be restricted to full-text and metadata-only records.

Subsequent investigation of the advanced search capabilities of the remaining 17 institutions showed that only two seemed to provide the advanced search function which I used on the University of Bath and ECS, University of Southampton repositories. However there is a RESTful interface to the search and so the search parameters used to search the University of Bath repository was used across the other ePrint repositories. The following searches were carried out:

Query 1: Total Number of Items

http://eprint.domain/cgi/search/quicksearch?screen=Public%3A%3AEPrintSearch&basic_merge=ALL&basic=web&full_text_status=public&full_text_status=restricted&full_text_status=none&groups_merge=ALL&satisfyall=ALL&order=-date%2Fcreators_name%2Ftitle&_action_search=Search

Query 2: Full text deposited (but access may be restricted)

http://eprint.domain/cgi/search/quicksearch?screen=Public%3A%3AEPrintSearch&basic_merge=ALL&basic=web&full_text_status=public&full_text_status=restricted&groups_merge=ALL&satisfyall=ALL&order=-date%2Fcreators_name%2Ftitle&_action_search=Search

Query 3: No full text available:

http://eprint.domai/cgi/search/quicksearch?screen=Public%3A%3AEPrintSearch&basic_merge=ALL&basic=web&full_text_status=none&groups_merge=ALL&satisfyall=ALL&order=-date%2Fcreators_name%2Ftitle&_action_search=Search

It was intended to use the survey methodology across the Russell Group universities which host an institutional repository based on the ePrints software. However it was not possible to get valid results for most of the repositories and it was subsequently discovered that this is an optional feature for ePrints repositories.

Rather than abandon this work I have decided to publish this post in order to encourage institutions which host an ePrints repository to implement this feature since I feel it would be beneficial to the repository community if we had a better picture of how institutions are using repositories to host full-text items.

The table below gives the results of the two test cases (from Bath and Southampton) together with details of the total number of items in the other repositories. If information on the numbers of full-text items becomes available I will update this post and annotate accordingly. [Note there was an error in the figures for the ECS repository. This has now been corrected in the table below.]

The table below gives the results of the findings for what seems to be the main repository from Russell Group Universities. Note that the results were gathered using the public advanced search interface where this was available. If information on the numbers of full-text items becomes available I will update this post and annotate accordingly.

At the time of writing we have to say that we do not know how many of the 183,299 items contain the full-text. All we can say is that there are at least 5,175 full-text items (or only 2.8%) – and this is based on the assumption that a full-text item represents the content of the metadata item, rather than for example, a PowerPoint slide used in the presentation of a paper.

An Opportunity for Developers

I should also like to point out that, as described on the DevCSI blog, the deadline for the Developer Challenge at Open Repositories 2011 (Austin, Texas) is Thursday 9 June. A CrowdVine page for the developer challenge describes how the Challenge is to “Show us the future of repositories“. Since “Remote presentations would be considered in exceptional circumstances” it strikes me that there might be an opportunity to submit an entry based on an analysis of the percentage of full-text items in repositories, but this would probably have to be done using an alternative approach. A suggestion for anyone who wold like to submit an based on this idea could be:

The future of repositories is to preserve the full text of research papers for future generations. We can see how well we are doing in implementing this vision which shows that xx% of repositories across the y sector already contain full-text items :-)

Or, if the results are disappointing:

The future of repositories is a gloomy one as only y% of repositories across the z sector contain full text items :-(

Alternatively we might conclude that new development is not required for those running ePrint repositories:

The future of repositories is reliant on the provision of evidence which can be used to policies and so ePrints repository managers should configure their services to provide the evidence describes in this post!

Is that an unreasonable suggestion?

Twitter conversation from Topsy: [View]

I was recently told about the Iamresearcher.com service, a repository of information about researchers and their research activities. “Not another one!” was one reaction I heard. But is there anything that can be learnt from this service, which has been developed by Mr Yang Yang, an MSc student at the University of Southampton? Les Carr, over on his Repository Man blog has been “Experimenting With Repository UI Design” and describes how he is “always on the lookout for engaging UI paradigms to inspire repository design“. Might this service provide any new UI design paradigms?

Things I Like

I have to admit that I was pleased with how easy it was to get started with the service. I signed up and asked the system to find papers associated with my email address. It found many of my papers, with much of the metadata being obtained from the University of Bath Opus repository. I them searched for other papers which weren’t included in the initial set and was able to claim them as belonging to me – including one short paper which had been published in the Russian Digital Libraries Journal in 2000 which I had forgotten about.

I can now view my 49 entries and sort them in various ways: in addition to the default date order I can also sort by item type; lead author; co-authors and keywords. The view of my co-authors (illustrated) was of particular interest: I hadn’t realised that I had written papers with 55 others.

In comparison the interface provided on my institutional repository service does now seem quite dated. However this is perhaps not unexpected as according to the Wikipedia entry the ePrints software (which is widely used across the UK) was created way back in 2000.

Revisiting the question as to whether we need another service which provides access to research information I would say ‘yes’. Such developments can help drive innovation. In this case ePrints developers are in a position to see more modern approaches to the user interface. In addition the service describes itself as “Web 3.o ready application” by which they seem to mean that the service “connects researcher and research students anywhere in the world using an intelligent network”.

I haven’t seem much evidence of Web 3.0 capabilities in the service, apart from being able to download details of my papers in FOAF format, but perhaps the “ready” word is providing a signal that such functionality is not yet available.

Things I Don’t Like

There are some typos on the data entry forms and some usability niggles, but nothing too significant – indeed after attending a recent Bathcamp Startup Night and hearing the suggestion that “If you’re not embarrassed about the launch version of your software then you released it too late” (a quote from the founder of LinkedIn) I welcome seeing this service before everything has been thoroughly checked.

The language used in the terms of service are somewhat worrying, however:

No Injunctive Relief.
In no event shall you seek or be entitled to rescission, injunctive or other equitable relief, or to enjoin or restrain the operation of the Service, exploitation of any advertising or other materials issued in connection therewith, or exploitation of the Services or any content or other material used or displayed through the Services.

It also seems that as a user of the service I undertake not to:

Duplicate, license, sublicense, publish, broadcast, transmit, distribute, perform, display, sell, rebrand, or otherwise transfer information found on iamResearcher (excluding content posted by you) except as permitted in this Agreement, iamResearcher’s developer terms and policies, or as expressly authorized by iamResearcher

Hmm. The service harvested its metadata from other repository services, such as the University of Bath’s Opus repository but does not allow others to reuse its content. This seems to undermine the benefits provided by permitting (indeed encouraging) others to make use of open data. In addition the service appears to be hypocritical, as the University of Bath’s repository policy (which was created using the OpenDOAR Policy tool) states that “The metadata must not be re-used in any medium for commercial purposes without formal permission“. Now the Iamresearcher.com service does not appear to be a commercial service – but its privacy policy states that “To support the Services we provide at no cost to our Users, as well as provide a more relevant and useful experience for our Users, we serve our own ads and also allow third party advertisements on the site“. If advertising does appear on the service, won’t it then be breaching the terms and conditions of the service from which it harvested its data?

Personally I have no problem with advertising being used to fund services where, as in this case, there are multiple providers of services. Indeed those who argue for openness of data should be willing to accept that data may be used for commercial purposes. However services which accept the opportunities provided by open data should accept that they should be providing similar conditions of usage.

The final concern that I have about the service is that currently it can only be accessed if you sign in. I feel this is counter-productive – indeed one person I mentioned this service to asked why he should bother. That’s a fair comment, I think. And seeing that the terms and conditions also state that users of the service are not allowed to:

Deep-link to the Site for any purpose, (i.e. including a link to a iamResearcher web page other than iamResearcher’s home page) unless expressly authorized in writing by iamResearcher or for the purpose of promoting your profile or a Group on iamResearcher as set forth in the Brand Guidelines;

I now wonder what benefits this service can provide to the research community. Developers of other repository services, however, should be able to learn from the technological enhancements the service provides, even if the business model is questionable.

The Need for Metrics

How might one measure the effectiveness of an institutional repository? An approach which is arising from various activities I am involved in related to evidence, value and impact is based on the need to identify the underlying purpose(s) of services and to gather evidence related to how such purposes are being addressed.

Therefore there is a need to initially identify the purposes of an institutional repository. Institutions may have a variety of different purposes (which is why, although gathering evidence can be important, drawing up league tables is often inappropriate). But let’s suggest that two key purposes may be: (1) maximising access to research publications and (2) ensuring long-term preservation of research publications. What measures may be appropriate for ensuring such purposes are being achieved?

For maximising access to research publications two important measures will be the numbers of items in the repository and the numbers of accesses to the items. Since the numbers themselves will have little meaning in isolation there will be a need to measure trends over time, with an expectation of growth in the numbers of items deposited (which show slow down once legacy items have been uploaded and only new items are being deposited) and continual increase in overall the traffic to the repository as the number of items grows and access to the items via various resource discovery services provides easier ways of findings such resources.

Access Statistics for Institutional Repositories

The relevance of such statistics is well-understood with, here at the University of Bath, the IRStats module for the ePrints repository service providing access to information such as details of all downloads, the overall number of downloaded items (100,003 at the time of writing), the trends over time and various other summaries, as illustrated.

However it is important to recognise that such measures only indirectly provide an indication of how well a repository may be doing in maximising access to research publications. In part traffic may be generated by users following links to content of no interest to them through use of search engines such as Google (which is responsible for providing 38% of traffic to the University of Bath repository, with another 10.2% arriving via Google Scholar). In addition even if a relevant paper is found and read, the ideas it contains may not be felt to be of direct interest and may not be used to inform subsequent research activities.

A citation to a resource will provide more tangible evidence of direct benefits of a repository to supporting research activities and work such as the MESUR metrics activity is looking to “investigate an array of possible impact metrics that includes not only frequency-based metrics (citation and hit counts), but also network-based metrics such as those employed in social network analysis and web search engines“. However in this post I will focus on evidence which can be easily gleaned from repositories themselves.

Whilst it is possible to point out various limitations in using such metrics the danger is that we lose sight of the fact that they can still have a role to play in providing a proxy indicator of value. So although repository items which are found and downloaded may not be of interest or may not be used, other items will be relevant and inform, either directly or indirectly, other research work. We might therefore assert that an increase in traffic may also have a positive correlation with an increase in use.

The Numbers of Items in Repositories

Measuring the numbers and growth in numbers of items in a repository would seem to be less problematic than access statistics. This measurement can reflect the effectiveness of a repository’s aims in supporting the preservation of research publications, as publication are migrates from departmental Web sites or individual’s personal home pages to a centrally managed environment. The growth in the numbers of items should also, of course, help in enhancing access to the papers too.

Repositories may, however, only provide access to the metadata about a paper and not access to the paper itself. This may be due to a number of factors including copyright restrictions, (perceived) difficulties in uploading document or the unavailability of the documents.

There may also be a need to be able to differentiate between the total number of distinct items in a repository and the numbers of formats which may be made available. Storage of the original master format is often recommended for preservation purposes and if ease-of-reuse of the content may be required (e.g. merging together various papers and producing a table of contents can be much easier if the original files are available, rather than a series of PDFs which can be more difficult to manipulate.

Alternative formats for items may also help to enhance access for users of mobile devices or users with disabilities who may require assistive technologies to process repository items. This then leads to the question of not only the formats provided but how those formats are being used: is a PDF easily processed by assistive technology or is it simply a scanned image which cannot be read by voice browsers? In addition, as suggested by preliminary research carried out by my colleagues Emma Tonkin and Andy Hewson described in a post on “Automated Accessibility Analysis of PDFs in Repositories“, might the cover pages automatically generated by repository systems created additional barriers to access of such resources?

Trends Across the Community

This post has outlined areas in which evidence should be gathered and used in order to be able to help demonstrate the value of an institutional repository service and help to ensure that a number of best practices are being addressed (and, if not, to be able to develop plans for implementing such best practices).

Although such work should be done within the context of an individual repository service there are also benefits to be gained from observing trends across the community. My colleague Paul Walk recently mentioned on the JISC-Repositories JICMail list UKOLN development of a prototype harvesting and aggregation system for metadata from UK Institutional repositories called ‘RepUK’. One aspect of this work is aggregation of metadata records from institutional repositories and visualisation of various aspects of the data quality. Mark Dewey, lead developer for this work, has released an initial prototype tool. As can be seen this can provide a visualisation of the growth in the number of records across the 133 repositories which have been harvested.

Discussion

This post has suggested that metrics are needed in order to help to provide answers, perhaps indirectly, to questions regarding the effectiveness of institutional repositories as well as to support and inform the development of the repositories and the adoption of best practices. Of course measuring the effectiveness of institutional repositories will also require user surveys, but this post only considers quantitative approaches which are summarised in the table below.

But what additional tools may be needed (I would welcome a mobile app for my iPod Touch along the lines of the stats app for WordPress blogs)?  What advice is needed in interpretting the findings (and avoiding misinterpretations?)  Your thoughts are welcomed.

Twitter conversation from Topsy: [View]

Back in December 2006 I wrote a post on Accessibility and Institutional Repositories in which I suggested that it might be “unreasonable to expect hundreds in not thousands of legacy [PDF] resources to have accessibility metadata and document structures applied to them, if this could be demonstrated to be an expensive exercise of only very limited potential benefit“. I went on to suggest that there is a need to “explore what may be regarded as ‘unreasonable’ we then need to define ‘reasonable’ actions which institutions providing institutional repositories would be expected to take“.

A discussion on the costs and complexities of implementing various best practices for depositing resources in repositories continued in September 2008 as I described in a post on Institutional Repositories and the Costs Of Doing It Right, with Les Carr suggesting that “If accessibility is currently out of reach for journal articles, then it is another potential hindrance for OA“. Les was arguing that the costs of providing accessibility resources in institutional repositories is too great and can act as a barrier to maximising open access to institutional research activities.

I agree – but that doesn’t mean that we should abandon any thoughts of exploring ways of enhancing accessibility. A paper on “From Web Accessibility to Web Adaptability” (available in PDF and HTML formats) described an approach called “Web Adaptability” which has the flexibility to account for a variety of contextual factors which is not possible with an approach based purely on conformance with WCAG guidelines. An accompanying blog post which summarised the paper described how the adaptability approach could be applied to institutional repositories”:

Adaptability and institutional repositories: Increasing numbers of universities are providing institutional repositories in order to enhance access to research publications and to preserve such resources for future generations. However many of the publications will be deposited as a PDF resource, which will often fail to conform with accessibility guidelines (e.g. images not being tagged for use with screen readers; text not necessarily being ‘linearised’ correctly for use with such devices, etc.). Rather than rejecting research publications which fail to conform with accessibility guidelines the Web adaptability approach would support the continued use and growth of institutional repositories, alongside an approach based on advocacy and education on ways of enhancing the accessibility of research publications, together with research into innovative ways of enhancing the accessibility of the resources.

The stakeholder approach to Web accessibility, originally developed by Jane Seale for use in an elearning context and described in a joint paper on Accessibility 2.0: People, Policies and Processes (available in PDF, MS Word and HTML formats) has been extended for use in a repository context. The approaches to engagement with some of the key stakeholders is given below:

Education: Training provided (a) for researchers to ensure they are made aware of importance of accessibility practices (including SEO benefits) and of techniques for implementing best practices and (b) for repository managers and policy makers to ensure that accessibility enhancements can be procured in new systems.

Feedback to developers: Ensure that suppliers and developers are aware of importance of accessibility issues  and enhancements featured in development plans.

Feedback to publishers: Ensure that publishers who provide templates are aware of importance of provision of accessible templates.

Auditing: Systematic auditing of papers in repositories to monitor extent of accessibility concerns and trends.

But is this approach valid?  Surely SENDA accessibility legislation requires conformance with WCAG guidelines? And if it is difficult to conform with such guidelines, surely the best approach is to keep a low profile?

BS 8878 Web Accessibility Code of Practice

The BS 8878 Web accessibility Code of practice was launched in December 2010.  A summary of an accompanying Webinar about the Code of Practice was described in a post on BS 8878: “Accessibility has been stuck in a rut of technical guidelines” – and it was interesting to hear how the code of practice has been written in the context of the Equal Act which has replaced the DDA.  I was also very pleased to hear of the user-focus which is at the heart of the code of practice, and how mainstream approaches on best practices have moved away from what was described as a “rut of technical guidelines“.

Although the Code of Practice is not available online and costs £100 to purchase an accompanying set of guidelines was produced by Abilitynet which I have used in the following summary. Note I had to request a copy of these guidelines and I can no longer find the link to contact details to request copies. However AbilityNet’s complete set of guidelines can be purchased for £4,740!

It seems that there is a clear financial barrier to the implementation of new accessibility guidelines. In order to minimise the costs to higher education (which would approach a quarter of a million pounds if all UK Universities were to purchase a copy at the list price!)  I’ll give my interpretation of how the code of practice could be applied in the context of institutional repositories. But please note that this is very much an initial set of suggestions and should not be considered to be legal advice!

The heart of the BS 8878 document is a 16 step plan:

1. Define the purpose.
2. Define the target audience.
3. Analyse the needs of the target audience.
4. Note any platform or technology preferences.
5. Define the relationship the product will have with its target audience.
6. Define the user goals and tasks.
7. Consider the degree of user experience the web product will aim to provide.
8. Consider inclusive design & user-personalised approaches to accessibility.
9. Choose the delivery platform to support.
10. Choose the target browsers, operating systems & assistive technologies to support.
11. Choose whether to create or procure the Web product.
12. Define the Web technologies to be used in the Web product.
13. Use Web guidelines to direct accessibility Web production
14. Assure the Web products accessibility through production (i.e. at all stages).
15. Communicate the Web product’s accessibility decisions at launch.
16. Plan to assure accessibility in all post-launch updates to the product.

Note that Step 13, which covers use of WCAG guidelines, may previously have been regarded as the only or the most significant policy item. BS 8878 places these guidelines in a more appropriate context.

Using BS 8878 for Institutional Repositories

A summary of how I feel each of these steps might be applied to institutional repositories is given below.

1. Define the purpose:
   The purposes of the repository service will be to enhance access to research papers and to support the long term preservation of the papers.
2. Define the target audience:
   The main target audience will be a global research community.
3. Analyse the needs of the target audience:
   Researchers may need to use assistive technologies to read PDFs.
4. Note any platform or technology preferences:
   PDFs may not include accessibility support.
5. Define the relationship the product will have with its target audience:
   The paper will be provided at a stable URI.
6. Define the user goals and tasks:
   Users will use various search tools to find resource. Paper with then be read on screen or printed.
7. Consider the degree of user experience the web product will aim to provide:
   Usability of the PDF document will be constrained by publisher’s template. Technical accessibility will be constrained by workflow processes.
8. Consider inclusive design & user-personalised approaches to accessibility:
   Usability of the PDF document will be constrained by publisher’s template. Technical accessibility will be constrained by workflow processes.
9. Choose the delivery platform to support:
   Aims to be available on devices with PDF support including mobile devices
10. Choose the target browsers, operating systems & assistive technologies to support:
    All?
11. Choose whether to create or procure the Web product:
    The service is provided by repository team.
12. Define the Web technologies to be used in the Web product:
    HTML interface to PDF resources.
13. Use Web guidelines to direct accessibility web production:
    HTML pages will seek to conform with WCAG 2.0 AA. PDF resources may not conform with PDF accessibility guidelines.
14. Assure the Web products accessibility through production (i.e. at all stages):
    Periodic audits of PDF accessibility planned.
15. Communicate the Web product’s accessibility decisions at launch:
    Accessibility statement to be published.
16. Plan to assure accessibility in all post-launch updates to the product:
    Periodic reviews of technical developments.

Step 15 requires the publication of an accessibility statement, which “states in an easy to understand and non-technical way the accessibility features of the site and any known limitations“. This will be the aspect of the accessibility work which will be visible to users of the service. But what might such an accessibility statement cover?

Current Approaches to Accessibility Statements for Repositories

The first step to answering this question was to see what accessibility statements are currently provided for institutional repositories.  An analysis of the first page of results for a Google search for “repository accessibility statement” provided only a single example of an accessibility statement for an institutional repository. This was provided by UBIR, the University of Bolton Institutional Repository and appears to be a description of WCAG conformance for the repository Web pages rather than the contents of the Web site :

Standards Compliance

1. All static pages follow U.S. Federal Government Section 508 Guidelines.
2. All static pages follow priorities 1 & 2 guidelines of the W3C Web Content Accessibility Guidelines.
3. All static pages validate as HTML 4.01 Transitional.
4. All static pages on this site use structured semantic markup. H2 tags are used for main titles, H3 and H4 tags for subtitles.

The Google results for other institutional repositories, including UEA and the University of Salford Informatics Research Institute Repository (USIR) were based on links to standard accessibility statements for the institutional Web site, with the statement for the University of Salford, for example, stating that:

The University of Salford strives to ensure that this website is accessible to everyone. If you have any questions or suggestions regarding the accessibility of this site, or if you come across a page or resource that does not meet your access needs, please contact the webmaster@salford.ac.uk, as we are continually striving to improve the experience for all of our visitors.

It seems that the contents of an institutional repository, the core purpose, after all,  of a repository, do not appear to have statements regarding the accessibility of such contents.  I will admit that I have only had a cursory exploration for such statements and would love to be proved wrong.  But for now let’s assume that the accessibility statement required for step 15 of BS 8878 will have to be produced from scratch.

A Possible Accessibility Statement For An Institutional Repository

Might the following be an appropriate statement for inclusion on an institutional repository?  Please note that I am not a repository manager so I don’t know if such a statement is realistic.  However I should also add that I have deposited 46 of my papers and related articles in the University of Bath repository and am aware of some of the difficulties in ensuring such items will conform with accessibility guidelines for PDFs, MS Word and HTML, the main formats used for depositing items.   Since it is likely to be difficult for the motivated individual author to address accessibility concerns for their own items, we cannot expect best practices to be applied for the 1,568 items deposited in 2010, never mind items deposited before then.

It is therefore not realistic to suggest that authors or repository managers should simply implement the advice on producing accessible PDFs provided by organisations such as JISC TechDis.  Rather the accessibility statement needs to be honest about the limitations of the service and difficulties which people with disabilities may have in accessing items hosted in institutional repositories.

The following draft accessibility statement is therefore suggested as providing a realistic summary regarding the accessibility of a typical repository service.

I hope this has provided something to initiative a discussion on ways in which institutional repositories can address accessibility issues which can provide barriers to researchers with disabilities and build on the successes repositories are having in addressing access barriers providing by copyright issues, complex business models and fragmented resources which may be difficult to find and retrieve.