My Response to WAI’s Website Accessibility Conformance Evaluation Methodology 1.0 Working Draft
Posted by Brian Kelly on 19 October 2012
Last week in a post entitled W3C WAI Invite Feedback on Website Accessibility Conformance Evaluation Methodology 1.0 Working Draft I highlighted the publication of WAI’s Website Accessibility Conformance Evaluation Methodology 1.0 working draft and encouraged readers to respond to the call for feedback.
The closing date for comments is tomorrow, 20 October 2012. I have submitted my comments which are given below.
Response to the WCAG-EM 1.0 Working Draft
The Web Accessibility Initiative’s work in providing guidelines which can help enhance the accessibility of Web resources for people with disabilities since WAI’s launch in 1997  is to be valued.
However, as might be expected (and is the case with many of the standards which have been developed over the years by W3C), the various guidelines which have been produced by WAI have shown to have limitations or proven inappropriate for use in a real-world context. Accessibility researchers and practitioners based primarily in the UK have been pro-active in identifying limitations of the WAI model and proposing ways in which the guidelines can be contexualised and used where appropriate. This work dates back to 2005 when a paper entitled “Forcing Standardization or Accommodating Diversity? A Framework for Applying the WCAG in the Real World” was presented at the W4A 2005 conference . Further work included papers on Contextual Web Accessibility – Maximizing the Benefit of Accessibility Guidelines , Accessibility 2.0: People, Policies and Processes , One World, One Web … But Great Diversity , From Web Accessibility to Web Adaptability , Developing Countries; Developing Experiences: Approaches to Accessibility for the Real World  and A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First .
The abstract for our most recent paper  summarised the concerns we have regarding the WAI model (which is based on three sets of guidelines – WCAG, UAAG and ATAG:
“This paper argues that web accessibility is not an intrinsic characteristic of a digital resource but is determined by complex political, social and other contextual factors, as well as technical aspects which are the focus of WAI standardisation activities. It can therefore be inappropriate to develop legislation or focus on metrics only associated with properties of the resource.
The authors describe the value of standards such as BS 8878 which focus on best practices for the process of developing web products and include a user focus.“
I have concerns that the WAI’s Website Accessibility Conformance Evaluation Methodology 1.0 working draft  could be counter-productive if it is used by policy-makers to mandate conformance with WCAG, rather than treating WCAG as a valuable set of guidelines whose use should be considered in context.
The WAI model itself provides one example of such contextual issues. WAI’s view of what it refers to as ‘universal accessibility‘ is that this requires conformance with WCAG, UAAG and ATAG guidelines. Since browsers which do not conform with ATAG are not ubiquitous it is clear that the values of WCAG conformance will be limited. In addition the ways in Web content is created has changed drastically since WAI was launched and the WAI model developed. Email messages sent to WAI mailing lists, for example, will be Web content hosted on the WAI’s mailing list archive on the W3C Web site. It is unlikely that such content will conform with WCAG guidelines.
A recent post entitled “John hit the ball”: Should Simple Language Be Mandatory for Web Accessibility?  highlighted that WAI have acknowledged that conformance with the current WCAG guidelines will n0t, as some people mistakenly think, address all disabilities. However, as described in the post, providing additional guidelines for incorporation in a future version of WCAG would be inappropriate as guidelines which mandate use of simple language would not be welcomed by everybody, for reasons described in the post and a more in-depth post on The complexities of simple: What simple language proponents should know about linguistics  by Dominik Lukes.
Beyond the limitations of the WAI model there are the contextual factors regarding the purposes of Web resources (which the WAI document highlights). The WAI model was developed at a time when the Web was being used primarily as an informational resource, although we were also seeing examples of commercial transactions being developed. But beyond the provision of information and the purchasing of products which are mentioned in the WAI document, there are also more complex areas such as learning and cultural appreciation for which there is a need to develop a better understanding of what is meant by such areas in a Web context.
It should also be noted that clarity provided on the scope of Web resources provided in the WAI document may ironically lead to organisations failing to provide Web resources which may provide accessibility benefits to some if they fail to conform fully with WCAG guidelines. This is likely to be particularly the case in the public sector, who may be required to provide Web sites which conform fully to WCAG guidelines.
In addition to dangers that this may lead to online resources failing to be deployed, there is also a need to consider the costs of providing resources which conform fully with WCAG guidelines, particularly at a time of economic constraints. To give a particular example a paper entitled Supporting PDF accessibility evaluation: early results from the FixRep project  analysed the provision of metadata in PDFs of (typically) peer-reviewed papers hosted in a university’s institutional repository and concluded:
“This means that only 10% of all PDFs processed have any likelihood of conforming to accessibility guidelines, and even then we would require further content level analysis to evaluate the extent to which they do indeed conform.”
It is felt (although further research is needed) that these findings are likely to be the case across institutional repositories more widely. Should we require that peer-reviewed papers should not be hosted on institutional repositories unless they conform with WCAG guidelines? If such a decision is made, what will the financial implications be and will “just-in-case accessibility” be an appropriate investment of scarce financial resources?
In light of such issues (which are discussed in more detail in the peer-reviewed papers which have been mentioned) what actions are appropriate for the Website Accessibility Conformance Evaluation Methodology 1.0 working draft? I would suggest that the document should explicitly mention the limitations of the WAI model (i.e. its dependencies of ATAG and UAAG) ; the need to address contexual factors and the need to address accessibility issues in a broader context including the context of use and purpose of the Web resource and the financial implications of conforming with the guidelines.
Finally I would suggest that document makes it clear that it would be inappropriate for policy-makers and legislators to enact legislation based solely on WCAG conformance. I would hasten to add that this is not to suggest that no interventions need to be made. Rather I would propose that it would be more appropriate to develop policies and legislation based on the processes surrounding the development of Web products as suggested in Accessibility 2.0: People, Policies and Processes . In the UK, such approaches have been described in the British Standard Institute’s BS 8878 Web Accessibility Code of Practice which is described at .
1. WAI Launch Agenda, WAI, http://www.w3.org/WAI/References/agenda
2. Forcing Standardization or Accommodating Diversity? A Framework for Applying the WCAG in the Real World, Kelly, B., Sloan, D., Phipps, L., Petrie, H. and Hamilton, F. Proceedings of the 2005 International Cross-Disciplinary Workshop on Web Accessibility (W4A). ISBN: 1-59593-036-1. http://opus.bath.ac.uk/438/
3. Contextual Web Accessibility – Maximizing the Benefit of Accessibility Guidelines, Sloan, D., Kelly, B., Heath, A., Petrie, H. Fraser, H. and Phipps, L. WWW 2006 Edinburgh, Scotland 22-26 May 2006. Conference Proceedings, http://opus.bath.ac.uk/402/
4. Accessibility 2.0: People, Policies and Processes, Kelly, B., Sloan, D., Brown, S., Seale, J, Petrie, H., Lauke, P. and Ball, S. WWW 2007 Banff, Canada, 7-11 May 2007. http://opus.bath.ac.uk/398/
5. One World, One Web … But Great Diversity, Kelly, B., Nevile, L., Draffan, EA. and Fanou, S. WWW 2008 Beijing, China, 21-22 April 2008. Proceedings of the 2008 international cross-disciplinary conference on Web accessibility (W4A), Beijing, China. Pages 141-147, Year of Publication: 2008. ISBN:978-1-60558-153-8 DOI: http://doi.acm.org/10.1145/1368044.1368078
6. From Web Accessibility to Web Adaptability, Kelly, B., Nevile, L., Sloan, D., Fanou, S., Ellison, R. and Herrod, L.
Disability and Rehability: Assistive Technology, Volume 4, Issue 4, July 2009, pages 212 – 226. DOI: 10.1080/17483100902903408
7. Developing Countries; Developing Experiences: Approaches to Accessibility for the Real World, Kelly, B., Lewthwaite, S. and Sloan, D. W4A2010, April 26-27, 2010, Raleigh, USA. Co-Located with the 19th International World Wide Web Conference. Copyright 2010 ACM ISBN: 978-1-4503-0045-2
8. A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First, Cooper, M., Sloan, D., Kelly, B. and Lewthwaite, S. W4A 2012, April 16-17, 2012, Lyon, France. Co-Located with the 21st International World Wide Web Conference. Copyright 2012 ACM ISBN 978-1-4503-1019-2
9. Website Accessibility Conformance Evaluation Methodology 1.0 working draft, WAI, 20 September 2012. http://www.w3.org/TR/2012/WD-WCAG-EM-20120920/
10. “John hit the ball”: Should Simple Language Be Mandatory for Web Accessibility?, Kelly, B., UK Web Focus blog, 19 Sept 2012, https://ukwebfocus.wordpress.com/2012/09/18/john-hit-the-ball-simple-language-mandatory-for-web-accessibility/
11. The complexities of simple: What simple language proponents should know about linguistics, Lukes, D. Metaphor Hacker blog, 28 Septemeber 2012, http://metaphorhacker.net/2012/09/the-complexities-of-simple-what-simple-language-proponents-should-know-about-linguistics/
12. Supporting PDF accessibility evaluation: early results from the FixRep project. In: 2nd Qualitative and Quantitative Methods in Libraries International Conference (QQML2010), 2010-05-25 – 2010-05-28, Chania. http://opus.bath.ac.uk/24958/
13. BS 8878 web accessibility standards (supersedes PAS 78) – all you need to know, Jonathan Hassell, http://www.hassellinclusion.com/bs8878/
Twitter conversation from: [Topsy]