WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: accessibility without testing?

for

From: John Foliot - Stanford Online Accessibility Program
Date: Mar 13, 2008 5:30PM

Tom Babinszki wrote:
> For example, consider JAWS and AJAX. According to the WCAG
> you can create perfectly accessible AJAX based products,

Really? Can you kindly point out where in WCAG it states this? Having
worked with this guideline since it's release in 1999 (back when AJAX was a
cleaning product), I would love to know how you reached this conclusion.

> but the JAWS
> screen reader might not read it properly. In this case, the screen
> reader manufacturers should put more effort into making their
> applications work with a set of internationally accepted and use
> guidelines.

Huh? At best, what we have today is ARIA, for which we have an incomplete
but growing series of test-case documents which illustrate how the various
ARIA roles and states should behave. But ARIA, and it's implementation is
pretty cutting-edge, and as far as I know about the only screen-reading type
application that currently plays nice with ARIA is FireVox plugin for
Firefox. You also neatly side-stepped the whole issue of *versioning*: many
people reliant on adaptive technology such as JAWS cannot always afford to
upgrade with every release; at $1,000 +/- per computer, it is often out of
reach for many who (sadly) struggle just to keep their heads above the
poverty line. So while it might be easy to blame the developers of screen
reading technology, the issue is way more complicated than stating that they
"..should put more effort in..."

> However, in the meantime, blind people will have
> difficulties using it. So, once you satisfied the accessibility
> standards you have picked, depending on your page, it might be a good
> idea to ensure that it works with a screen reader.

Actually, putting a screen reader in the hands of somebody who rarely uses a
screen reader will do little to actually verify the accessibility of a
document.

Screen readers are complex and sophisticated pieces of software that require
almost daily use to master, and most web developers cannot devote the time
necessary to properly learn how to use a tool such as JAWS: it's really more
than just ready your page out loud. You need to understand the various ways
that users of the tool (who themselves have varying degrees of competence
with the tool) can navigate around the screen / web page. You need to
understand about verbosity settings, how the tools can be personalized for
individuals (which may or may not affect outcomes), the various keystroke
combinations used to interact with the tool
[http://www.wats.ca/show.php?contentid=48] and much, much more. Finally,
just because your page "works" in JAWS is no more a guarantee that it is
"accessible" then saying that it also works in Internet Explorer. In short,
save your money and don't buy JAWS unless you plan on using it every day.

A much better methodology is to actually find a daily user of adaptive
technology and work with that person to do human testing of your site. Sit
with that person and have them review the site in question. Ask questions,
try to achieve "results", but avoid directing the user on how to do
anything... Sighted testers using JAWS is not a real test: you can see
things and whether intentional or not, your "seeing" will skew the results.
Instead, have the non-sighted user go through the site and verbalize their
"journey". What's good for them? What's bad for them? Set a task and ask
if they can achieve it. Never mind "how", the real question is can they,
and can they do so easily? Ask them to summarize the key points on the
page. Does the summary accurately reflect the visualization? Often large
areas of screen real estate is dedicated to something "important", yet
semantically the content is relegated to just another paragraph. In short,
try to glean from the user their impression and usability of the page. If
they can achieve everything that the sighted user can do, *then* you can
safely conclude that the page is functionally accessible. (This may also
contradict some technical measurements, at which point you as the "expert"
need to weigh the pros and cons and report back to the content owner - for
example if a page does not validate, it will not pass WCAG AA - even though
the page might be totally accessible to all of your test subjects.)

I realize that for most developers this level of testing is quite extensive,
and probably beyond reach more often than not. That's really too bad (and
I'm being sincere here), because at many levels it is this type of testing
that should be done, but often is not. But if you *really* want to get a
better understanding of the issue, you *really* need to go though the above
exercise at least once in your career. Seek out volunteers (or trade
services) - most communities have support groups for their constituents (and
remember that web accessibility is more than just visual disability).

So to summarize, and return to the question initially posed: to truly
determine accessibility *does* require human testing, but by humans who
access web content using the various AT tools out there, not by web
developers buying a piece of software and attempting to replicate foreign
experiences.

Cheers!
JF
==================================John Foliot
Academic Technology Consultant
Stanford Online Accessibility Program
http://soap.stanford.edu
Stanford University
Tel: 650-862-4603
==================================