WebAIM - Web Accessibility In Mind

E-mail List Archives

Testing with Screen Readers (was RE: Wai Aria how useful?)

for

From: John Foliot
Date: Jul 27, 2010 5:21PM

Interesting thread, and some thoughts:

There is a real tension between building standards compliant sites that
work with all technologies versus 'testing' with one tool, and building in
hacks and other 'repair strategies' to support bad software. Just ask
those unfortunates who must still support IE6. If you really must test
with a screen reader, then test with more than one: you test with more
than one browser, right?

The other point that must be considered is that to really be able to
'test' with any screen-reader, you should be a daily screen-reader user.
The problem we have is developers who want to do the right thing, but are
not competent with using Adaptive Technology, so really, all they are
doing is listening to their site read aloud: they are often missing out on
much of the interactivity daily users apply to using their software. For
production class developers, they are often better served by asking (and
paying) daily AT users to do the testing for them, rather than pretending
that opening up NVDA, or the time-crippled JAWs eval copy (which is
illegal, but...) and listening to a site read aloud and calling it
"testing".

Finally, as a very real question, what exactly are you testing *for* when
using a screen reader? If a developer is testing to find holes, I would
suggest they are approaching the problem from the wrong angle: you should
be testing that solutions implemented work as desired. So the developer
should be thinking about the problems and solutions long before they get
to the functional testing stage, and there are plenty of developer tools
out there that can confirm if your code is moving in the right direction
(understanding how a DOM inspector works is crucial); achieving real
accessible content means making some decisions at the foundational stage -
it's architectural. Waiting for a walk-through using AT to find problems
is almost certainly a recipe for ensuring there will be problems to
find...


As always, IMHO

JF



> -----Original Message-----
> From: <EMAIL REMOVED> [mailto:webaim-forum-
> <EMAIL REMOVED> ] On Behalf Of <EMAIL REMOVED>
> Sent: Tuesday, July 27, 2010 1:37 PM
> To: WebAIM Discussion List
> Subject: Re: [WebAIM] Wai Aria how useful?
>
> As a screen reader user I would not consider those valid grounds for
> testing
> purposes. The criteria especially that it is free is not a subjective
> requirement to ascertain usage. I think I would stick with the tried
> and
> true that has been around for testing purposes to insure compliance. It
> is
> disappointing to even think that web designers would consider
> compromising
> testing standards in this manner.
> chuck
> ----- Original Message -----
> From: "Steven Faulkner" < <EMAIL REMOVED> >
> To: "WebAIM Discussion List" < <EMAIL REMOVED> >
> Sent: Tuesday, July 27, 2010 6:41 AM
> Subject: Re: [WebAIM] Wai Aria how useful?
>
>
> Hi Michael,
>
> >Why would you consider user testing with a screen reader that is not
> yet
> widely used to be useful?
>
> according to the latest webaim screen reader survey NVDA is commonly
> used by
> 25% of respondents.
>
> Also it is a screen reader that is much more widley available than
> other
> screen readers (localized into 20+ languages) and is free.
>
> regards
> steve
> On 27 July 2010 14:36, < <EMAIL REMOVED> > wrote:
>
> > Why would you consider user testing with a screen reader that is not
> yet
> > widely used to be useful? I agree that NVDA has better support for
> ARIA at
> > this time than the main stream screen readers. But using NVDA for
> user
> > testing for visually impaired users would be like relying on Arora as
> your
> > browser for user testing for non-disabled users.
> >
> > Mike Moore
> > (512) 424-4159
> >
> >
> > -----Original Message-----
> > From: <EMAIL REMOVED> [mailto:
> > <EMAIL REMOVED> ] On Behalf Of Birkir Rúnar
> Gunnarsson
> > Sent: Monday, July 26, 2010 5:21 PM
> > To: WebAIM Discussion List
> > Subject: Re: [WebAIM] Wai Aria how useful?
> >
> > This is true, and this is why I think usability testing, in general,
> > should be done with NVDA rather than with the commercial screen
> > readers, or, at least, alongside the major screen readers.
> > I am very impressed with the NVDA performance, especially in this
> area.
> > Cheers
> > -B
> >
> > On 7/26/10, Patrick H. Lauke < <EMAIL REMOVED> > wrote:
> > > also worth mentioning yahoo!'s YUI work there
> > >
> > > --
> > > Patrick H. Lauke
> > >
> > >
> > > On 26 Jul 2010, at 21:39, "Hoffman, Allen" < <EMAIL REMOVED> >
> > wrote:
> > >
> > >> Just curious:
> > >> Dojo and Jquery are the two always named sets of building blocks
> which
> > >> include ARIA.
> > >> Of the set of similar building blocks, is this 2% 50% or what?
> > >> This proliferation of such building block sets which may or may
> not
> > >> include easy use of accessibility features seems to be a real
> challenge
> > to
> > >> me.
> > >>
> > >>
> > >>
> > >> -----Original Message-----
> > >> From: Birkir Rúnar Gunnarsson [mailto: <EMAIL REMOVED> ]
> > >> Sent: Thursday, July 22, 2010 6:43 PM
> > >> To: WebAIM Discussion List
> > >> Subject: Re: [WebAIM] Wai Aria how useful?
> > >>
> > >> Steve
> > >>
> > >> Thanks very much for all the info.
> > >> Like I said I am fairly new to all of this and it appears I
> > >> misunderstood a few things, but I am all too happy to be corrected
> and
> > >> will start with the pointers in your post for further research.
> > >> The screen reader ownership survey surprises me though, it may be
> > >> location specific, because I know the situation in a few countries
> in
> > >> Europe and the user SR ownership seems to differ from this survey,
> so
> > >> it is a good thing.
> > >> I will post back with questions/comments after I update my
> knowledge
> > >> some. *grin*
> > >> Cheers
> > >> -B
> > >>
> > >> On 7/22/10, Steven Faulkner < <EMAIL REMOVED> > wrote:
> > >>> hi birkir,
> > >>>
> > >>> you wrote:
> > >>> "You definitely cannot expect your users to be so up-to-date
> given the
> > >>> price
> > >>> of upgrades..."
> > >>>
> > >>> The webaim screen readers survey differs from your conclusion
> > >>>
> > >>> "The vast majority of respondents updated their primary screen
> reader
> > >>> within
> > >>> the previous year."
> > >>> http://www.webaim.org/projects/screenreadersurvey2/#demographics
> > >>>
> > >>>
> > >>> you wrote:
> > >>> "Another issue is clashes between screen reader key strokes while
> > >>> navigating a web page and assigned Aria keyboard shortcuts. "
> > >>>
> > >>> There are no "ARIA keyboard shortcuts" as such:
> > >>> The set of keyboard shortcuts defined for use in Google Reader
> has
> > >>> NOTHING
> > >>> to do with ARIA.
> > >>>
> > >>> What ARIA promotes is the implementation of keyboard interactions
> for
> > >>> widgets that are the same or similar to those of controls in
> desktop
> > >>> software applications
> > >>>
> > >>> "The model for keyboard support for Web 2.0 widgets are graphical
> user
> > >>>> interface (GUI) operating systems like Microsoft Windows, Mac OS
> X;
> > and
> > >>>> other desktop operating systems like GNOME and GTK. "
> > >>>
> > >>> http://www.w3.org/WAI/PF/aria-practices/#kbd_generalnav
> > >>>
> > >>> There is an ongoing effort to specify keyboard keystrokes to be
> used
> > for
> > >>> web
> > >>> based controls:
> > >>> http://dev.aol.com/dhtml_style_guide
> > >>>
> > >>> These keyboard interaction best practices are being implemented
> in
> > >>> javascript UI libraries such as DOJO and JQUERY so that users can
> rely
> > >>> upon
> > >>> the same or similar keystrokes being usable on for example a
> 'button'
> > >>> regardless of whether its a native HTML button <button> or a
> button
> > built
> > >>> from divs and spans with added script to provide the user
> interaction.
> > >>>
> > >>>> Another issue is clashes between screen reader key strokes while
> > >>>> navigating a web page and assigned Aria keyboard shortcuts. The
> > >>>> application element is supposed to turn off screen reader key
> > >>>> functionality but this does not always seem to work properly
> > >>>
> > >>> There is no clash per se, when in document reading mode, most key
> > strokes
> > >>> are consumed by the AT, this is why most windows based AT have a
> > special
> > >>> mode for interacting with native controls on web pages, which
> allows
> > >>> keystrokes to be passed to the browser rather than consumed by
> the AT.
> > >>> ARIA
> > >>> extends this method to the custom controls built using a
> combination
> > >>> of
> > >>> HTML/scripting and CSS.
> > >>>
> > >>> The use of role="application" on an element indicates to AT that
> > support
> > >>> it
> > >>> they should switch from document to application mode (so users
> can
> > >>> interact
> > >>> with controls using the keyboard). It is true that it sometimes
> does
> > not
> > >>> work, but i would suggest it is no more or less buggy than many
> of the
> > >>> other
> > >>> features in popular commercial screen readers.
> > >>>
> > >>> you wrote:
> > >>> "/basically, yopu have to expect not all users can use Aria and
> even
> > >>> for those who can, a lot of thought needs to go in to the key
> mapping
> > >>> and you must ensure that the user can access a simple keyboard
> help
> > >>> overview in a convenient format somehow (perhaps have it
> downloadable
> > >>> as a .txt file or plain html page that opens in a new window or
> tab)."
> > >>>
> > >>> again what you are talking about has nothing to do with ARIA, in
> fact
> > it
> > >>> is
> > >>> the opposite:
> > >>> The use of ARIA on custom controls provides users with
> information
> > about
> > >>> the
> > >>> role/state and properties of the control which typically results
> in
> > >>> the
> > >>> AT
> > >>> providing hints about how to interact with the control.
> > >>>
> > >>> Tests and examples of what AT report when ARIA roles are
> encountered:
> > >>> http://www.paciellogroup.com/blog/aria-tests/user-input-
> widgets.html
> > >>> note these results are from 18 months ago.
> > >>>
> > >>> Regards
> > >>> Stevef
> > >>>
> > >>>
> > >>>
> > >>> 2010/7/22 Birkir Rúnar Gunnarsson < <EMAIL REMOVED> >
> > >>>
> > >>>> There are a few inherent problems with Aria.
> > >>>> To use it I believe that you need Firefox 3.5 or newer, IE8. As
> for
> > >>>> screen readers I think it is Jaws 10 and above, Window Eyes 7
> and
> > >>>
> > >>> above and Hal/Dolphin 11.2 and above (NVDA does a good job of
> > >>>> recognizing Aria).
> > >>>> You definitely cannot expect your users to be so up-to-date
> given the
> > >>>> price of upgrades and relatively lousy set of improvements at
> least
> > >>>> in
> > >>>> Jaws lately (no bashing intended, Ijust feel that upgrades from
> Jaws
> > >>>> 9
> > >>>> have hardly been worth it).
> > >>>> Another issue is clashes between screen reader key strokes while
> > >>>> navigating a web page and assigned Aria keyboard shortcuts. The
> > >>>> application element is supposed to turn off screen reader key
> > >>>> functionality but this does not always seem to work properly (I
> will
> > >>>> admit I lack expertees in this area, I am looking at it but this
> was
> > >>>> discussed at length at the ICCHP conference I just attended).
> > >>>> Thus a screen reader user must pass every keysroke to an Aria
> page
> > >>>> through the screen reader by using the designed pass through
> key, and
> > >>>> that is a relatively advanced operation (or rather, having the
> user
> > >>>> recognize the need to do this is fairly advanced, one has to
> > >>>> understand what type of page is being encounterred and what that
> > >>>> means
> > >>>> for navigation).
> > >>>> Even if both of these issues are resoled and you have a user
> with
> > >>>> compatible SR and browser and the switching off works, there is
> still
> > >>>> a worrying lack of standards regarding keyboard mapping of aria
> > >>>> elements, which means the user has to continually return to some
> type
> > >>>> of on page help to see what keystrokes are required for what
> action
> > >>>> (See GoogleReader as an example).
> > >>>> These help messages do not (perhaps cannot) appear in a virtual
> > >>>> buffer, meaning it is hard to copy them and recall them in a
> text
> > >>>> document, so you have to kepp clicking on some type of help /
> > >>>> overview
> > >>>> on the page that tells you the functionality of each key stroke.
> > >>>> There is a brilliant video demo about this, I will post it if I
> > >>>> manage
> > >>>> to dig it up from my conference lecture notes, which I will have
> time
> > >>>> for over the weekend.
> > >>>> /basically, yopu have to expect not all users can use Aria and
> even
> > >>>> for those who can, a lot of thought needs to go in to the key
> mapping
> > >>>> and you must ensure that the user can access a simple keyboard
> help
> > >>>> overview in a convenient format somehow (perhaps have it
> downloadable
> > >>>> as a .txt file or plain html page that opens in a new window or
> tab).
> > >>>> Hope some of these thoughts help, plesae keep us updated and I
> will
> > >>>> post back once I have sorted out my notes.
> > >>>> Cheers
> > >>>> -B
> > >>>>
> > >>>> On 7/22/10, Seth Kane < <EMAIL REMOVED> > wrote:
> > >>>>> I have had little to no luck with ARIA unless you have the
> latest
> > >>>>> and
> > >>>> great
> > >>>>> version of both browsers and screen readers. It isn't ready
> for the
> > >>>> lowest
> > >>>>> common denominator just yet. Maybe HTML5 will be better.
> > >>>>>
> > >>>>>
> > >>>>> Seth Kane
> > >>>>>
> > >>>>>
> > >>>>> -----Original Message-----
> > >>>>> From: <EMAIL REMOVED>
> > >>>>> [mailto: <EMAIL REMOVED> ] On Behalf Of
> > >>>>> <EMAIL REMOVED>
> > >>>>> Sent: Wednesday, July 21, 2010 2:13 PM
> > >>>>> To: WebAIM Discussion List
> > >>>>> Subject: Re: [WebAIM] Wai Aria how useful?
> > >>>>>
> > >>>>> Nancy Johnson wrote:
> > >>>>>> How useful is WAI Aria to the average screen reader user who
> may
> > >>>>>> not
> > >>>>>> be
> > >>>>>> technically inclined?
> > >>>>>
> > >>>>> It depends on what version of a screenreader and browser they
> > >>>>> have. With JAWS 11 + Firefox 3, for example, live regions get
> > >>>>> announced in ways that some people find helpful (depending on
> how
> > >>>>> well they have encoded).
> > >>>>>
> > >>>>>> Does this also help the mobility impaired user who is not
> visually
> > >>>>>> impaired?
> > >>>>>
> > >>>>> I have found that there are circumstances in which ARIA
> > >>>>> navigation changes the way the wonderful Firefox add-on
> > >>>>> Mouseless Browsing interacts with a webpage in a way that gives
> > >>>>> me more control over Ajax drop-down menus. I haven't narrowed
> it
> > >>>>> down to exactly what features make things better, but I've been
> > >>>>> meaning to.
> > >>>>>
> > >>>>> -deborah
> > >>>>>