WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: Testing with Screen Readers (was RE: Wai Aria how useful?)

for

From: Birkir Rúnar Gunnarsson
Date: Jul 28, 2010 7:00AM

At the conference I attended in Vienna (and some of you attended as
well, I believe) there was talk about the need for screen reader/AT
software simulation tools or plug ins for development environments, so
the developers get a feel for the problem and how their code would
work for a screen reader user.
It is an interesting idea, though there are some limitations, e.g.
that one should expect an SR user, or user of any other AT, to have
developed some skills with the product and be able to do more with it
than, perhaps, a developer would see at first glance.
Also building accessibility testing and compliance into CMS and other
development software would definitely, I think, be a very good step,
and I know there are projects going on in the area.
I see the practical problem of testing with real users, it is
expensive, there are marked differences in what each screen reader
does well and does badly, so you might need different testing based on
your potential user group etc.
Sometimes a particular SR has not implemented technology that exists,
and it is hard for the developer to do anything about it.
Cheers
-B

On 7/28/10, Langum, Michael J < <EMAIL REMOVED> > wrote:
> John writes:
>> The other point that must be considered is that to really be able to
>> 'test' with any screen-reader,
>> you should be a daily screen-reader user. ....
>
> I agree in principle, but in the "real world" time and resources do not
> permit the additional time and cost of testing by "a daily screen-reader
> user."
>
> I am only able to do detailed testing at the code level (e.g. "alt" text;
> semantic structure; <label> for <input>, column and row headers, etc. We do
> randomized testing on some content, and react immediately if we get any user
> complaints.
>
> Jared, Do you think there might be value in doing a survey on testing
> methodologies? Do developers actually test? Do they go beyond the use of
> automated testing tools? Do they do "functional" testing with various AT
> products? Do they do this functional testing on each file developed?
>
> -- Mike
>
>
>
>
>> -----Original Message-----
>> From: <EMAIL REMOVED> [mailto:webaim-forum-
>> <EMAIL REMOVED> ] On Behalf Of <EMAIL REMOVED>
>> Sent: Tuesday, July 27, 2010 1:37 PM
>> To: WebAIM Discussion List
>> Subject: Re: [WebAIM] Wai Aria how useful?
>>
>> As a screen reader user I would not consider those valid grounds for
>> testing purposes. The criteria especially that it is free is not a
>> subjective requirement to ascertain usage. I think I would stick with
>> the tried and
>> true that has been around for testing purposes to insure compliance. It
>> is
>> disappointing to even think that web designers would consider
>> compromising
>> testing standards in this manner.
>> chuck
>> ----- Original Message -----
>> From: "Steven Faulkner" < <EMAIL REMOVED> >
>> To: "WebAIM Discussion List" < <EMAIL REMOVED> >
>> Sent: Tuesday, July 27, 2010 6:41 AM
>> Subject: Re: [WebAIM] Wai Aria how useful?
>>
>>
>> Hi Michael,
>>
>> >Why would you consider user testing with a screen reader that is not
>> yet
>> widely used to be useful?
>>
>> according to the latest webaim screen reader survey NVDA is commonly
>> used by 25% of respondents.
>>
>> Also it is a screen reader that is much more widley available than
>> other screen readers (localized into 20+ languages) and is free.
>>
>> regards
>> steve
>> On 27 July 2010 14:36, < <EMAIL REMOVED> > wrote:
>>
>> > Why would you consider user testing with a screen reader that is not
>> yet
>> > widely used to be useful? I agree that NVDA has better support for
>> ARIA at
>> > this time than the main stream screen readers. But using NVDA for
>> user
>> > testing for visually impaired users would be like relying on Arora
>> > as
>> your
>> > browser for user testing for non-disabled users.
>> >
>> > Mike Moore
>> > (512) 424-4159
>> >
>> >
>> > -----Original Message-----
>> > From: <EMAIL REMOVED> [mailto:
>> > <EMAIL REMOVED> ] On Behalf Of Birkir Rúnar
>> Gunnarsson
>> > Sent: Monday, July 26, 2010 5:21 PM
>> > To: WebAIM Discussion List
>> > Subject: Re: [WebAIM] Wai Aria how useful?
>> >
>> > This is true, and this is why I think usability testing, in general,
>> > should be done with NVDA rather than with the commercial screen
>> > readers, or, at least, alongside the major screen readers. I am very
>> > impressed with the NVDA performance, especially in this
>> area.
>> > Cheers
>> > -B
>> >
>> > On 7/26/10, Patrick H. Lauke < <EMAIL REMOVED> > wrote:
>> > > also worth mentioning yahoo!'s YUI work there
>> > >
>> > > --
>> > > Patrick H. Lauke
>> > >
>> > >
>> > > On 26 Jul 2010, at 21:39, "Hoffman, Allen" < <EMAIL REMOVED> >
>> > wrote:
>> > >
>> > >> Just curious:
>> > >> Dojo and Jquery are the two always named sets of building blocks
>> which
>> > >> include ARIA.
>> > >> Of the set of similar building blocks, is this 2% 50% or what?
>> > >> This proliferation of such building block sets which may or may
>> not
>> > >> include easy use of accessibility features seems to be a real
>> challenge
>> > to
>> > >> me.
>> > >>
>> > >>
>> > >>
>> > >> -----Original Message-----
>> > >> From: Birkir Rúnar Gunnarsson
>> > >> [mailto: <EMAIL REMOVED> ]
>> > >> Sent: Thursday, July 22, 2010 6:43 PM
>> > >> To: WebAIM Discussion List
>> > >> Subject: Re: [WebAIM] Wai Aria how useful?
>> > >>
>> > >> Steve
>> > >>
>> > >> Thanks very much for all the info.
>> > >> Like I said I am fairly new to all of this and it appears I
>> > >> misunderstood a few things, but I am all too happy to be
>> > >> corrected
>> and
>> > >> will start with the pointers in your post for further research.
>> > >> The screen reader ownership survey surprises me though, it may be
>> > >> location specific, because I know the situation in a few
>> > >> countries
>> in
>> > >> Europe and the user SR ownership seems to differ from this
>> > >> survey,
>> so
>> > >> it is a good thing.
>> > >> I will post back with questions/comments after I update my
>> knowledge
>> > >> some. *grin*
>> > >> Cheers
>> > >> -B
>> > >>
>> > >> On 7/22/10, Steven Faulkner < <EMAIL REMOVED> > wrote:
>> > >>> hi birkir,
>> > >>>
>> > >>> you wrote:
>> > >>> "You definitely cannot expect your users to be so up-to-date
>> given the
>> > >>> price
>> > >>> of upgrades..."
>> > >>>
>> > >>> The webaim screen readers survey differs from your conclusion
>> > >>>
>> > >>> "The vast majority of respondents updated their primary screen
>> reader
>> > >>> within
>> > >>> the previous year."
>> > >>> http://www.webaim.org/projects/screenreadersurvey2/#demographics
>> > >>>
>> > >>>
>> > >>> you wrote:
>> > >>> "Another issue is clashes between screen reader key strokes
>> > >>> while navigating a web page and assigned Aria keyboard
>> > >>> shortcuts. "
>> > >>>
>> > >>> There are no "ARIA keyboard shortcuts" as such:
>> > >>> The set of keyboard shortcuts defined for use in Google Reader
>> has
>> > >>> NOTHING
>> > >>> to do with ARIA.
>> > >>>
>> > >>> What ARIA promotes is the implementation of keyboard
>> > >>> interactions
>> for
>> > >>> widgets that are the same or similar to those of controls in
>> desktop
>> > >>> software applications
>> > >>>
>> > >>> "The model for keyboard support for Web 2.0 widgets are
>> > >>> graphical
>> user
>> > >>>> interface (GUI) operating systems like Microsoft Windows, Mac
>> > >>>> OS
>> X;
>> > and
>> > >>>> other desktop operating systems like GNOME and GTK. "
>> > >>>
>> > >>> http://www.w3.org/WAI/PF/aria-practices/#kbd_generalnav
>> > >>>
>> > >>> There is an ongoing effort to specify keyboard keystrokes to be
>> used
>> > for
>> > >>> web
>> > >>> based controls:
>> > >>> http://dev.aol.com/dhtml_style_guide
>> > >>>
>> > >>> These keyboard interaction best practices are being implemented
>> in
>> > >>> javascript UI libraries such as DOJO and JQUERY so that users
>> > >>> can
>> rely
>> > >>> upon
>> > >>> the same or similar keystrokes being usable on for example a
>> 'button'
>> > >>> regardless of whether its a native HTML button <button> or a
>> button
>> > built
>> > >>> from divs and spans with added script to provide the user
>> interaction.
>> > >>>
>> > >>>> Another issue is clashes between screen reader key strokes
>> > >>>> while navigating a web page and assigned Aria keyboard
>> > >>>> shortcuts. The application element is supposed to turn off
>> > >>>> screen reader key functionality but this does not always seem
>> > >>>> to work properly
>> > >>>
>> > >>> There is no clash per se, when in document reading mode, most
>> > >>> key
>> > strokes
>> > >>> are consumed by the AT, this is why most windows based AT have a
>> > special
>> > >>> mode for interacting with native controls on web pages, which
>> allows
>> > >>> keystrokes to be passed to the browser rather than consumed by
>> the AT.
>> > >>> ARIA
>> > >>> extends this method to the custom controls built using a
>> combination
>> > >>> of
>> > >>> HTML/scripting and CSS.
>> > >>>
>> > >>> The use of role="application" on an element indicates to AT that
>> > support
>> > >>> it
>> > >>> they should switch from document to application mode (so users
>> can
>> > >>> interact
>> > >>> with controls using the keyboard). It is true that it sometimes
>> does
>> > not
>> > >>> work, but i would suggest it is no more or less buggy than many
>> of the
>> > >>> other
>> > >>> features in popular commercial screen readers.
>> > >>>
>> > >>> you wrote:
>> > >>> "/basically, yopu have to expect not all users can use Aria and
>> even
>> > >>> for those who can, a lot of thought needs to go in to the key
>> mapping
>> > >>> and you must ensure that the user can access a simple keyboard
>> help
>> > >>> overview in a convenient format somehow (perhaps have it
>> downloadable
>> > >>> as a .txt file or plain html page that opens in a new window or
>> tab)."
>> > >>>
>> > >>> again what you are talking about has nothing to do with ARIA, in
>> fact
>> > it
>> > >>> is
>> > >>> the opposite:
>> > >>> The use of ARIA on custom controls provides users with
>> information
>> > about
>> > >>> the
>> > >>> role/state and properties of the control which typically results
>> in
>> > >>> the
>> > >>> AT
>> > >>> providing hints about how to interact with the control.
>> > >>>
>> > >>> Tests and examples of what AT report when ARIA roles are
>> encountered:
>> > >>> http://www.paciellogroup.com/blog/aria-tests/user-input-
>> widgets.html
>> > >>> note these results are from 18 months ago.
>> > >>>
>> > >>> Regards
>> > >>> Stevef
>> > >>>
>> > >>>
>> > >>>
>> > >>> 2010/7/22 Birkir Rúnar Gunnarsson < <EMAIL REMOVED> >
>> > >>>
>> > >>>> There are a few inherent problems with Aria.
>> > >>>> To use it I believe that you need Firefox 3.5 or newer, IE8. As
>> for
>> > >>>> screen readers I think it is Jaws 10 and above, Window Eyes 7
>> and
>> > >>>
>> > >>> above and Hal/Dolphin 11.2 and above (NVDA does a good job of
>> > >>>> recognizing Aria).
>> > >>>> You definitely cannot expect your users to be so up-to-date
>> given the
>> > >>>> price of upgrades and relatively lousy set of improvements at
>> least
>> > >>>> in
>> > >>>> Jaws lately (no bashing intended, Ijust feel that upgrades from
>> Jaws
>> > >>>> 9
>> > >>>> have hardly been worth it).
>> > >>>> Another issue is clashes between screen reader key strokes
>> > >>>> while navigating a web page and assigned Aria keyboard
>> > >>>> shortcuts. The application element is supposed to turn off
>> > >>>> screen reader key functionality but this does not always seem
>> > >>>> to work properly (I
>> will
>> > >>>> admit I lack expertees in this area, I am looking at it but
>> > >>>> this
>> was
>> > >>>> discussed at length at the ICCHP conference I just attended).
>> > >>>> Thus a screen reader user must pass every keysroke to an Aria
>> page
>> > >>>> through the screen reader by using the designed pass through
>> key, and
>> > >>>> that is a relatively advanced operation (or rather, having the
>> user
>> > >>>> recognize the need to do this is fairly advanced, one has to
>> > >>>> understand what type of page is being encounterred and what
>> > >>>> that means for navigation).
>> > >>>> Even if both of these issues are resoled and you have a user
>> with
>> > >>>> compatible SR and browser and the switching off works, there is
>> still
>> > >>>> a worrying lack of standards regarding keyboard mapping of aria
>> > >>>> elements, which means the user has to continually return to
>> > >>>> some
>> type
>> > >>>> of on page help to see what keystrokes are required for what
>> action
>> > >>>> (See GoogleReader as an example).
>> > >>>> These help messages do not (perhaps cannot) appear in a virtual
>> > >>>> buffer, meaning it is hard to copy them and recall them in a
>> text
>> > >>>> document, so you have to kepp clicking on some type of help /
>> > >>>> overview on the page that tells you the functionality of each
>> > >>>> key stroke. There is a brilliant video demo about this, I will
>> > >>>> post it if I manage
>> > >>>> to dig it up from my conference lecture notes, which I will have
>> time
>> > >>>> for over the weekend.
>> > >>>> /basically, yopu have to expect not all users can use Aria and
>> even
>> > >>>> for those who can, a lot of thought needs to go in to the key
>> mapping
>> > >>>> and you must ensure that the user can access a simple keyboard
>> help
>> > >>>> overview in a convenient format somehow (perhaps have it
>> downloadable
>> > >>>> as a .txt file or plain html page that opens in a new window or
>> tab).
>> > >>>> Hope some of these thoughts help, plesae keep us updated and I
>> will
>> > >>>> post back once I have sorted out my notes.
>> > >>>> Cheers
>> > >>>> -B
>> > >>>>
>> > >>>> On 7/22/10, Seth Kane < <EMAIL REMOVED> > wrote:
>> > >>>>> I have had little to no luck with ARIA unless you have the
>> latest
>> > >>>>> and
>> > >>>> great
>> > >>>>> version of both browsers and screen readers. It isn't ready
>> for the
>> > >>>> lowest
>> > >>>>> common denominator just yet. Maybe HTML5 will be better.
>> > >>>>>
>> > >>>>>
>> > >>>>> Seth Kane
>> > >>>>>
>> > >>>>>
>> > >>>>> -----Original Message-----
>> > >>>>> From: <EMAIL REMOVED>
>> > >>>>> [mailto: <EMAIL REMOVED> ] On Behalf Of
>> > >>>>> <EMAIL REMOVED>
>> > >>>>> Sent: Wednesday, July 21, 2010 2:13 PM
>> > >>>>> To: WebAIM Discussion List
>> > >>>>> Subject: Re: [WebAIM] Wai Aria how useful?
>> > >>>>>
>> > >>>>> Nancy Johnson wrote:
>> > >>>>>> How useful is WAI Aria to the average screen reader user who
>> may
>> > >>>>>> not
>> > >>>>>> be
>> > >>>>>> technically inclined?
>> > >>>>>
>> > >>>>> It depends on what version of a screenreader and browser they
>> > >>>>> have. With JAWS 11 + Firefox 3, for example, live regions get
>> > >>>>> announced in ways that some people find helpful (depending on
>> how
>> > >>>>> well they have encoded).
>> > >>>>>
>> > >>>>>> Does this also help the mobility impaired user who is not
>> visually
>> > >>>>>> impaired?
>> > >>>>>
>> > >>>>> I have found that there are circumstances in which ARIA
>> > >>>>> navigation changes the way the wonderful Firefox add-on
>> > >>>>> Mouseless Browsing interacts with a webpage in a way that
>> > >>>>> gives me more control over Ajax drop-down menus. I haven't
>> > >>>>> narrowed
>> it
>> > >>>>> down to exactly what features make things better, but I've
>> > >>>>> been meaning to.
>> > >>>>>
>> > >>>>> -deborah