E-mail List Archives

Re: Testing with Screen Readers (was RE: Wai Aria how useful?)

for

From: Langum, Michael J
Date: Jul 28, 2010 6:48AM


John writes:
> The other point that must be considered is that to really be able to 'test' with any screen-reader,
> you should be a daily screen-reader user. ....

I agree in principle, but in the "real world" time and resources do not permit the additional time and cost of testing by "a daily screen-reader user."

I am only able to do detailed testing at the code level (e.g. "alt" text; semantic structure; <label> for <input>, column and row headers, etc. We do randomized testing on some content, and react immediately if we get any user complaints.

Jared, Do you think there might be value in doing a survey on testing methodologies? Do developers actually test? Do they go beyond the use of automated testing tools? Do they do "functional" testing with various AT products? Do they do this functional testing on each file developed?

-- Mike




> -----Original Message-----
> From: <EMAIL REMOVED> [mailto:webaim-forum-
> <EMAIL REMOVED> ] On Behalf Of <EMAIL REMOVED>
> Sent: Tuesday, July 27, 2010 1:37 PM
> To: WebAIM Discussion List
> Subject: Re: [WebAIM] Wai Aria how useful?
>
> As a screen reader user I would not consider those valid grounds for
> testing purposes. The criteria especially that it is free is not a
> subjective requirement to ascertain usage. I think I would stick with
> the tried and
> true that has been around for testing purposes to insure compliance. It
> is
> disappointing to even think that web designers would consider
> compromising
> testing standards in this manner.
> chuck
> ----- Original Message -----
> From: "Steven Faulkner" < <EMAIL REMOVED> >
> To: "WebAIM Discussion List" < <EMAIL REMOVED> >
> Sent: Tuesday, July 27, 2010 6:41 AM
> Subject: Re: [WebAIM] Wai Aria how useful?
>
>
> Hi Michael,
>
> >Why would you consider user testing with a screen reader that is not
> yet
> widely used to be useful?
>
> according to the latest webaim screen reader survey NVDA is commonly
> used by 25% of respondents.
>
> Also it is a screen reader that is much more widley available than
> other screen readers (localized into 20+ languages) and is free.
>
> regards
> steve
> On 27 July 2010 14:36, < <EMAIL REMOVED> > wrote:
>
> > Why would you consider user testing with a screen reader that is not
> yet
> > widely used to be useful? I agree that NVDA has better support for
> ARIA at
> > this time than the main stream screen readers. But using NVDA for
> user
> > testing for visually impaired users would be like relying on Arora
> > as
> your
> > browser for user testing for non-disabled users.
> >
> > Mike Moore
> > (512) 424-4159
> >
> >
> > -----Original Message-----
> > From: <EMAIL REMOVED> [mailto:
> > <EMAIL REMOVED> ] On Behalf Of Birkir Rúnar
> Gunnarsson
> > Sent: Monday, July 26, 2010 5:21 PM
> > To: WebAIM Discussion List
> > Subject: Re: [WebAIM] Wai Aria how useful?
> >
> > This is true, and this is why I think usability testing, in general,
> > should be done with NVDA rather than with the commercial screen
> > readers, or, at least, alongside the major screen readers. I am very
> > impressed with the NVDA performance, especially in this
> area.
> > Cheers
> > -B
> >
> > On 7/26/10, Patrick H. Lauke < <EMAIL REMOVED> > wrote:
> > > also worth mentioning yahoo!'s YUI work there
> > >
> > > --
> > > Patrick H. Lauke
> > >
> > >
> > > On 26 Jul 2010, at 21:39, "Hoffman, Allen" < <EMAIL REMOVED> >
> > wrote:
> > >
> > >> Just curious:
> > >> Dojo and Jquery are the two always named sets of building blocks
> which
> > >> include ARIA.
> > >> Of the set of similar building blocks, is this 2% 50% or what?
> > >> This proliferation of such building block sets which may or may
> not
> > >> include easy use of accessibility features seems to be a real
> challenge
> > to
> > >> me.
> > >>
> > >>
> > >>
> > >> -----Original Message-----
> > >> From: Birkir Rúnar Gunnarsson
> > >> [mailto: <EMAIL REMOVED> ]
> > >> Sent: Thursday, July 22, 2010 6:43 PM
> > >> To: WebAIM Discussion List
> > >> Subject: Re: [WebAIM] Wai Aria how useful?
> > >>
> > >> Steve
> > >>
> > >> Thanks very much for all the info.
> > >> Like I said I am fairly new to all of this and it appears I
> > >> misunderstood a few things, but I am all too happy to be
> > >> corrected
> and
> > >> will start with the pointers in your post for further research.
> > >> The screen reader ownership survey surprises me though, it may be
> > >> location specific, because I know the situation in a few
> > >> countries
> in
> > >> Europe and the user SR ownership seems to differ from this
> > >> survey,
> so
> > >> it is a good thing.
> > >> I will post back with questions/comments after I update my
> knowledge
> > >> some. *grin*
> > >> Cheers
> > >> -B
> > >>
> > >> On 7/22/10, Steven Faulkner < <EMAIL REMOVED> > wrote:
> > >>> hi birkir,
> > >>>
> > >>> you wrote:
> > >>> "You definitely cannot expect your users to be so up-to-date
> given the
> > >>> price
> > >>> of upgrades..."
> > >>>
> > >>> The webaim screen readers survey differs from your conclusion
> > >>>
> > >>> "The vast majority of respondents updated their primary screen
> reader
> > >>> within
> > >>> the previous year."
> > >>> http://www.webaim.org/projects/screenreadersurvey2/#demographics
> > >>>
> > >>>
> > >>> you wrote:
> > >>> "Another issue is clashes between screen reader key strokes
> > >>> while navigating a web page and assigned Aria keyboard
> > >>> shortcuts. "
> > >>>
> > >>> There are no "ARIA keyboard shortcuts" as such:
> > >>> The set of keyboard shortcuts defined for use in Google Reader
> has
> > >>> NOTHING
> > >>> to do with ARIA.
> > >>>
> > >>> What ARIA promotes is the implementation of keyboard
> > >>> interactions
> for
> > >>> widgets that are the same or similar to those of controls in
> desktop
> > >>> software applications
> > >>>
> > >>> "The model for keyboard support for Web 2.0 widgets are
> > >>> graphical
> user
> > >>>> interface (GUI) operating systems like Microsoft Windows, Mac
> > >>>> OS
> X;
> > and
> > >>>> other desktop operating systems like GNOME and GTK. "
> > >>>
> > >>> http://www.w3.org/WAI/PF/aria-practices/#kbd_generalnav
> > >>>
> > >>> There is an ongoing effort to specify keyboard keystrokes to be
> used
> > for
> > >>> web
> > >>> based controls:
> > >>> http://dev.aol.com/dhtml_style_guide
> > >>>
> > >>> These keyboard interaction best practices are being implemented
> in
> > >>> javascript UI libraries such as DOJO and JQUERY so that users
> > >>> can
> rely
> > >>> upon
> > >>> the same or similar keystrokes being usable on for example a
> 'button'
> > >>> regardless of whether its a native HTML button <button> or a
> button
> > built
> > >>> from divs and spans with added script to provide the user
> interaction.
> > >>>
> > >>>> Another issue is clashes between screen reader key strokes
> > >>>> while navigating a web page and assigned Aria keyboard
> > >>>> shortcuts. The application element is supposed to turn off
> > >>>> screen reader key functionality but this does not always seem
> > >>>> to work properly
> > >>>
> > >>> There is no clash per se, when in document reading mode, most
> > >>> key
> > strokes
> > >>> are consumed by the AT, this is why most windows based AT have a
> > special
> > >>> mode for interacting with native controls on web pages, which
> allows
> > >>> keystrokes to be passed to the browser rather than consumed by
> the AT.
> > >>> ARIA
> > >>> extends this method to the custom controls built using a
> combination
> > >>> of
> > >>> HTML/scripting and CSS.
> > >>>
> > >>> The use of role="application" on an element indicates to AT that
> > support
> > >>> it
> > >>> they should switch from document to application mode (so users
> can
> > >>> interact
> > >>> with controls using the keyboard). It is true that it sometimes
> does
> > not
> > >>> work, but i would suggest it is no more or less buggy than many
> of the
> > >>> other
> > >>> features in popular commercial screen readers.
> > >>>
> > >>> you wrote:
> > >>> "/basically, yopu have to expect not all users can use Aria and
> even
> > >>> for those who can, a lot of thought needs to go in to the key
> mapping
> > >>> and you must ensure that the user can access a simple keyboard
> help
> > >>> overview in a convenient format somehow (perhaps have it
> downloadable
> > >>> as a .txt file or plain html page that opens in a new window or
> tab)."
> > >>>
> > >>> again what you are talking about has nothing to do with ARIA, in
> fact
> > it
> > >>> is
> > >>> the opposite:
> > >>> The use of ARIA on custom controls provides users with
> information
> > about
> > >>> the
> > >>> role/state and properties of the control which typically results
> in
> > >>> the
> > >>> AT
> > >>> providing hints about how to interact with the control.
> > >>>
> > >>> Tests and examples of what AT report when ARIA roles are
> encountered:
> > >>> http://www.paciellogroup.com/blog/aria-tests/user-input-
> widgets.html
> > >>> note these results are from 18 months ago.
> > >>>
> > >>> Regards
> > >>> Stevef
> > >>>
> > >>>
> > >>>
> > >>> 2010/7/22 Birkir Rúnar Gunnarsson < <EMAIL REMOVED> >
> > >>>
> > >>>> There are a few inherent problems with Aria.
> > >>>> To use it I believe that you need Firefox 3.5 or newer, IE8. As
> for
> > >>>> screen readers I think it is Jaws 10 and above, Window Eyes 7
> and
> > >>>
> > >>> above and Hal/Dolphin 11.2 and above (NVDA does a good job of
> > >>>> recognizing Aria).
> > >>>> You definitely cannot expect your users to be so up-to-date
> given the
> > >>>> price of upgrades and relatively lousy set of improvements at
> least
> > >>>> in
> > >>>> Jaws lately (no bashing intended, Ijust feel that upgrades from
> Jaws
> > >>>> 9
> > >>>> have hardly been worth it).
> > >>>> Another issue is clashes between screen reader key strokes
> > >>>> while navigating a web page and assigned Aria keyboard
> > >>>> shortcuts. The application element is supposed to turn off
> > >>>> screen reader key functionality but this does not always seem
> > >>>> to work properly (I
> will
> > >>>> admit I lack expertees in this area, I am looking at it but
> > >>>> this
> was
> > >>>> discussed at length at the ICCHP conference I just attended).
> > >>>> Thus a screen reader user must pass every keysroke to an Aria
> page
> > >>>> through the screen reader by using the designed pass through
> key, and
> > >>>> that is a relatively advanced operation (or rather, having the
> user
> > >>>> recognize the need to do this is fairly advanced, one has to
> > >>>> understand what type of page is being encounterred and what
> > >>>> that means for navigation).
> > >>>> Even if both of these issues are resoled and you have a user
> with
> > >>>> compatible SR and browser and the switching off works, there is
> still
> > >>>> a worrying lack of standards regarding keyboard mapping of aria
> > >>>> elements, which means the user has to continually return to
> > >>>> some
> type
> > >>>> of on page help to see what keystrokes are required for what
> action
> > >>>> (See GoogleReader as an example).
> > >>>> These help messages do not (perhaps cannot) appear in a virtual
> > >>>> buffer, meaning it is hard to copy them and recall them in a
> text
> > >>>> document, so you have to kepp clicking on some type of help /
> > >>>> overview on the page that tells you the functionality of each
> > >>>> key stroke. There is a brilliant video demo about this, I will
> > >>>> post it if I manage
> > >>>> to dig it up from my conference lecture notes, which I will have
> time
> > >>>> for over the weekend.
> > >>>> /basically, yopu have to expect not all users can use Aria and
> even
> > >>>> for those who can, a lot of thought needs to go in to the key
> mapping
> > >>>> and you must ensure that the user can access a simple keyboard
> help
> > >>>> overview in a convenient format somehow (perhaps have it
> downloadable
> > >>>> as a .txt file or plain html page that opens in a new window or
> tab).
> > >>>> Hope some of these thoughts help, plesae keep us updated and I
> will
> > >>>> post back once I have sorted out my notes.
> > >>>> Cheers
> > >>>> -B
> > >>>>
> > >>>> On 7/22/10, Seth Kane < <EMAIL REMOVED> > wrote:
> > >>>>> I have had little to no luck with ARIA unless you have the
> latest
> > >>>>> and
> > >>>> great
> > >>>>> version of both browsers and screen readers. It isn't ready
> for the
> > >>>> lowest
> > >>>>> common denominator just yet. Maybe HTML5 will be better.
> > >>>>>
> > >>>>>
> > >>>>> Seth Kane
> > >>>>>
> > >>>>>
> > >>>>> -----Original Message-----
> > >>>>> From: <EMAIL REMOVED>
> > >>>>> [mailto: <EMAIL REMOVED> ] On Behalf Of
> > >>>>> <EMAIL REMOVED>
> > >>>>> Sent: Wednesday, July 21, 2010 2:13 PM
> > >>>>> To: WebAIM Discussion List
> > >>>>> Subject: Re: [WebAIM] Wai Aria how useful?
> > >>>>>
> > >>>>> Nancy Johnson wrote:
> > >>>>>> How useful is WAI Aria to the average screen reader user who
> may
> > >>>>>> not
> > >>>>>> be
> > >>>>>> technically inclined?
> > >>>>>
> > >>>>> It depends on what version of a screenreader and browser they
> > >>>>> have. With JAWS 11 + Firefox 3, for example, live regions get
> > >>>>> announced in ways that some people find helpful (depending on
> how
> > >>>>> well they have encoded).
> > >>>>>
> > >>>>>> Does this also help the mobility impaired user who is not
> visually
> > >>>>>> impaired?
> > >>>>>
> > >>>>> I have found that there are circumstances in which ARIA
> > >>>>> navigation changes the way the wonderful Firefox add-on
> > >>>>> Mouseless Browsing interacts with a webpage in a way that
> > >>>>> gives me more control over Ajax drop-down menus. I haven't
> > >>>>> narrowed
> it
> > >>>>> down to exactly what features make things better, but I've
> > >>>>> been meaning to.
> > >>>>>
> > >>>>> -deborah