E-mail List Archives

Search E-mail List Archives for

Number of posts in this thread: 7 (In chronological order)

From: John Foliot
Date: Tue, Jul 27 2010 5:21PM
Subject: Testing with Screen Readers (was RE: Wai Aria how useful?)
No previous message | Next message →

Interesting thread, and some thoughts:

There is a real tension between building standards compliant sites that
work with all technologies versus 'testing' with one tool, and building in
hacks and other 'repair strategies' to support bad software. Just ask
those unfortunates who must still support IE6. If you really must test
with a screen reader, then test with more than one: you test with more
than one browser, right?

The other point that must be considered is that to really be able to
'test' with any screen-reader, you should be a daily screen-reader user.
The problem we have is developers who want to do the right thing, but are
not competent with using Adaptive Technology, so really, all they are
doing is listening to their site read aloud: they are often missing out on
much of the interactivity daily users apply to using their software. For
production class developers, they are often better served by asking (and
paying) daily AT users to do the testing for them, rather than pretending
that opening up NVDA, or the time-crippled JAWs eval copy (which is
illegal, but...) and listening to a site read aloud and calling it
"testing".

Finally, as a very real question, what exactly are you testing *for* when
using a screen reader? If a developer is testing to find holes, I would
suggest they are approaching the problem from the wrong angle: you should
be testing that solutions implemented work as desired. So the developer
should be thinking about the problems and solutions long before they get
to the functional testing stage, and there are plenty of developer tools
out there that can confirm if your code is moving in the right direction
(understanding how a DOM inspector works is crucial); achieving real
accessible content means making some decisions at the foundational stage -
it's architectural. Waiting for a walk-through using AT to find problems
is almost certainly a recipe for ensuring there will be problems to
find...


As always, IMHO

JF



>

From: ckrugman
Date: Tue, Jul 27 2010 6:42PM
Subject: Re: Wai Aria how useful?
← Previous message | Next message →

It is also important to note that here in the U.S. most rehab agencies will
provide JAWS or Window Eyes for their rehab clients who are receiving
equipment.
Chuck
----- Original Message -----
From: < = EMAIL ADDRESS REMOVED = >
To: < = EMAIL ADDRESS REMOVED = >
Sent: Tuesday, July 27, 2010 12:05 PM
Subject: Re: [WebAIM] Wai Aria how useful?


>" NVDA is rapidly growing in popularity. Whilst it lacks some of the
>functionality that other screen readers provide, it's increasingly used as
>a backup option and in time I think we'll see it gain primary ground as
>well.
>
> Jaws is undoubtedly the most popular primary option, but the WebAIM survey
> reports that 49% of people use more than one screen reader. Of the
> alternatives, NVDA is the most popular with nearly 26% of people choosing
> it.
> http://webaim.org/projects/screenreadersurvey2/"
>
> I am very familiar with the results of the WebAIM survey. I was one of the
> respondents. I think that the survey supports my view that NVDA in most
> cases it is not the best platform for user testing. Only 3% of respondents
> listed NVDA as their primary screen reader. I believe that user testing
> should be conducted using the user's primary screen reader and browser
> combination unless you are testing for use in a closed environment where
> the user will not have a choice of platform, including their OS, AT and
> other software.
>
> I believe that there is a risk in assuming that the population of survey
> respondents is fully representative of screen reader users in general. My
> gut instinct is that the population may have been a bit more tech savvy.
> For example 10% of the respondents reported that they do not have a
> disability. I would be very surprised to find that 10% of JAWS users do
> not have a disability. I would also expect that 10% is heavily
> concentrated among accessibility experts. Even among the 90% of
> respondents who reported a disability I would not be surprised to learn
> that there was a disproportionate representation of folks who could be
> considered technology and/or accessibility experts. I am not suggesting
> that the results of the survey are invalid, just that some of the more
> surprising findings such as use of multiple screen readers, and recent
> updates of AT may not truly reflect the screen reader user population in
> general.
>
> NVDA is a good tool for troubleshooting problems. It helps us confirm when
> problems are related to the AT and not the application, document, or
> system under test. We are starting a pilot program to evaluate NVDA and/or
> SAToGo as a supplemental assistive technology for our staff who use a
> screen reader. Since most of our staff who use a screen reader are not
> technologists and most could be considered "average" in their computer
> skills, it will be interesting to get their feedback on the usefulness of
> a second screen reader.
>
> Mike Moore
>
>
>

From: ckrugman
Date: Tue, Jul 27 2010 7:03PM
Subject: Re: Wai Aria how useful?
← Previous message | Next message →

As a user I am referring to the need to test using multiple products to
replicate the actual operating conditions of screen reader users that will
be experienced in the individual setting. Tried and true standards refer to
the widely accessible products that are used on a regular basis by a large
number of screen reader users. In other words the tried and true standards
refer to widely available products that are used e.g. Jaws and Window Eyes
and that testing needs to occur using multiple screen readers to insure
total accessibility.
chuck
----- Original Message -----
From: "Ron Stewart" < = EMAIL ADDRESS REMOVED = >
To: "'WebAIM Discussion List'" < = EMAIL ADDRESS REMOVED = >
Sent: Tuesday, July 27, 2010 2:03 PM
Subject: Re: [WebAIM] Wai Aria how useful?


What testing standards are you referring too, what is tried and true?
Testing with a specific AT product, in this case a brand of screen reader
does not insure any concrete measure of accessibility it only insure
compliance with that particular products approach to access in a given
operating systems with a given set of products.

Ron Stewart

From: Langum, Michael J
Date: Wed, Jul 28 2010 6:48AM
Subject: Re: Testing with Screen Readers (was RE: Wai Aria how useful?)
← Previous message | Next message →

John writes:
> The other point that must be considered is that to really be able to 'test' with any screen-reader,
> you should be a daily screen-reader user. ....

I agree in principle, but in the "real world" time and resources do not permit the additional time and cost of testing by "a daily screen-reader user."

I am only able to do detailed testing at the code level (e.g. "alt" text; semantic structure; <label> for <input>, column and row headers, etc. We do randomized testing on some content, and react immediately if we get any user complaints.

Jared, Do you think there might be value in doing a survey on testing methodologies? Do developers actually test? Do they go beyond the use of automated testing tools? Do they do "functional" testing with various AT products? Do they do this functional testing on each file developed?

-- Mike




>

From: Birkir Rúnar Gunnarsson
Date: Wed, Jul 28 2010 7:00AM
Subject: Re: Testing with Screen Readers (was RE: Wai Aria how useful?)
← Previous message | Next message →

At the conference I attended in Vienna (and some of you attended as
well, I believe) there was talk about the need for screen reader/AT
software simulation tools or plug ins for development environments, so
the developers get a feel for the problem and how their code would
work for a screen reader user.
It is an interesting idea, though there are some limitations, e.g.
that one should expect an SR user, or user of any other AT, to have
developed some skills with the product and be able to do more with it
than, perhaps, a developer would see at first glance.
Also building accessibility testing and compliance into CMS and other
development software would definitely, I think, be a very good step,
and I know there are projects going on in the area.
I see the practical problem of testing with real users, it is
expensive, there are marked differences in what each screen reader
does well and does badly, so you might need different testing based on
your potential user group etc.
Sometimes a particular SR has not implemented technology that exists,
and it is hard for the developer to do anything about it.
Cheers
-B

On 7/28/10, Langum, Michael J < = EMAIL ADDRESS REMOVED = > wrote:
> John writes:
>> The other point that must be considered is that to really be able to
>> 'test' with any screen-reader,
>> you should be a daily screen-reader user. ....
>
> I agree in principle, but in the "real world" time and resources do not
> permit the additional time and cost of testing by "a daily screen-reader
> user."
>
> I am only able to do detailed testing at the code level (e.g. "alt" text;
> semantic structure; <label> for <input>, column and row headers, etc. We do
> randomized testing on some content, and react immediately if we get any user
> complaints.
>
> Jared, Do you think there might be value in doing a survey on testing
> methodologies? Do developers actually test? Do they go beyond the use of
> automated testing tools? Do they do "functional" testing with various AT
> products? Do they do this functional testing on each file developed?
>
> -- Mike
>
>
>
>
>>

From: Jared Smith
Date: Wed, Jul 28 2010 8:39AM
Subject: Re: Testing with Screen Readers (was RE: Wai Aria how useful?)
← Previous message | Next message →

On Wed, Jul 28, 2010 at 6:47 AM, Langum, Michael J wrote:

> Jared,  Do you think there might be value in doing a survey on testing methodologies?

Yes. Stay tuned. ;-)

I don't think that develop testing in screen readers is useless.
There's a lot that can be gained by simply listening to your page in a
screen reader. Of course most of the things that can be identified
this way can also be identified through automated tools, particularly
something like the Text-only View in WAVE, which approximates what a
screen reader might read.

Screen reader testing best identifies interaction and structural
issues that aren't apparent visually or even through reading through
the page. Thus it becomes more vital for more interactive pages.
Fortunately, it's not overly difficult to learn how to use screen
readers to do basic interaction. We have articles for JAWS
(http://webaim.org/articles/jaws/), NVDA
(http://webaim.org/articles/nvda/), and (very soon) VoiceOver to get
you started.

Three additional points:

- Screen reader testing should be but one aspect of a broader
evaluation methodology.

- Some caution needs to be employed in blind screen reader testing.
How do they know if something is inaccessible if it's inaccessible?
It's quite possible that something could be skipped entirely and they
may not even know it. Watching the screen reader tester interact with
the page is very valuable.

- Be careful with accessibility testing with screen reader users.
Instead, conduct usability testing with many users, including screen
reader users. If you focus only on accessibility issues (this image
has poor alt text, this text box is missing a label, etc.) you could
end up with a very technically accessible web site that is entirely
unusable as a whole to screen reader users.

Jared Smith
WebAIM

From: Michael.Moore
Date: Wed, Jul 28 2010 8:42AM
Subject: Re: Testing with Screen Readers (was RE: Wai Aria how useful?)
← Previous message | No next message

Excellent summation of the issue and the considerations John. For whatever its worth to the conversation I will share our testing protocol as a real world example. Assuming that working for a state agency is considered to be the real world.

1. Test for accessibility standards compliance. We use section 508 since this is what is required by Texas law for a state agency. (Hardware, software, office equipment, telecommunications, web and electronic documents) An important note here is that for most vendor generated content and applications this is the limit of what they are contractually bound to do.
2. Test for valid markup. We use W3C validation for HTML/CSS. Unfortunately we often make exceptions here because the authoring tools/CMS systems used do not generate valid code - e.g. MS SharePoint (Exceptions are not made for public content)
3. Test for proper document structure. Headings, lists, tables etc. Again it is occasionally necessary to make exceptions particularly for systems that rely on Table layout for the interface components - e.g. PeopleSoft (Exceptions are not made for public content)
4. Test with agency standard assistive technology. For screen readers this is currently JFW 9 on Windows XP SP3 with IE7 for web content, Adobe Reader 9 for PDF, and MS Office 2007 for Word, Excel, and PPT. 90% of what we test is for internal use, so we control the platform.

Given that approximately 5% of our rely on AT to perform their daily jobs it is absolutely critical that all electronic information resources that they use work with their assistive technology. Thus the strong emphasis on ensuring that everything works with our standard AT.

Mike Moore
(512) 424-4159