E-mail List Archives
Thread: Differences Between Testing With JAWS And NVDA
Number of posts in this thread: 10 (In chronological order)
From: Jim Homme
Date: Fri, Mar 18 2022 8:00AM
Subject: Differences Between Testing With JAWS And NVDA
No previous message | Next message →
Hi,
I usually do accessibility testing with NVDA. For those of you who test with JAWS, what differences do you notice when comparing the results between the two screen readers and what JAWS settings do you change from the defaults?
Thanks.
Jim
=========Jim Homme
Senior Digital Accessibility Consultant
Bender Consulting Services
412-787-8567
https://www.benderconsult.com/
Support the dreams of independence through employment for students with disabilities with your Amazon purchases.
https://smile.amazon.com/ch/83-0988251
From: Birkir R. Gunnarsson
Date: Fri, Mar 18 2022 8:11AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
NVDA is better for testing because it does not try to make up for
missing or incorrect coding, specifically it does not try to guess the
intended label of an unlabeled form control.
Jaws does that, which makes a lot of sense for regular users, but is
bad for testers, it hides a real problem. Also Jaws gets it wrong
sometimes, about 5 to 10% of the itme, in my rough estimate.
I love testing with Jaws because I can quickly find the problematic
element by using the displayAdvancedElementInfo script with a keyobard
key, it gives me instant insight into where the problem lies and helps
me find the element more quickly in the developer tools (presing
shift-f10 does not always work to get there).
Whether I'm using NVDA or Jaws, I often force the app into
forms/application mode (to test whether a custom element can be
activated with the keyboard, if you are in browse mode and press enter
or spacebar, the screen reader sends a click event, not a keyboard
event, to the element, so you can activate an element with a screen
reader that you couldn't with the keyboard only).
I also change the display graphics setting in Jaws to "show all
images" to see which ones do not have alt text and also I have to
alter the settings to see all the ARIA landmarks.
Other than that I try to test with the default values, that is the
most predictable screen reader user experience.
Cheers
-B
On 3/18/22, Jim Homme < = EMAIL ADDRESS REMOVED = > wrote:
> Hi,
> I usually do accessibility testing with NVDA. For those of you who test with
> JAWS, what differences do you notice when comparing the results between the
> two screen readers and what JAWS settings do you change from the defaults?
>
> Thanks.
>
> Jim
>
> =========> Jim Homme
> Senior Digital Accessibility Consultant
> Bender Consulting Services
> 412-787-8567
> https://www.benderconsult.com/
> Support the dreams of independence through employment for students with
> disabilities with your Amazon purchases.
> https://smile.amazon.com/ch/83-0988251
>
> > > > >
--
Work hard. Have fun. Make history.
From: Jim Homme
Date: Fri, Mar 18 2022 8:26AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
Hi,
One reason I'm asking is that NVDA does not seem to show some unlabeled images.
Jim
=========Jim Homme
Senior Digital Accessibility Consultant
Bender Consulting Services
412-787-8567
https://www.benderconsult.com/
Support the dreams of independence through employment for students with disabilities with your Amazon purchases.
https://smile.amazon.com/ch/83-0988251
-----Original Message-----
From: WebAIM-Forum < = EMAIL ADDRESS REMOVED = > On Behalf Of Birkir R. Gunnarsson
Sent: Friday, March 18, 2022 10:12 AM
To: WebAIM Discussion List < = EMAIL ADDRESS REMOVED = >
Subject: Re: [WebAIM] Differences Between Testing With JAWS And NVDA
NVDA is better for testing because it does not try to make up for missing or incorrect coding, specifically it does not try to guess the intended label of an unlabeled form control.
Jaws does that, which makes a lot of sense for regular users, but is bad for testers, it hides a real problem. Also Jaws gets it wrong sometimes, about 5 to 10% of the itme, in my rough estimate.
I love testing with Jaws because I can quickly find the problematic element by using the displayAdvancedElementInfo script with a keyobard key, it gives me instant insight into where the problem lies and helps me find the element more quickly in the developer tools (presing
shift-f10 does not always work to get there).
Whether I'm using NVDA or Jaws, I often force the app into forms/application mode (to test whether a custom element can be activated with the keyboard, if you are in browse mode and press enter or spacebar, the screen reader sends a click event, not a keyboard event, to the element, so you can activate an element with a screen reader that you couldn't with the keyboard only).
I also change the display graphics setting in Jaws to "show all images" to see which ones do not have alt text and also I have to alter the settings to see all the ARIA landmarks.
Other than that I try to test with the default values, that is the most predictable screen reader user experience.
Cheers
-B
On 3/18/22, Jim Homme < = EMAIL ADDRESS REMOVED = > wrote:
> Hi,
> I usually do accessibility testing with NVDA. For those of you who
> test with JAWS, what differences do you notice when comparing the
> results between the two screen readers and what JAWS settings do you change from the defaults?
>
> Thanks.
>
> Jim
>
> =========> Jim Homme
> Senior Digital Accessibility Consultant Bender Consulting Services
> 412-787-8567
> https://www.benderconsult.com/
> Support the dreams of independence through employment for students
> with disabilities with your Amazon purchases.
> https://smile.amazon.com/ch/83-0988251
>
> > > archives at http://webaim.org/discussion/archives
> >
--
Work hard. Have fun. Make history.
From: Nathan Clark
Date: Fri, Mar 18 2022 8:31AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
I find that NVDA is way better than Jaws when it comes to testing.
On 3/18/22, Jim Homme < = EMAIL ADDRESS REMOVED = > wrote:
> Hi,
> One reason I'm asking is that NVDA does not seem to show some unlabeled
> images.
>
> Jim
>
> =========> Jim Homme
> Senior Digital Accessibility Consultant
> Bender Consulting Services
> 412-787-8567
> https://www.benderconsult.com/
> Support the dreams of independence through employment for students with
> disabilities with your Amazon purchases.
> https://smile.amazon.com/ch/83-0988251
>
> -----Original Message-----
> From: WebAIM-Forum < = EMAIL ADDRESS REMOVED = > On Behalf Of
> Birkir R. Gunnarsson
> Sent: Friday, March 18, 2022 10:12 AM
> To: WebAIM Discussion List < = EMAIL ADDRESS REMOVED = >
> Subject: Re: [WebAIM] Differences Between Testing With JAWS And NVDA
>
> NVDA is better for testing because it does not try to make up for missing or
> incorrect coding, specifically it does not try to guess the intended label
> of an unlabeled form control.
> Jaws does that, which makes a lot of sense for regular users, but is bad for
> testers, it hides a real problem. Also Jaws gets it wrong sometimes, about 5
> to 10% of the itme, in my rough estimate.
> I love testing with Jaws because I can quickly find the problematic element
> by using the displayAdvancedElementInfo script with a keyobard key, it gives
> me instant insight into where the problem lies and helps me find the element
> more quickly in the developer tools (presing
> shift-f10 does not always work to get there).
> Whether I'm using NVDA or Jaws, I often force the app into
> forms/application mode (to test whether a custom element can be activated
> with the keyboard, if you are in browse mode and press enter or spacebar,
> the screen reader sends a click event, not a keyboard event, to the element,
> so you can activate an element with a screen reader that you couldn't with
> the keyboard only).
> I also change the display graphics setting in Jaws to "show all images" to
> see which ones do not have alt text and also I have to alter the settings to
> see all the ARIA landmarks.
>
> Other than that I try to test with the default values, that is the most
> predictable screen reader user experience.
> Cheers
> -B
>
> On 3/18/22, Jim Homme < = EMAIL ADDRESS REMOVED = > wrote:
>> Hi,
>> I usually do accessibility testing with NVDA. For those of you who
>> test with JAWS, what differences do you notice when comparing the
>> results between the two screen readers and what JAWS settings do you
>> change from the defaults?
>>
>> Thanks.
>>
>> Jim
>>
>> =========>> Jim Homme
>> Senior Digital Accessibility Consultant Bender Consulting Services
>> 412-787-8567
>> https://www.benderconsult.com/
>> Support the dreams of independence through employment for students
>> with disabilities with your Amazon purchases.
>> https://smile.amazon.com/ch/83-0988251
>>
>> >> >> archives at http://webaim.org/discussion/archives
>> >>
>
>
> --
> Work hard. Have fun. Make history.
> > > http://webaim.org/discussion/archives
> > > > > >
--
Nathan Clark
QA Automation Analyst Tech team
Accessibility assistant
CPACC
cell: 410-446-7259
email: = EMAIL ADDRESS REMOVED =
101 Village Blvd
Princeton, NJ 08540
SMBE & Minority Owned Business
--
Webinar Registration - Zoom
<https://zoom.us/webinar/register/WN_okvOMWHMSNKsnCSEBxap7A>
From: Geethavani.Shamanna
Date: Fri, Mar 18 2022 8:33AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
I recently tested a modal dialog with both NVDA and Jaws. With NVDA the dialog worked as expected. However, with Jaws, with the virtual cursor turned on, I was able to access other parts of the page when within the dialog.
I have found Jaws unreliable in other instances as well, particularly when testing forms.
-----Original Message-----
From: WebAIM-Forum < = EMAIL ADDRESS REMOVED = > On Behalf Of Birkir R. Gunnarsson
Sent: 18 March 2022 14:12
To: WebAIM Discussion List < = EMAIL ADDRESS REMOVED = >
Subject: Re: [WebAIM] Differences Between Testing With JAWS And NVDA
CAUTION: This mail comes from outside the University. Please consider this before opening attachments, clicking links, or acting on the content.
NVDA is better for testing because it does not try to make up for missing or incorrect coding, specifically it does not try to guess the intended label of an unlabeled form control.
Jaws does that, which makes a lot of sense for regular users, but is bad for testers, it hides a real problem. Also Jaws gets it wrong sometimes, about 5 to 10% of the itme, in my rough estimate.
I love testing with Jaws because I can quickly find the problematic element by using the displayAdvancedElementInfo script with a keyobard key, it gives me instant insight into where the problem lies and helps me find the element more quickly in the developer tools (presing
shift-f10 does not always work to get there).
Whether I'm using NVDA or Jaws, I often force the app into forms/application mode (to test whether a custom element can be activated with the keyboard, if you are in browse mode and press enter or spacebar, the screen reader sends a click event, not a keyboard event, to the element, so you can activate an element with a screen reader that you couldn't with the keyboard only).
I also change the display graphics setting in Jaws to "show all images" to see which ones do not have alt text and also I have to alter the settings to see all the ARIA landmarks.
Other than that I try to test with the default values, that is the most predictable screen reader user experience.
Cheers
-B
On 3/18/22, Jim Homme < = EMAIL ADDRESS REMOVED = > wrote:
> Hi,
> I usually do accessibility testing with NVDA. For those of you who
> test with JAWS, what differences do you notice when comparing the
> results between the two screen readers and what JAWS settings do you change from the defaults?
>
> Thanks.
>
> Jim
>
> =========> Jim Homme
> Senior Digital Accessibility Consultant Bender Consulting Services
> 412-787-8567
> https://www.benderconsult.com/
> Support the dreams of independence through employment for students
> with disabilities with your Amazon purchases.
> https://smile.amazon.com/ch/83-0988251
>
> > > archives at http://webaim.org/discussion/archives
> >
--
Work hard. Have fun. Make history.
From: Patrick H. Lauke
Date: Fri, Mar 18 2022 8:38AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
On 18/03/2022 14:33, Geethavani.Shamanna wrote:
> I recently tested a modal dialog with both NVDA and Jaws. With NVDA the dialog worked as expected. However, with Jaws, with the virtual cursor turned on, I was able to access other parts of the page when within the dialog.
Indeed, I think this is down to NVDA doing some extra magic when it sees
aria-modal="true", while JAWS still doesn't.
P
--
Patrick H. Lauke
https://www.splintered.co.uk/ | https://github.com/patrickhlauke
https://flickr.com/photos/redux/ | https://www.deviantart.com/redux
twitter: @patrick_h_lauke | skype: patrick_h_lauke
From: Steve Green
Date: Fri, Mar 18 2022 8:43AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
We don't change anything from the default except the speech rate.
With regard to the differences, it depends on the objective of the testing. When doing a WCAG audit, we mostly use screen readers to help us understand the code, not to actually test anything. For this purpose, any differences are irrelevant. With extremely few exceptions, you should not be making pass / fail decisions on the basis of screen reader behaviours. Decisions should be based on inspection of the code and user interface.
We do use screen readers to test that ARIA live regions are working correctly because it can be difficult or impossible to test this by code inspection, depending on how it has been implemented. We see some differences, such as JAWS announcing messages twice, but usually these are just peculiarities of the screen readers and coding methods and don't affect the pass / fail decision.
When doing an expert review, we are interested in the user experience, so we test with both JAWS and NVDA with the aim of achieving a website that works well with both of them. As such, it doesn't matter if one is more "correct" than the other. There are people out there using both of them, so websites need to work well with both.
In my experience, most of the differences in behaviour arise from bad coding that would violate one or more WCAG success criteria. If a website is fully WCAG 2.1 AA conformant, there are not usually many differences.
Steve Green
Managing Director
Test Partners Ltd
-----Original Message-----
From: WebAIM-Forum < = EMAIL ADDRESS REMOVED = > On Behalf Of Jim Homme
Sent: 18 March 2022 14:00
To: WebAIM Discussion List < = EMAIL ADDRESS REMOVED = >
Subject: [WebAIM] Differences Between Testing With JAWS And NVDA
Hi,
I usually do accessibility testing with NVDA. For those of you who test with JAWS, what differences do you notice when comparing the results between the two screen readers and what JAWS settings do you change from the defaults?
Thanks.
Jim
=========Jim Homme
Senior Digital Accessibility Consultant
Bender Consulting Services
412-787-8567
https://www.benderconsult.com/
Support the dreams of independence through employment for students with disabilities with your Amazon purchases.
https://smile.amazon.com/ch/83-0988251
From: Patrick H. Lauke
Date: Fri, Mar 18 2022 8:44AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
On 18/03/2022 14:00, Jim Homme wrote:
> Hi,
> I usually do accessibility testing with NVDA. For those of you who test with JAWS, what differences do you notice when comparing the results between the two screen readers and what JAWS settings do you change from the defaults?
Two aspects where I find the most differences:
* if focus is lost (e.g. you activate a button, and the button is
dynamically changed to be "disabled", or is set to display:none, or
something similar where whatever element you had focus on is yanked from
the page/focus cycle), most modern browsers silently error-correct for
this next time you TAB/SHIFT+TAB, trying to find the closest next
focusable element to whatever used to have the focus before and now
isn't there anymore. with NVDA, this seems to also apply to using
reading keys. As the browser heuristics/error correction fallback isn't
really reliable, this can give a false sense that the author has
correctly handled focus. Testing with JAWS however, as soon as the
currently focused element disappears, JAWS resets to the start of the
document unless focus has been explicitly handled via JS to go somewhere
else. I prefer the latter for testing.
* NVDA seems be be a lot more lenient (at least when testing with
Chrome) to things like CSS-based images where the container has been
given an aria-label, e.g. <span class="icon" aria-label="Delete"></span>
- it announces the aria-label most of the times. With JAWS, it only
works reliably if the author also gave the container an actual proper
role, like role="img" or similar. Again, better to test in JAWS in these
cases to spot this.
P
--
Patrick H. Lauke
https://www.splintered.co.uk/ | https://github.com/patrickhlauke
https://flickr.com/photos/redux/ | https://www.deviantart.com/redux
twitter: @patrick_h_lauke | skype: patrick_h_lauke
From: Patrick H. Lauke
Date: Fri, Mar 18 2022 8:46AM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | Next message →
...and piggy-backing on what Steve Green just posted, yes - these
differences I mentioned (focus lost/reset, use of aria-label where it's
not consistently supported) would be picked up in a deeper technical
review. but to get an initial feel, JAWS surfaces these things much more
readily
--
Patrick H. Lauke
https://www.splintered.co.uk/ | https://github.com/patrickhlauke
https://flickr.com/photos/redux/ | https://www.deviantart.com/redux
twitter: @patrick_h_lauke | skype: patrick_h_lauke
From: David Engebretson Jr.
Date: Fri, Mar 18 2022 5:13PM
Subject: Re: Differences Between Testing With JAWS And NVDA
← Previous message | No next message
When I test I usually test with NVDA, JAWS, and Voiceover on Chrome, Firefox, and Edge.
The matrix of work exposes errors in the user agent (the browser) and/or the assistive technology or the code itself.
It's a lot of work but it is really helpful to test with detail so we can determine where the error is actually coming from.
Keep up the great work!
David
-----Original Message-----
From: WebAIM-Forum < = EMAIL ADDRESS REMOVED = > On Behalf Of Jim Homme
Sent: Friday, March 18, 2022 7:00 AM
To: WebAIM Discussion List < = EMAIL ADDRESS REMOVED = >
Subject: [WebAIM] Differences Between Testing With JAWS And NVDA
Hi,
I usually do accessibility testing with NVDA. For those of you who test with JAWS, what differences do you notice when comparing the results between the two screen readers and what JAWS settings do you change from the defaults?
Thanks.
Jim
=========Jim Homme
Senior Digital Accessibility Consultant
Bender Consulting Services
412-787-8567
https://www.benderconsult.com/
Support the dreams of independence through employment for students with disabilities with your Amazon purchases.
https://smile.amazon.com/ch/83-0988251