WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: Accessibility user testing

for

From: Dana Douglas
Date: Jul 19, 2016 8:26AM

Hi Zack,

If a user encounters an issue during a usability test, of course you want to observe how they attempt to overcome that issue, but once you have learned what you need to learn, it is often best to move on to the next task so can continue evaluating other aspects of the site. For example, If you observe a screen reader user struggling with a form field that was not labeled properly, you can wait to see how they would attempt to overcome that issue, but there is no need to let them struggle for an extended period of time. By that point, you know what the issue is and that you need to fix it, so you can then move on to the next task. (Also, as Dave said, hopefully these types of issues have been uncovered through automated or manual inspection before the usability test.) Either way, in that scenario, I might help the user by confirming the information they should enter in a form field or directing them to a button or link on the page they could not find. At the same time, I would ask for feedback on what the user expected and how they typically deal with such issues. The bottom line is that you want to make the most of your time with the participant.

Dana


From: Zack McCartney [mailto: <EMAIL REMOVED> ]
Sent: Monday, July 18, 2016 5:40 PM
To: Dana Douglas < <EMAIL REMOVED> >; <EMAIL REMOVED>
Subject: Re: [WebAIM] Accessibility user testing

Hi Dana,

First off, thanks so much for your detailed reply and all the advice. And for the resources, too. This sort of test is totally outside my comfort zone, so I really appreciate the starting points :)

I wanted to follow up on your point about providing assistance to the participant: do you have any advice for how to assist and instruct participants when the encounter barriers?

I'm unsure about doing this well in usability testing in general, even more so with participants using a screenreader.

Thanks again for all your help and welcoming me to the forum!
Zack


On Mon, Jul 18, 2016 at 11:06 AM Dana Douglas < <EMAIL REMOVED> <mailto: <EMAIL REMOVED> >> wrote:
Zack,

Welcome! Including people with disabilities in a usability test is a great way to evaluate a site's accessibility. Here are a few resources you may find helpful (there are certainly others as well) :

Slides from my 2015 UXPA presentation on the topic: http://www.slideshare.net/UXPA/uxpa-2015-why-how-to-include-people-with-disabilities-pw-ds-in-your-usability-testsdouglas-and-davis-6252015-49918852
A UXPA Magazine article by Mary Hunter Utt on the topic: http://uxpamagazine.org/guerilla_tactics/
An article from Deque Systems on the topic: http://www.deque.com/blog/incorporate-users-disabilities-ux-testing/

In general, the test can be virutally the same as any other usability test (same tasks, methodology, etc.). You will want to make sure the web conference tool you're using is accessible for screen reader users (and other assistive technologies). Skype may be the best bet - participants can share their screen through Skype. The test should be as realistic as possible. If users would not have any outside instructions or information in the real world, you should not provide that information during the usabilty test either. However, if that presents a significant barrier (first of all, you know you have an accessibility issue to fix!), but then you can provide assistance to move on and gather additional feedback on other aspects of the site.

Good luck!

Dana Douglas

-----Original Message-----
From: Zack McCartney [mailto: <EMAIL REMOVED> <mailto: <EMAIL REMOVED> >]
Sent: Friday, July 15, 2016 3:10 PM
To: <EMAIL REMOVED> <mailto: <EMAIL REMOVED> >
Subject: [WebAIM] Accessibility user testing

Hi there!

First off, hello all! This is my first post, excited to start learning more about web accessibility.

Anyway, I work at a web development agency and I've been tasked with running a usability test on a web application we've built with a participant using a screenreader. Our development team just made a bunch of updates to the site to move it closer to ADA (Americans with Disabilities
Act) compliance, so we're trying to find out if our first pass actually improved the site's accessibility and what work still needs to be done.

The problem is: I've never run a usability test with a participant using a screenreader. I have basic experience running usability tests, so I have an ok handle on how to moderate a test session, but I want to learn the basics of testing the user-friendliness of web accessibility features.

Specifically:

- Do y'all have any advice on how to test the usability of a site's
accessibility features?


- What adaptations, if any, should I think to make to my typical
usability test setup?
- The participant and I will be connecting over the phone, I'm hoping
over video call, with him sharing his screen. I have no idea if this'll
work or if asking him to navigate through a video conferencing app (Google
Hangouts) could complicate the test unnecessarily.


- Should I provide the participant instructions or can I (or rather,
typical of interacting with the web via screenreader) leave them in the
dark, let them figure out the site on their own?
- For a typical usability test, I'd want to the participant to know
as little as possible about the site under test, as I want to learn how
people figure out how to use a site on first encounter. But, I don't know
if omitting usage instructions — part of our dev team's accessibility work
— would prevent the user from even interacting with the site. I want them
to at the very least to be able to access the site, even if it's still
tricky to use on screenreader.


Thanks!
Zack McCartney

PS Sorry if my question shows my ignorance of web accessibility i.e.
anything sounds goofy or dumb. I'm totally new to the topic, trying to get up to speed. :)