WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: Screen Reader Survey Results

for

From: Jared Smith
Date: Jun 6, 2012 1:20PM

Tim, thank you so much for your thoughts. They are very valuable.

On Wed, Jun 6, 2012 at 12:49 PM, Tim Harshbarger wrote:
> However, I tend to think of it as more advisory than conclusive.

I don't think it's every been suggested that the surveys are
conclusive. We know the screen reader survey is not representative of
all screen reader users. We would love to do more in-depth research
with controlled subjects to analyze actual behaviors and preferences.
We just need someone to fund such an expensive and difficult research
project.

> User surveys like this measure what people believe their behavior is and not actually what their behavior is.

All surveys do this. Researching beyond perceived behavior is
difficult. Of bigger concern is that presenting perceived behavior as
fact can be detrimental to accessibility. If you ask users if they
want a feature or not, they'll almost always say they want it, though
if every site provided that feature it would probably make the web
less accessible.

> From the survey, I get that the respondents definitely felt heading levels were useful.  I'm not sure that tells me how they find it useful.  Does it have to be h1, h2, h3?

These are things that are hard to get at in a survey. There were many
respondents that indicated in the comments that they had no idea what
we even meant by "headings levels" or "heading 1", so asking more
specific details would just cause more confusion for many respondents.

> Honestly, I saw some groups pushing a lot harder to get their members to take the survey this year.

This is a concern that we have as well. We never intend it to be a
popularity contest, but it is in some ways. Enough so that there was
one respondent that took over 5 hours to complete the survey nearly
150 times so he could "better represent" his favorite screen reader
and to lodge his repeated complaints against Facebook accessibility
(to be clear, duplicate responses are NOT included in the survey
results). At the same time, we love that we're getting more
respondents to the surveys which lends more credibility to the
results.

Jared