WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: Something's amiss with the latest survey

for

From: Jared Smith
Date: Aug 28, 2015 2:26PM

Karl -

Thanks for starting the dialogue.

First and foremost, we never purport or suggest that the survey responses
are representative of market share. Or that they represent screen reader
users in general. The only thing we can definitively say is that they are
representative of survey respondents. The more respondents there are, the
safer it is to say that the responses *might* begin to be representative of
those other things.

The ZoomText numbers are certainly surprising. AI Squared has indicated to
us that based on their sales and usage data that they felt ZoomText users
were underrepresented on our previous surveys. They know how many licenses
they've sold - and believed that our survey numbers simply didn't jive with
theirs. Others in the broader AT field have suggested the same. AI has made
significant product improvements and have marketed their products well,
including the release of ZoomText for Mac and free Window-Eyes licenses for
Office users. I anticipated an increase, but not nearly this significant.

There are, I think, three primary factors that might explain the increase
in ZoomText use among respondents:
1. ZoomText users were disproportionately represented.
2. The survey reached a broader low vision audience than in previous years.
3. Our perceptions of screen reader usage (as reinforced by previous survey
data) are not accurate.

#1 - AI Squared marketed this survey heavily. But it was also pushed on
distribution lists for almost all of the other screen reader product
mailing lists (as it has for every previous survey we've conducted). Would
the data be more valuable if nobody ever marketed the survey and we had
very few responses? I don't think so.

Is it likely that AI Squared's push for survey respondents resulted in
inflated numbers? Of course. We fully acknowledge this. But I don't think
you can explain the notable increase based on this factor alone.

#2 - As Jennison suggested, the survey was completed by a broader
population and one that included more visually impaired (as opposed to
blind) screen reader users than ever before. Many ZoomText users primarily
use its magnification features and use the screen reader functionality to
supplement magnification. Some may not really consider themselves as
primary users of screen readers, and may not have even known of previous
surveys.

That more of these users completed the survey is wonderful. This means that
the responses are more (not at all fully) representative of screen reader
users in general.

#3 - This is the one that is most important for us to consider - that our
perception of which screen readers are most commonly used is inaccurate.
And that this misperception has been reinforced by previous survey data
that underrepresented ZoomText users. Without significant and rigorous
research, we'll never know. Until that occurs, I think the WebAIM survey
data is the best available.

In reality, the ZoomText jump was probably influenced by all these factors.
Regardless of how accurate or inaccurate you think the ZoomText numbers
might be, I think the overall considerations are the same:
1. ZoomText is a technology that we shouldn't ignore.
2. We need to better consider the low vision experience with screen readers
(primarily ZoomText).
3. There is increasing diversity among screen reader and browser usage
which introduces notable burden on authors to ensure compatibility with
them all - the true solution to which is only increased support for web
standards.
4. We need to be cautious in thinking our experiences and perceptions of
screen reader users are representative of all screen reader users.

The reason we started conducting these surveys was to correct and/or verify
our assumptions. Two things I've learned from our survey data are that they
should be treated with caution and that ignoring them altogether is a
usually a bad idea.

Jared Smith
WebAIM.org