WebAIM - Web Accessibility In Mind

E-mail List Archives

Re: What we found when we tested tools on the world's least-accessible webpage


From: Birkir R. Gunnarsson
Date: Feb 24, 2017 8:35PM

Well said Jared and co.
I really do not like tools that flag every possible error ( often with
little to no user impact) as an error, and flag everything as
As the saying goes "if everything is critical, then nothing is critical.".
For those who may secretly think that I sold out after transferring
to a mainstream company, no, on the contrary, I get more uptight about
usability things that are not necessarily WCAG violations, but I
concentrate on things that I believe make a real difference to real
users, and tools that report a bunch of false or borderline technical
accessibility problems with questionable user impact are not helping
the journey towards a truly inclusive digital experience.
In my evaluation of a variety of accessibility tools, I shockingly
found this was often the case (and, no WebAIM was noet among them, I
can't wait for the WebAIM keyboard accessible Firefox plug-in).

On 2/24/17, Moore,Michael (Accessibility) (HHSC)
< <EMAIL REMOVED> > wrote:
> I agree with that assessment. Some things like empty data cells in a table
> that they suggested was an error, I would have a hard time teaching our
> developers to interpret the results if things like that were flagged. You
> will also be happy to know that our current process for our developers
> starts with the WAVE tool in Chrome and has resulted in a dramatic drop in
> accessibility bugs found at QA time.
> Mike Moore
> EIR (Electronic Information Resources) Accessibility Coordinator
> Texas Health and Human Services Commission
> Civil Rights Office
> (512) 438-3431 (Office)
> Making electronic information and services accessible to people with
> disabilities is everyone's job. I am here to help.
> -----Original Message-----
> From: WebAIM-Forum [mailto: <EMAIL REMOVED> ] On Behalf
> Of Jared Smith
> Sent: Friday, February 24, 2017 3:23 PM
> To: WebAIM Discussion List < <EMAIL REMOVED> >
> Subject: Re: [WebAIM] What we found when we tested tools on the world's
> least-accessible webpage
> Thanks for sharing this. Their premise that automated tools are limited is
> spot on. Their methodology for reviewing the tools, however, is rather
> questionable. Many of the "barriers" that they identified have no or
> negligible end user impact. They seem to suggest that if a tool does not
> indicate an "error" for every possible interpretation of some accessibility
> issue or guideline, that somehow that tool is flawed.
> Our approach with WAVE is to facilitate human evaluation and focus the
> evaluator on things that actually have an impact - not sending them on a
> wild goose chase fixing "errors" that don't have any impact on actual end
> user accessibility. This study would suggest that the tool that flags the
> most "errors" is somehow best.
> They also made significant errors in their analysis of WAVE. I found at
> least 8 items that WAVE readily flags that they somehow overlooked or
> recorded incorrectly. I've notified them of these errors -
> https://github.com/alphagov/accessibility-tool-audit/issues/3 - and hope
> they update their results accordingly.
> Jared Smith
> WebAIM.org
> > > http://webaim.org/discussion/archives
> > > > > >

Work hard. Have fun. Make history.