E-mail List Archives

Search E-mail List Archives for

Number of posts in this thread: 12 (In chronological order)

From: Chris Hoffman
Date: Tue, Aug 24 2010 8:45PM
Subject: Proposal for an online, crowdsourced accessibility testing platform
No previous message | Next message →

Hello everyone,

I've been reading this list for several years now, and it's been hard
not to notice that many, if not most, of the questions posed are of
the form, "Is writing HTML like such-and-such accessible?" or "What is
the most accessible way to put X on my web page?"

It has also become apparent, time and again, that although there are
many ways to increase the accessibility of content online, such as
adhering to WCAG guidelines or using online accessibility checkers
like WAVE or Cynthia Says, the only way to _really_ ensure that
content is accessible is to test it with actual people with actual
disabilities who are using actual assistive technology. That is, a web
page that passes all of the automated accessibility checks and adheres
to accessibility standards is not accessible unless it can be used by
real people.

To that end, I have had the following idea rolling around in my head
for quite a while: Why not create a site that presents pairs of
alternative HTML snippets and asks users out in the universe whether
each one is more of less accessible? Visitors could (anonymously)
record their choices ("A is slightly more accessible than B"), as well
as any specific notes and the assistive technology they were using,
and the resulting data could be made available to web designers and
developers.

My first question is, does anything like this (namely, open A/B
testing for accessibility) already exist?
The second is, regardless of the answer to the first question, whether
there is anyone on this list who would be interested in collaborating
on such a project?

I'm taking it as a given that there are lots of questions that would
need to be answered to make this actually work, but for now am taking
the initial step of getting it out of my head and into the open.

Thanks & regards,

Chris

From: D Hubbard
Date: Wed, Aug 25 2010 4:03AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

Chris,

I don't know if anything like this exists but I think it's a good idea and
does it really matter that much if there's more than one site for the same
purpose? Keep us posted, I'll be interested.

The thing too is some of us have more than one screen reader, and the
feedback, I would think would be quite interesting and informative.

-Diane
----- Original Message -----
From: "Chris Hoffman" < = EMAIL ADDRESS REMOVED = >
To: "WebAIM Discussion List" < = EMAIL ADDRESS REMOVED = >
Sent: Tuesday, August 24, 2010 10:44 PM
Subject: [WebAIM] Proposal for an online,crowdsourced accessibility testing
platform


> Hello everyone,
>
> I've been reading this list for several years now, and it's been hard
> not to notice that many, if not most, of the questions posed are of
> the form, "Is writing HTML like such-and-such accessible?" or "What is
> the most accessible way to put X on my web page?"
>
> It has also become apparent, time and again, that although there are
> many ways to increase the accessibility of content online, such as
> adhering to WCAG guidelines or using online accessibility checkers
> like WAVE or Cynthia Says, the only way to _really_ ensure that
> content is accessible is to test it with actual people with actual
> disabilities who are using actual assistive technology. That is, a web
> page that passes all of the automated accessibility checks and adheres
> to accessibility standards is not accessible unless it can be used by
> real people.
>
> To that end, I have had the following idea rolling around in my head
> for quite a while: Why not create a site that presents pairs of
> alternative HTML snippets and asks users out in the universe whether
> each one is more of less accessible? Visitors could (anonymously)
> record their choices ("A is slightly more accessible than B"), as well
> as any specific notes and the assistive technology they were using,
> and the resulting data could be made available to web designers and
> developers.
>
> My first question is, does anything like this (namely, open A/B
> testing for accessibility) already exist?
> The second is, regardless of the answer to the first question, whether
> there is anyone on this list who would be interested in collaborating
> on such a project?
>
> I'm taking it as a given that there are lots of questions that would
> need to be answered to make this actually work, but for now am taking
> the initial step of getting it out of my head and into the open.
>
> Thanks & regards,
>
> Chris
>

From: Tim Harshbarger
Date: Wed, Aug 25 2010 7:09AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

Chris,

I think this is a good idea. I might be willing to assist.

We would probably want to capture information on why someone felt
something was more or less accessible.

I am also wondering if there might be ways to capture performance
information as well. The fact is user comments are likely to provide a
lot of preference information and maybe not much performance
information. I find I am also interested in knowing things like which
approach allows the user to complete a task quicker and which approach
is less likely to introduce errors into the user's ability to complete
the task.

It may also be possible to use a site like this to provide information
on how well specific versions of AT support accessibility features.

Thanks!
Tim

From: deborah.kaplan
Date: Wed, Aug 25 2010 7:48AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

Chris Hoffman wrote:

> To that end, I have had the following idea rolling around in my head
> for quite a while: Why not create a site that presents pairs of
> alternative HTML snippets and asks users out in the universe whether
> each one is more of less accessible?

I think this is a wonderful idea, Chris, and I would find it very
useful as both the developer and as a user of adaptive
technology.

One thing that might be useful would be the ability for a
relatively large pool of developers (either open or requiring
some kind of registration) could add code snippets. So instead of
it being a site where somebody had to gateway in all of the code
being compared, a developer could just say "I've been considering
the following two pieces of code; could you guys compare?"

Actually, over at Dreamwidth, we have something like this (for
our code base in particular, of course; not in general). We have
the dw_accessibility community
(<http://dw-accessibility.dreamwidth.org/>;), where developers
regularly post questions, often with two separate samples of user
interfaces, and ask the community to pitch in with opinions about
which ones are more accessible and why. It's been a fabulous
resource for the developers, and many of the users with
disabilities have indicated that they've been very pleased with
the site improvements and how they've been focused on serving the
user base's needs. I would love it if we got a larger pool of
participants over there, and I'd welcome anyone -- developers or
end-users, with or without disabilities -- to join the community
and pitch in. Let me know if you need an invite code for a
Dreamwidth account, and I will happily send you one.

(Dreamwidth is a blogging and social networking platform with a
fabulous diversity statement which has had accessibility as a
top-level priority from its inception.)

In any case, I'd love to see something like this spread out to
being a more general resource for ALL developers, not just
developers of that particular code base. Hey, it could even be
run on a Dreamwidth community. :-)

(Please forgive my evangelism. I get very bouncy about working on
a project where the developers care so much about universal
design and accessibility.)

-Deborah

From: Jared Smith
Date: Wed, Aug 25 2010 8:36AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

We've mulled around the idea of taking this approach with a future
WebAIM screen reader survey - presenting various paired samples and
asking which is more accessible. The fundamental problem and
difficulty with this approach is that things that are inaccessible are
inaccessible - you'd have to be very careful that you're asking the
questions and presenting the samples in a way that is useful and gives
you accurate data. It is quite likely that a screen reader user could
identify something as being the most accessible while a significant
amount of inaccessible content or functionality was not presented to
them. How would they know the content is missing?

It's difficult to analyze the accessibility of distinct elements in a
vacuum entirely separated from a broader web page or online
experience.

WebAIM would be interested in collaborating on such a project. We
actually have quite a few ideas noted already.

Jared Smith
WebAIM

On Tue, Aug 24, 2010 at 8:44 PM, Chris Hoffman < = EMAIL ADDRESS REMOVED = > wrote:
> Hello everyone,
>
> I've been reading this list for several years now, and it's been hard
> not to notice that many, if not most, of the questions posed are of
> the form, "Is writing HTML like such-and-such accessible?" or "What is
> the most accessible way to put X on my web page?"
>
> It has also become apparent, time and again, that although there are
> many ways to increase the accessibility of content online, such as
> adhering to WCAG guidelines or using online accessibility checkers
> like WAVE or Cynthia Says, the only way to _really_ ensure that
> content is accessible is to test it with actual people with actual
> disabilities who are using actual assistive technology. That is, a web
> page that passes all of the automated accessibility checks and adheres
> to accessibility standards is not accessible unless it can be used by
> real people.
>
> To that end, I have had the following idea rolling around in my head
> for quite a while: Why not create a site that presents pairs of
> alternative HTML snippets and asks users out in the universe whether
> each one is more of less accessible? Visitors could (anonymously)
> record their choices ("A is slightly more accessible than B"), as well
> as any specific notes and the assistive technology they were using,
> and the resulting data could be made available to web designers and
> developers.
>
> My first question is, does anything like this (namely, open A/B
> testing for accessibility) already exist?
> The second is, regardless of the answer to the first question, whether
> there is anyone on this list who would be interested in collaborating
> on such a project?
>
> I'm taking it as a given that there are lots of questions that would
> need to be answered to make this actually work, but for now am taking
> the initial step of getting it out of my head and into the open.
>
> Thanks & regards,
>
> Chris
>

From: Terrill Bennett
Date: Fri, Aug 27 2010 9:30AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

I haven't seen an offer to host an accessibility project, so in order
to get this project-thingy off-center...

* I host my domains with DreamHost.com. They offer unlimited
everything that counts (domains, sub domains, mySql databases, disk
space, bandwith, user's, email accounts, etc).

* Think LAMP.

* The downside is: it's a shared server. "Timing" things could be a problem.

* Dedicated hosting is available, for a fee: $69 USD/month with one
year pre-payment. Other options available.

* It is possible to set-up user's with SSH access to a specific
domain, for those who choose to help administrate.

* Willing to set-up any software we agree on such as Joomla,
WordPress, Moodle, Wikimedia, Twiki, Drupal, phpBBS,
Your_Favorite_Package_Goes_Here.

So there it is. Feel free to up the ante with your own offer!

-- Terrill --



> We've mulled around the idea of taking this approach with a future
> WebAIM screen reader survey - presenting various paired samples and
> asking which is more accessible. The fundamental problem and
> difficulty with this approach is that things that are inaccessible are
> inaccessible - you'd have to be very careful that you're asking the
> questions and presenting the samples in a way that is useful and gives
> you accurate data. It is quite likely that a screen reader user could
> identify something as being the most accessible while a significant
> amount of inaccessible content or functionality was not presented to
> them. How would they know the content is missing?
>
> It's difficult to analyze the accessibility of distinct elements in a
> vacuum entirely separated from a broader web page or online
> experience.
>
> WebAIM would be interested in collaborating on such a project. We
> actually have quite a few ideas noted already.
>
> Jared Smith
> WebAIM
>
>

From: Bevi Chagnon | PubCom
Date: Fri, Aug 27 2010 11:54AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

I'm very excited about this proposal.

I'd like to contribute examples for testing that I run into with federal
government documents, especially those that are more technical in nature.
WCAG 2 and ITC 2 (508 for the US fed govt) don't address many of the
situations my govt clients are in.

I can contribute services:
-- technical samples (including Word documents, Excel spreadsheets, PDFs
made from InDesign and office software)
-- writing and online training demonstrating the technique/best practice.

But we'll need someone to manage this project and coordinate all the
players, tasks, and final recommendations.
Any suggestions?

--Bevi

: : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : :
Bevi Chagnon | PubCom | = EMAIL ADDRESS REMOVED = | 301-585-8805
Government publishing specialists, trainers, consultants | print, press,
web, Acrobat PDF & 508
Online at the blog: It's 2010. Where's your career heading?
www.pubcom.com/newsletter

From: Ben Caldwell
Date: Fri, Aug 27 2010 4:03PM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

Hi Chris,

As WCAG 2.0 was entering the candidate recommendation stage in late
2008, we built an experimental tool to collect data on accessibility
support for a variety of technology uses and techniques. While it does
not include the ability to directly compare different techniques, it is
similar in many ways to what you're describing.

Accessibility Supported
http://accsupported.trace.wisc.edu

Behind the scenes, the site includes a basic interface that allows users
to submit test results with comments about what they found. Users with
appropriate permissions can then compare submitted test results with
existing data and collaborate on whether the results should be published.

Jared is right that it's difficult to look at some of this in a vacuum.
It's also a challenge to gather data that is useful over time given the
frequency with which new versions of operating systems, browsers and
assistive technologies are released.

We've had limited resources for further development on this project, but
would be happy to talk with you and others who may be interested in
doing something like this about what we learned in the process.

Thanks,

-Ben

--
Ben Caldwell |< = EMAIL ADDRESS REMOVED = >
Trace Research and Development Center<http://trace.wisc.edu>;


On 08/24/2010 09:44 PM, Chris Hoffman wrote:
> Hello everyone,
>
> I've been reading this list for several years now, and it's been hard
> not to notice that many, if not most, of the questions posed are of
> the form, "Is writing HTML like such-and-such accessible?" or "What is
> the most accessible way to put X on my web page?"
>
> It has also become apparent, time and again, that although there are
> many ways to increase the accessibility of content online, such as
> adhering to WCAG guidelines or using online accessibility checkers
> like WAVE or Cynthia Says, the only way to _really_ ensure that
> content is accessible is to test it with actual people with actual
> disabilities who are using actual assistive technology. That is, a web
> page that passes all of the automated accessibility checks and adheres
> to accessibility standards is not accessible unless it can be used by
> real people.
>
> To that end, I have had the following idea rolling around in my head
> for quite a while: Why not create a site that presents pairs of
> alternative HTML snippets and asks users out in the universe whether
> each one is more of less accessible? Visitors could (anonymously)
> record their choices ("A is slightly more accessible than B"), as well
> as any specific notes and the assistive technology they were using,
> and the resulting data could be made available to web designers and
> developers.
>
> My first question is, does anything like this (namely, open A/B
> testing for accessibility) already exist?
> The second is, regardless of the answer to the first question, whether
> there is anyone on this list who would be interested in collaborating
> on such a project?
>
> I'm taking it as a given that there are lots of questions that would
> need to be answered to make this actually work, but for now am taking
> the initial step of getting it out of my head and into the open.
>
> Thanks& regards,
>
> Chris
>

From: Christophe Strobbe
Date: Fri, Aug 27 2010 8:30PM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

Hi,

This proposal reminds me of the BenToWeb project, in which I was
involved a few years ago. The idea was to create a suite of test
files for WCAG 2.0. We developed tools and a metadata format ("Test
Case Description Language - TCDL") to describe the test files. The
test suites are still available at <http://www.bentoweb.org/ts>;, and
the tools at <http://www.bentoweb.org/tools>;. Unfortunately, the
project ended before WCAG 2 reached Candidate Recommendation. After
the end of the project, part of the last test suite was transferred
to WAI (see <http://www.w3.org/WAI/ER/tests/>;), with an updated
metadata format (TCDL 2.0) that was not tied to the conventions used
in BenToWeb. If you are interested in working on this kind of
activity, please contact Shadi Abou-Zahra at W3C and myself.

Best regars,

Christophe Strobbe


At 04:44 25/08/2010, Chris Hoffman wrote:
>Hello everyone,
>
>I've been reading this list for several years now, and it's been hard
>not to notice that many, if not most, of the questions posed are of
>the form, "Is writing HTML like such-and-such accessible?" or "What is
>the most accessible way to put X on my web page?"
>
>It has also become apparent, time and again, that although there are
>many ways to increase the accessibility of content online, such as
>adhering to WCAG guidelines or using online accessibility checkers
>like WAVE or Cynthia Says, the only way to _really_ ensure that
>content is accessible is to test it with actual people with actual
>disabilities who are using actual assistive technology. That is, a web
>page that passes all of the automated accessibility checks and adheres
>to accessibility standards is not accessible unless it can be used by
>real people.
>
>To that end, I have had the following idea rolling around in my head
>for quite a while: Why not create a site that presents pairs of
>alternative HTML snippets and asks users out in the universe whether
>each one is more of less accessible? Visitors could (anonymously)
>record their choices ("A is slightly more accessible than B"), as well
>as any specific notes and the assistive technology they were using,
>and the resulting data could be made available to web designers and
>developers.
>
>My first question is, does anything like this (namely, open A/B
>testing for accessibility) already exist?
>The second is, regardless of the answer to the first question, whether
>there is anyone on this list who would be interested in collaborating
>on such a project?
>
>I'm taking it as a given that there are lots of questions that would
>need to be answered to make this actually work, but for now am taking
>the initial step of getting it out of my head and into the open.
>
>Thanks & regards,
>
>Chris


--
Christophe Strobbe
K.U.Leuven - Dept. of Electrical Engineering - SCD
Research Group on Document Architectures
Kasteelpark Arenberg 10 bus 2442
B-3001 Leuven-Heverlee
BELGIUM
tel: +32 16 32 85 51
http://www.docarch.be/
Twitter: @RabelaisA11y
---
"Better products and services through end-user empowerment"
http://www.usem-net.eu/
---
Please don't invite me to LinkedIn, Facebook, Quechup or other
"social networks". You may have agreed to their "privacy policy", but
I haven't.

From: Chris Hoffman
Date: Fri, Aug 27 2010 10:39PM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

There a lot of great comments and suggestions here, and I'm excited to
see so much enthusiasm for this project. Here are few of my thoughts,
after having read all of yours:

1. The HTML snippets that we test should be presented in the context
of a larger site. I'm thinking that we could build a dummy company or
university site, or at least selected pages from such a site, and
present them with only the code that's under test being changed.

2. The caveat of #1 is that the contextual part of the tests (as well
as instructions for the testers and any other supplementary material)
would have to be as close to universally accessible as possible.

3. For each test presentation, testers will each be directed to just
one of the two alternatives. They will be given specific tasks
(navigate to the product page for such-and-such widgets) or specific
questions to answer (who is the vice president of the company?),
rather that just being asked to choose which of two presentations is
accessible. We can gather data google-analytics style that traces the
paths users take and the efficiency of each test.

4. The entire test suite should be open to (I imagine moderated)
submissions from the public.

I think the next step is to find a wiki or some other central spot to
collect and process ideas and get a preliminary spec started. Any
suggestions on that front?

Regards,

Chris

From: adam solomon
Date: Sat, Aug 28 2010 11:21PM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | Next message →

I can add my two cents as well - issues that come up in our education ministry's websites which have to be accessible to WCAG 2 (though I would have to translate everything). Things like auto-complete widgets, calendars, and the like. Why do we need a dedicated server?

> From: = EMAIL ADDRESS REMOVED =
> To: = EMAIL ADDRESS REMOVED =
> Date: Fri, 27 Aug 2010 13:53:53 -0400
> Subject: Re: [WebAIM] Proposal for an online, crowdsourced accessibility testing platform
>
> I'm very excited about this proposal.
>
> I'd like to contribute examples for testing that I run into with federal
> government documents, especially those that are more technical in nature.
> WCAG 2 and ITC 2 (508 for the US fed govt) don't address many of the
> situations my govt clients are in.
>
> I can contribute services:
> -- technical samples (including Word documents, Excel spreadsheets, PDFs
> made from InDesign and office software)
> -- writing and online training demonstrating the technique/best practice.
>
> But we'll need someone to manage this project and coordinate all the
> players, tasks, and final recommendations.
> Any suggestions?
>
> --Bevi
>
> : : : : : : : : : : : : : : : : : : :
> : : : : : : : : : : : : :
> Bevi Chagnon | PubCom | = EMAIL ADDRESS REMOVED = | 301-585-8805
> Government publishing specialists, trainers, consultants | print, press,
> web, Acrobat PDF & 508
> Online at the blog: It's 2010. Where's your career heading?
> www.pubcom.com/newsletter
>
>
>

From: ckrugman
Date: Tue, Aug 31 2010 7:12AM
Subject: Re: Proposal for an online, crowdsourced accessibility testing platform
← Previous message | No next message

As a screen reader user this sounds like a good idea. Don't have the skills
to collaborate on something like this as this outside of my areas of
expertise.
Chuck
----- Original Message -----
From: "Chris Hoffman" < = EMAIL ADDRESS REMOVED = >
To: "WebAIM Discussion List" < = EMAIL ADDRESS REMOVED = >
Sent: Tuesday, August 24, 2010 7:44 PM
Subject: [WebAIM] Proposal for an online,crowdsourced accessibility testing
platform


> Hello everyone,
>
> I've been reading this list for several years now, and it's been hard
> not to notice that many, if not most, of the questions posed are of
> the form, "Is writing HTML like such-and-such accessible?" or "What is
> the most accessible way to put X on my web page?"
>
> It has also become apparent, time and again, that although there are
> many ways to increase the accessibility of content online, such as
> adhering to WCAG guidelines or using online accessibility checkers
> like WAVE or Cynthia Says, the only way to _really_ ensure that
> content is accessible is to test it with actual people with actual
> disabilities who are using actual assistive technology. That is, a web
> page that passes all of the automated accessibility checks and adheres
> to accessibility standards is not accessible unless it can be used by
> real people.
>
> To that end, I have had the following idea rolling around in my head
> for quite a while: Why not create a site that presents pairs of
> alternative HTML snippets and asks users out in the universe whether
> each one is more of less accessible? Visitors could (anonymously)
> record their choices ("A is slightly more accessible than B"), as well
> as any specific notes and the assistive technology they were using,
> and the resulting data could be made available to web designers and
> developers.
>
> My first question is, does anything like this (namely, open A/B
> testing for accessibility) already exist?
> The second is, regardless of the answer to the first question, whether
> there is anyone on this list who would be interested in collaborating
> on such a project?
>
> I'm taking it as a given that there are lots of questions that would
> need to be answered to make this actually work, but for now am taking
> the initial step of getting it out of my head and into the open.
>
> Thanks & regards,
>
> Chris
>