Thread Subject: Re: Cognitive - compatibility

Note

This archival content is maintained by WebAIM and NCDAE on behalf of TEITAC and the U.S. Access Board . Additional details on the updates to section 508 and section 255 can be found at the Access Board web site.

From: Peter Korn
Date: Wed, Mar 07 2007 7:10 AM

Hi Jonathan,

Hmmm... I think we are close to saying the same thing here. I come at
this from the view of technology implementation. If we decide that it
is desirable to support the use case of a user (potentially with a
cognitive disability) getting a definition of every non-graphic piece of
text on the screen, then we must either:

a. build a dictionary into every application that renders text, with
some guidelines for how to invoke the dictionary lookup for every piece
of text it renders (dialog boxes and menu items as well as content), or
b. define one or more "rendered text retrieval" APIs which all
applications support for every piece of text they render (which is then
used by a 3rd party dictionary/definition lookup program which defines
its own mechanism for invoking the dictionary lookup), or
c. hope that one or more independent software developers sees enough of
a market in developing such software to duplicate the
hacks/reverse-engineering/off-screen-model work of our existing Windows
screen readers in order to create such software, or
d. some combination of the above

Please don't base your assumptions of what is and isn't possible on what
MSAA does and does not provide. We have done a pretty remarkable job of
providing exactly this support in the UNIX/GNOME desktop. With the
collection of applications that Sun ships with Solaris 10 and later, and
that Ubuntu ships with Ubuntu 6.10 and later (to name two OSes I'm
fairly familiar with), exactly this information is provided via a
supported API, and compliance with/implementation of this API is quite
good. No off-screen model would be needed to implement such a feature,
greatly lowering the cost to a 3rd party (e.g. AT) developer to
developing such software.

I say this with confidence because the flat-review feature of our Orca
screen reader is able to review word by word (and letter by letter)
every piece of non-graphic text in the applications we ship in Solaris.
For example, starting with the top-left piece of text in the StarOffice
Writer application, Orca will speak and highlight every word of text,
from "File" in the menu bar, through all of the text content, and ending
with the static text in the statusbar at the bottom of the page. In
fact, it will also speak and highlight the icons in the toolbars (as
well as the words of the font style and font name in the combo-boxes in
the toolbars). No off-screen model is ever used to do this. No
hacks/reverse-engineering is ever used to do this.


Regards,

Peter Korn
Accessibility Architect,
Sun Microsystems, Inc.

> <quote>
> To do this today, you would need to either have an off-screen model (and
> all of the video hooks and reverse engineering implied by such) like
> those in our screen readers, or be running on a system which provides
> </quote>
>
> I also have concerns about how this would work for non-content materials.
> For example, are you asking for the ability to select static text in a
> dialog box with the keyboard in order to get a definition of the word(s).
> Providing keyboard access to such an item is likely to be confusing,
> currently difficult to implement, and may cause issues for keyboard only
> users. Providing a mouse based hit test when the mouse is placed over the
> word such as accessibleObjectFromPoint would work in cases were MSAA or
> another API was available. However, even in new Windows form libraries
> static text isn't always written as separate objects that can be accessed by
> point. Often static text is not it's own object. If the text is exposed
> through OS settings then technology such as this should be 3rd party
> software like a screen reader that would build an off-screen model of the
> text and then provide this type of access.
>
> Jonathan
>
> -----Original Message-----
> From: = EMAIL ADDRESS REMOVED =
> [mailto: = EMAIL ADDRESS REMOVED = ] On Behalf Of Peter Korn
> Sent: Tuesday, March 06, 2007 7:51 PM
> To: TEITAC Web/Software Subcommittee
> Subject: Re: [teitac-websoftware] Cognitive - compatibility
>
> Hi Allen,
>
>
>> Is there AT now that can allow the user to highlight a word in anything
>> on screen and offer spelling and/or dictionary function?
>> Ditto for reading out loud--but I'm sure this part is available.
>>
>>
>
> Software like this used to exist for the Macintosh, a long time ago. It
> was really quite slick.
>
> To do this today, you would need to either have an off-screen model (and
> all of the video hooks and reverse engineering implied by such) like
> those in our screen readers, or be running on a system which provides
> all of the text that is on the screen via a standard API. We're
> starting to get there on the Macintosh. We're already there in GNOME
> UNIX systems. But in neither place do we have such a utility.
>
> Hmmm.... That'd be a good Google Summer of Code project for a summer
> intern...
>
>
>> Second, what are the underlying obstacles to such software beyond such
>> things as scanned images of text?
>>
>>
>
> Having a well-respected API for getting all of the text & text
> boundaries from the apps running on the system.
>
>
>> In other words if a software application meets the other existing
>> software requirements now, will AT of this type generally work? If so,
>> is this requirement redundant,, or do we want to require this be a
>> built-in feature in preference to add-on?
>>
>>
>
> I personally think this is the kind of feature that works best as a
> separate utility - one user interface that works the same everywhere on
> the desktop.
>
>
> Regards,
>
> Peter Korn
> Accessibility Architect,
> Sun Microsystems, Inc.
>

WebAIM is an initiative of:
Center for Persons with Disabilities (CPD) Utah State University