Testing with Screen Readers:
Questions and Answers

Question:

What are the benefits of testing web content with screen readers?

Answer:

Listening to your web content rather than looking at it can be an "eye-opening" experience (pardon the pun) that takes sighted users out of their normal comfort zone. It gives sighted users a chance to evaluate their content from an entirely different perspective: from the perspective of a blind person. A lot of times you'll end up finding mistakes that would have been hard to catch visually. For example, spelling mistakes become very obvious when you hear words mispronounced by the screen reader. Screen readers are also very good for checking the accuracy and quality of alt text. Recently I was listening to a web page and realized that the alt text was completely wrong. The graphic said "Search" but the alt text said "Options." It took me a moment to figure out why the screen reader wasn't reading the alt text for the "Search" button, but then I realized that it was reading the alt text, even though the alt text was wrong. Screen readers can also help you identify problems with reading order, table markup, form elements, and many other aspects of accessibility.

Question:

Should I always test my web content for accessibility using a screen reader?

Answer:

Perhaps. If you know how to use a screen reader, this kind of test can be extremely valuable. If you don't know how to use a screen reader, testing with a screen reader can be frustrating and counterproductive. In fact, you could mistakenly think that nearly everything you've created is inaccessible, when the real problem may be that you just don't know how to use a screen reader properly.

Question:

So if I don't know how to use a screen reader, I shouldn't even try?

Answer:

Well, that would be an easy way out, but before you start making excuses, let's take a look at what you'd be missing out on. Screen reader users are one of the primary beneficiaries of your accessibility efforts, so it makes sense to understand their needs. Of course, you don't want to fall into the trap of thinking that accessibility is only relevant to screen reader users. Too many people focus on blindness to the exclusion of people with other disability types (motor, auditory, cognitive, low vision, etc.) whose needs are just as relevant. But since you asked about screen reader testing, let's address that particular need specifically.

Although a screen reader isn't a "browser" in the same way that Firefox, Opera, Safari, Netscape, and Internet Explorer are browsers (in fact, in most cases the screen reader depends on those browsers), screen readers are a way of accessing web content that is different from the way that sighted people use browsers. If you don't understand these differences, you won't understand what the accessibility challenges are for screen reader users, and you won't be able to design effectively for this audience.

Question:

What are the main differences between the way sighted users and screen reader users access web content?

Answer:

Blind users access web content in fundamentally different ways compared to sighted users. To begin with, screen readers require the use of keyboard shortcuts, most of which the user has to memorize. Sighted users are accustomed to using a mouse for navigation. They're also accustomed to visually scanning a page in all directions almost simultaneously. Both of these are habits that a sighted person must set aside while testing with a screen reader.

But habits and techniques of navigating web content are only part of the story. In a very real sense, screen readers make you think differently. A sighted person tends to think of web sites in terms of blocks of information organized visually. Most web pages have navigation features either at the top or side of the viewable area. They often have graphics designed to attract your attention to "important" elements like new content, specially-priced merchandise, or other things. Good designs draw your visual attention to the content and use visual cues to help you understand the site's organization in a matter of 1 or 2 seconds.

Screen reader users cannot survey the entirety of a web page with such immediacy. Web content is linear and text-based. They don't think in terms of left and right or position on the page. It is irrelevant to them whether the most important content is visually in the middle with the boldest colors and most artistic design. Positioning and visual artistry do not inherently help or hinder the accessibility of web content for screen reader users. This type of information is simply useless to them.

Question:

Then how do screen reader users experience web pages?

Answer:

Screen reader users hear the title of the web page (assuming there is one), then each text element in the order that it appears in the source code of the document. Now, I don't mean that they hear the source code, but if you could look at the source code and strip it of all of its "tags" and markup, you'd be left with just the text. It is this text, and this order of the text that are relevant. For a crude approximation of this linearized effect, you could copy the entire web page (not the source code, but the regular web page in the browser) by hitting CTRL + A (in Windows) or Apple + A (in Mac OS), then paste everything into a text editor like Notepad or SimpleText.

I said this is a crude approximation because screen reader users actually have more information available to them than this, and they have multiple ways of navigating this information. For example, rather modern screen readers let users know where lists begin and end, and even how many items there are in a list. Screen readers allow users to navigate through data tables, going from cell to cell and (assuming the table has been marked up correctly) informing the user what the headings are for each cell. Screen reader users can also navigate from heading to heading, get a list of links organized alphabetically, use the tab key to navigate from link to link, and search within the page for keywords, among other methods. So even though the content is linear, screen reader users have many options of different ways to navigate through that content, and everyone has their own favorite methods. You never could say that screen reader users "always" do things "this way" or "that way." There are too many individual differences.

It is rare that a screen reader user would listen to an entire web page from start to finish without skipping some portion, such as the navigation links at the top of the page, the copyright information at the bottom, or other parts in-between. Users are more likely to listen to an entire page if they are unfamiliar with a site and if the content is very important to them, but most of the time they attempt to find what they are looking for as quickly as possible. Searches within pages and lists of links (or tabbing from link to link) are among the most common methods used to find content quickly.

Question:

So how do I know which of all these methods screen reader users are going to use on my web content?

Answer:

You don't. You can probably assume that someone will use all of those methods at some point or another. You can't control how users will interact with your content. The only thing you can do is make sure that your design doesn't make it hard to use the various kinds of interactions available.

Question:

What kinds of things would make it hard?

Answer:

I'll give a few examples. You can't navigate through the headings of a web page if the page doesn't have headings. You can't hear the meaning of a graphic unless it has alternative text. Non-descriptive link text like "click here" and "more" offer little or no clues as to what will happen when the user selects them. These are some of the barriers. Following the principles of the Web Content Accessibility Guidelines - external link (WCAG) or Section 508 guidelines - external link will definitely help remove barriers, though they are not fool-proof. As a matter of fact, no method is fool-proof. You'll have the most success by paying attention to principles and guidelines and testing your content.

Question:

How can I learn all the screen reader keyboard shortcuts?

Answer:

WebAIM has compiled a list of keyboard shortcuts for JAWS (by Freedom Scientific - external link). The online Window Eyes tutorial - external link by GW Micro - external link contains information about keyboard shortcuts under "Appendix A.1: Hot Keys - Quick Reference Guide." Keyboard shortcuts for HAL (by Dolphin - external link) are available from the help menu after installing the program.

Question:

Wait a minute! How many screen reader programs do I need to learn?

Answer:

Realistically, no one should be expected to learn all of these programs to expert proficiency. Most screen reader users only own one brand. The problem, of course, is that all of these brands do exist, and there are slight differences between them in terms of how they render content.

Question:

How big are these "slight differences," and do designers have to take these differences into account when they design content?

Answer:

That's a good question. Many of the differences are more superficial, and don't matter much from a content designer's perspective. For example, JAWS says "link" before each link. Most of the differences between screen readers fall into the "interesting but not important" category. On the other hand, there are sometimes differences between screen readers in terms of the technologies they support, or the bugs they have, or some other substantial area of difference. For example, when Adobe first included accessibility features in Adobe Reader, only a couple of screen readers supported these features. Now this capability is more widespread among screen readers.

Question:

Do you really expect developers to keep track of all of the differences like that?

Answer:

In a sense, yes, but in other ways, not necessarily. As new web technologies come out, it is important to keep track of the large developments in the field. One way to keep up-to-date is to periodically visit sites like WebAIM or subscribe to email discussion lists on the subject. Good Developers keep up with new browser versions, new web technologies, and web standards. Screen readers should be added to the list. Developers should at least be aware of the different brands of screen readers and the technologies each supports.

On the other hand, overall it is better for developers to pay more attention to the principles and guidelines of accessibility, rather than to the differences between screen readers. We wouldn't want developers to design specifically for one brand of screen reader, for example, if doing so would make the content unfriendly to users of other brands of screen readers.

Question:

So should I test my content with all the screen readers?

Answer:

You certainly could, and you may learn quite a bit if you do. This would be especially true of complex content with JavaScript, Flash, or PDF files. It would be wise to test it in as many technologies as possible, including a wide range of screen readers. For less complex content, though, testing in one or two screen readers is usually enough.

Question:

Which "one or two?"

Answer:

The two main screen readers in the United States are JAWS and Window Eyes. Test your content in at least one of these. Most screen readers are about equal in terms of capabilities with web content, so you can't go wrong with any of them. It's just a matter of which ones you feel most comfortable using. HAL is less common, but it could be worth looking at too.

Question:

Wouldn't it just be easier to let a blind user test my web content, so I don't have to learn all this stuff?

Answer:

That's a fabulous idea, in some ways, and a not-so-fabulous idea in other ways. Blind users are your real audience, so there's a lot you can learn from them. Many of them would be happy to share their expertise and opinions with you. Large organizations, especially, should consider hiring blind experts to do just that. Some users are available as consultants on projects. Whenever you can have them test your content directly, by all means, do that.

There are a few not-so-fabulous sides to this scenario, though—or at least there are some caveats that you need to take into consideration.

First of all, not all blind people are proficient screen reader users. Some are; some aren't. Inexperienced users can offer a valuable perspective (the perspective of novice), but they may not be able to give you the broader perspective of screen reader users in general. In some cases, inexperienced blind users may give you bad advice as a result of their lack of expertise. On the other hand, expert screen reader users may not be able to give you the full perspective of less experienced users. In other words, as with any user test, you have to make sure that your users accurately reflect the target audience. You don't want to fall into the trap of saying "Fred, our blind screen reader user, said we should do it this way." Fred may or may not be representative of the majority of screen reader users. Ideally you would have a user group consisting of several individuals of varying levels of expertise. It may not be feasible or practical to always have such a group on the payroll, but it wouldn't be a bad idea to occasionally invest in user group studies, especially before launching new versions of complex web sites.

Second, blind users may or may not be able to recognize when content is inaccessible to them. For example, if the main content of a web page consists of complex, inaccessible JavaScript, the user may not even know that the main content is there at all. If the screen reader doesn't read it to them, the content may as well not exist. A sighted user, on the other hand, could look at the content and listen to it. This person could recognize that none of the main content is being read. Of course, having sight can also be a disadvantage in this testing process. Sighted users may rely too much on what they see, and not realize that not everything they see is being read by the screen reader.

The lesson to be learned here is that you should have a variety of people with a variety of abilities test your content. It is best to not rely on just one person's interpretation. One way to involve blind users is to solicit their advice on the WebAIM discussion list and other online forums. Sometimes they'll offer their advice for free. Other times it may be more appropriate to contract with them to provide testing or evaluation services.

Question:

Isn't that a lot of work to get that many opinions from that many different kinds of users?

Answer:

Sometimes, yes, it can be a lot of work. Maybe you won't have the time or resources to do this for every piece of web content. Most people would have a hard time getting a user group of any kind together on a regular basis, and it can sometimes be a challenge to find a users with a variety of types of disabilities, I'll admit that.

Still, when implementing big, site-wide changes, or when designing a new interface, or when performing a comprehensive evaluation of the accessibility of existing content, this is still good advice. Even expert designers who have been designing with accessibility in mind for a long time can benefit from user groups like this. I'm certainly no exception.

On a practical level, there is more than one way to get this kind of feedback. The expensive, time-consuming option is to carry out formal studies. Such studies can be very helpful and in-depth. But you can't do these studies very often, unless you happen to have a lot of money and time at your disposal. The cheaper method is to periodically have a few people—colleagues, associates, people on discussion forums, etc.—check pages and give you informal feedback.

Question:

How expensive are screen readers?

Answer:

Expensive. JAWS is usually the most expensive, priced between about US $895 and US $1495, depending on the options and discounts applied. Window Eyes prices are about the same, though just slightly less expensive. JAWS and Window Eyes can also read word processors, spreadsheets, personal finance programs, the operating system, and many other applications.

Question:

So I'm going to have to spend a lot of money, right?

Answer:

Actually, no. If you have a lot of money, go ahead and buy them all. If you have a little money, buy Home Page Reader. If you don't want to spend money, there is an alternative: download timed evaluation copies of JAWS and Window Eyes. These timed versions run for 40 minutes at a time. After 40 minutes, you have to restart your computer in order to use the screen reader again. (Yes, you really do have to restart your computer before the screen reader works again. You can't just restart the screen reader.) These timed evaluation copies can be ideal for web developers to use to test their content. A lot of the time you can do all of your testing within the 40 minute time limit. Sometimes you may need to restart your computer to continue testing, but that's not such a bad deal, especially when you consider that the software was "free."

Question:

In the end, is it all worth it?

Answer:

If you're implying that it's inconvenient to have to learn how to make content accessible to screen reader users, then I'll answer in the form of a question: How important is it to you to be able use the internet? I don't know about you, but the internet has become an integral and important part of my life. For me, the real "inconvenience" would be not being able to take advantage of what the internet has to offer. For screen reader users, this unfortunate inconvenience is all too real.