Testing with Screen Readers
Questions and Answers

What are the benefits of testing web content with screen readers?

Listening to your web content rather than looking at it can be an "eye-opening" experience (pardon the pun) that takes sighted users out of their normal comfort zone. It gives sighted users a chance to evaluate their content from an entirely different perspective: from the perspective of a blind person. A lot of times you'll end up finding mistakes that would have been hard to catch visually. For example, spelling mistakes become very obvious when you hear words mispronounced by the screen reader. Screen readers are also very good for checking the accuracy and quality of image alternative text. Screen readers can also help you identify problems with reading order, table markup, form elements, and many other aspects of accessibility.

Should I always test my web content for accessibility using a screen reader?

Perhaps. If you know how to use a screen reader, this kind of test can be extremely valuable, especially for more complex or dynamic content. If you don't know how to use a screen reader, testing with a screen reader can be frustrating and counterproductive. In fact, you could mistakenly think that nearly everything you've created is inaccessible, when the real problem may be that you just don't know how to use a screen reader properly. WebAIM provides articles on Using JAWS to Evaluate Web Accessibility, Using NVDA to Evaluate Web Accessibility, and Using VoiceOver to Evaluate Web Accessibility which teach basic usage of these popular screen readers.

So if I don't know how to use a screen reader, I shouldn't even try?

Well, that would be an easy way out, but before you start making excuses, let's take a look at what you'd be missing out on. Screen reader users are one of the primary beneficiaries of your accessibility efforts, so it makes sense to understand their needs. Of course, you don't want to fall into the trap of thinking that accessibility is only relevant to screen reader users. Too many people focus on blindness to the exclusion of people with other disability types (motor, auditory, cognitive, low vision, etc.) whose needs are just as relevant.

Although a screen reader isn't a "browser" in the same way that Chrome, Firefox, Safari, and Internet Explorer are browsers (in fact, in most cases the screen reader depends on those browsers), screen readers are a way of accessing web content that is different from the way that sighted people use browsers. If you don't understand these differences, you won't understand what the accessibility challenges are for screen reader users, and you won't be able to design effectively for this audience.

What are the main differences between the way sighted users and screen reader users access web content?

Blind users access web content in fundamentally different ways compared to sighted users. To begin with, screen readers exclusively or primarily use the keyboard for navigation. Many keyboard shortcuts are available, most of which the user has to memorize. Sighted users are accustomed to using a mouse for navigation. They're also accustomed to visually scanning a page in all directions almost simultaneously. Both of these are habits that a sighted person must set aside while testing with a screen reader.

But habits and techniques of navigating web content are only part of the story. In a very real sense, screen readers make you think differently. A sighted person tends to think of web sites in terms of blocks of information organized visually. Most web pages have navigation features either at the top or side of the viewable area. They often have graphics designed to attract your attention to "important" elements like new content, specially-priced merchandise, or other things. Good designs draw your visual attention to the content and use visual cues to help you understand the site's organization almost instantaneously.

Screen reader users cannot survey the entirety of a web page with such immediacy. Web content is linear and text-based. They don't often think in terms of left and right or position on the page. It is irrelevant to them whether the most important content is visually in the middle with the boldest colors and most artistic design. Positioning and visual artistry do not inherently help or hinder the accessibility of web content for screen reader users. This type of information is simply useless to them.

Then how do screen reader users experience web pages?

Screen reader users hear the title of the web page (assuming there is one), then each text element in the order that it appears in the source code of the document. Now, they don't hear the source code directly, but all text content within the page markup. It is this text, and this order of the text that are relevant. For a crude approximation of this linearized effect, you could copy the entire web page (not the source code, but the regular web page in the browser), then paste everything into a text editor that does not support text formatting, styling, or images. You can also use the Web Developer Firefox Toolbar or WAVE to view the underlying content of the document.

This is still a crude approximation because screen reader users actually have more information available to them, and they have multiple ways of navigating this information. For example, modern screen readers let users know where lists begin and end, and even how many items there are in a list. Screen readers allow users to navigate through data tables, going from cell to cell and (assuming the table has been marked up correctly) informing the user what the headings are for each cell. Screen reader users can also navigate from heading to heading, get a list of links organized alphabetically, use the tab key to navigate to links and form controls, and search within the page for keywords, among other methods. So even though the content is linear, screen reader users have many options of different ways to navigate through that content, and everyone has their own favorite methods. You never could say that screen reader users "always" do things "this way" or "that way." There are too many individual differences.

It is rare that a screen reader user would listen to an entire web page from start to finish without skipping some portion, such as the navigation links at the top of the page, the copyright information at the bottom, or other parts in-between. Users are more likely to listen to an entire page if they are unfamiliar with a site and if the content is very important to them, but most of the time they attempt to find what they are looking for as quickly as possible. Headings, landmarks, searches within pages, and lists of links (or tabbing from link to link) are among the most common methods used to find content quickly.

So how do I know which of all these methods screen reader users are going to use on my web content?

You don't. You can probably assume that someone will use all of those methods at some point or another. You can't control how users will interact with your content. The only thing you can do is make sure that your design doesn't make it hard to use the various kinds of interactions available.

What kinds of things would make it hard?

You can't navigate through the headings of a web page if the page doesn't have headings. You can't hear the meaning of a graphic unless it has alternative text. Non-descriptive link text such as "click here" and "more" offer little or no clues as to what will happen when the user selects them. These are some of the barriers. Following the principles of the Web Content Accessibility Guidelines (WCAG) or Section 508 guidelines will definitely help remove barriers, though they are not fool-proof. As a matter of fact, no method is fool-proof. You'll have the most success by paying attention to principles and guidelines and testing your content.

How can I learn all the screen reader keyboard shortcuts?

WebAIM has compiled a list of keyboard shortcuts for JAWS and keyboard shortcuts for NVDA. Other screen readers typically provide keyboard shortcut directions in their help files.

Wait a minute! How many screen reader programs do I need to learn?

Realistically, no one should be expected to learn all of these programs to expert proficiency. Many screen reader users use multiple screen readers for different situations or type of content. The problem, of course, is that all of these brands do exist, and there are slight differences between them in terms of how they read and access content.

How big are these "slight differences," and do designers have to take these differences into account when they design content?

That's a good question. Many of the differences are more superficial, and don't matter much from a content designer's perspective. For example, JAWS says "link" before each link. Most of the differences between screen readers fall into the "interesting but not important" category. On the other hand, there are sometimes differences between screen readers in terms of the technologies they support, or the bugs they have, or some other substantial area of difference. For example, when Adobe first included accessibility features in Adobe Reader, only a couple of screen readers supported these features. Now this capability is more widespread among screen readers.

Do you really expect developers to keep track of all of the differences like that?

In a sense, yes, but in other ways, not necessarily. As new web technologies come out, it is important to keep track of the large developments in the field. One way to keep up-to-date is to periodically visit sites like WebAIM or subscribe to email discussion lists on the subject. Good developers keep up with new browser versions, new web technologies, and web standards. Screen readers should be added to the list. Developers should at least be aware of the different brands of screen readers and the technologies each supports.

On the other hand, overall it is better for developers to pay more attention to the principles and guidelines of accessibility, rather than to the differences between screen readers. We wouldn't want developers to design specifically for one brand of screen reader, for example, if doing so would make the content unfriendly to users of other brands of screen readers.

So should I test my content with all the screen readers?

You certainly could, and you may learn quite a bit if you do. This would be especially true of JavaScript, Flash, or PDF content. It would be wise to test it in as many technologies as possible, including a wide range of screen readers. For less complex content, though, testing in one or two screen readers is usually enough. If you follow guidelines, you can be reasonably assured that your content will be screen reader friendly.

Which "one or two?"

JAWS is currently the most popular screen reader, though VoiceOver, NVDA, and SystemAccess are becoming much more popular due to their features and low cost. Test your content in at least one of these. Most screen readers are about equal in terms of capabilities with web content, so you can't go wrong with any of them. It's just a matter of which ones you feel most comfortable using.

Wouldn't it just be easier to let a blind user test my web content, so I don't have to learn all this stuff?

That's a fabulous idea, in some ways, and a not-so-fabulous idea in other ways. Blind users are part of your real audience, so there's a lot you can learn from them. Many of them would be happy to share their expertise and opinions with you. Large organizations, especially, should consider hiring blind experts to do just that.

There are a few not-so-fabulous sides to this scenario, though—or at least there are some caveats that you need to take into consideration.

First of all, not all blind people are proficient screen reader users. Some are; some aren't. Inexperienced users can offer a valuable perspective (the perspective of novice), but they may not be able to give you the broader perspective of screen reader users in general. In some cases, inexperienced blind users may give you bad advice as a result of their lack of expertise. On the other hand, expert screen reader users may not be able to give you the full perspective of less experienced users. In other words, as with any user test, you have to make sure that your users accurately reflect the target audience. You don't want to fall into the trap of saying "Fred, our blind screen reader user, said we should do it this way." Fred may or may not be representative of the majority of screen reader users. He may also have customized settings that render content differently than the vast majority of other screen reader users. Ideally you would have a user group consisting of several individuals of varying levels of expertise. It may not be feasible or practical to always have such a group on the payroll, but it wouldn't be a bad idea to occasionally invest in user group studies, especially before launching new versions of complex web sites.

Second, blind users may or may not be able to recognize when content is inaccessible to them. For example, if the main content of a web page consists of complex, inaccessible JavaScript, the user may not even know that the main content is there at all. If the screen reader doesn't read it to them, the content may as well not exist. A sighted user, on the other hand, could look at the content and listen to it. This person could recognize that none of the main content is being read. Of course, having sight can also be a disadvantage in this testing process. Sighted users may rely too much on what they see, and not realize that not everything they see is being read by the screen reader.

The lesson to be learned here is that you should have a variety of people with a variety of abilities test your content. It is best to not rely on just one person's interpretation. One way to involve blind users is to solicit their advice on the WebAIM discussion list and other online forums. Sometimes they'll offer their advice for free. Other times it may be more appropriate to contract with them to provide testing or evaluation services.

Isn't that a lot of work to get that many opinions from that many different kinds of users?

Sometimes, yes, it can be a lot of work. Maybe you won't have the time or resources to do this for every piece of web content. Most people would have a hard time getting a user group of any kind together on a regular basis, and it can sometimes be a challenge to find a users with a variety of types of disabilities, I'll admit that.

Still, when implementing big, site-wide changes, or when designing a new interface, or when performing a comprehensive evaluation of the accessibility of existing content, this is still good advice. Even expert designers who have been designing with accessibility in mind for a long time can benefit from user groups like this.

On a practical level, there is more than one way to get this kind of feedback. The expensive, time-consuming option is to carry out formal studies. Such studies can be very helpful and in-depth. But you can't do these studies very often, unless you happen to have a lot of money and time at your disposal. The cheaper method is to periodically have a few people—colleagues, associates, people on discussion forums, etc.—check pages and give you informal feedback.

How expensive are screen readers?

JAWS is usually the most expensive, typically priced at over US $1000. Window Eyes prices are about the same. JAWS and Window Eyes can also read word processors, spreadsheets, personal finance programs, the operating system, and many other applications. VoiceOver comes with the relatively inexpensive Mac operating system. NVDA is free, but does not provide full system access - it generally only reads web content and basic documents.

So I'm going to have to spend a lot of money, right?

Actually, no. If you have a lot of money, go ahead and buy them all. If you have a Mac with OS X, you already have VoiceOver. If you have a little money, use NVDA. If you don't want to spend money, there is an alternative: download timed evaluation copies of JAWS and Window Eyes. These timed versions run for 40 minutes at a time. After 40 minutes, you have to restart your computer in order to use the screen reader again. (Yes, you really do have to restart your computer before the screen reader works again. You can't just restart the screen reader.) These timed evaluation copies can be ideal for web developers to use to test their content. A lot of the time you can do all of your testing within the 40 minute time limit. Sometimes you may need to restart your computer to continue testing, but that's not such a bad deal, especially when you consider that the software was "free.". UPDATE: We have been apprised that the terms of use for the JAWS demonstration version specifically prohibits evaluation and testing.

In the end, is it all worth it?

If you're implying that it's inconvenient to have to learn how to make content accessible to screen reader users, then I'll answer in the form of a question: How important is it to you to be able use the internet? I don't know about you, but the internet has become an integral and important part of my life. For me, the real "inconvenience" would be not being able to take advantage of what the internet has to offer. For screen reader users, this unfortunate inconvenience is all too real.