Testing with Screen Readers
Questions and Answers

What are the benefits of testing web content with screen readers?

Listening to your web content rather than looking at it can be an "eye-opening" experience (pardon the pun) that takes sighted users out of their normal comfort zone. It gives sighted users a chance to evaluate their content from an entirely different perspective: from the perspective of a blind person. A lot of times you'll end up finding mistakes that would have been hard to catch visually. For example, spelling mistakes become very obvious when you hear words mispronounced by the screen reader. Screen readers are also very good for checking the accuracy and quality of image alternative text. Screen readers can also help you identify problems with reading order, table markup, form elements, and many other aspects of accessibility.

Should I always test my web content for accessibility using a screen reader?

Perhaps. If you know how to use a screen reader, this kind of test can be extremely valuable, especially for more complex or dynamic content. If you don't know how to use a screen reader, testing with a screen reader can be frustrating and counterproductive. In fact, you could mistakenly think that nearly everything you've created is inaccessible, when the real problem may be that you just don't know how to use a screen reader properly. WebAIM provides articles on Using JAWS to Evaluate Web Accessibility, Using NVDA to Evaluate Web Accessibility, and Using VoiceOver to Evaluate Web Accessibility which teach basic usage of these popular screen readers.

So if I don't know how to use a screen reader, I shouldn't even try?

Well, that would be an easy way out, but before you start making excuses, let's take a look at what you'd be missing out on. Screen reader users are one of the primary beneficiaries of your accessibility efforts, so it makes sense to understand their needs. Of course, you don't want to fall into the trap of thinking that accessibility is only relevant to screen reader users. Too many people focus on blindness to the exclusion of people with other disability types (motor, auditory, cognitive, low vision, etc.) whose needs are just as relevant.

Although a screen reader isn't a "browser" in the same way that Chrome, Firefox, Safari, and Internet Explorer are browsers (in fact, in most cases the screen reader depends on those browsers), screen readers are a way of accessing web content that is different from the way that sighted people use browsers. If you don't understand these differences, you won't understand what the accessibility challenges are for screen reader users, and you won't be able to design effectively for this audience.

What are the main differences between the way sighted users and screen reader users access web content?

Blind users access web content in fundamentally different ways compared to sighted users. To begin with, screen readers exclusively or primarily use the keyboard for navigation. Many keyboard shortcuts are available, most of which the user has to memorize. Sighted users are accustomed to using a mouse for navigation. They're also accustomed to visually scanning a page in all directions almost simultaneously. Both of these are habits that a sighted person must set aside while testing with a screen reader.

But habits and techniques of navigating web content are only part of the story. In a very real sense, screen readers make you think differently. A sighted person tends to think of web sites in terms of blocks of information organized visually. Most web pages have navigation features either at the top or side of the viewable area. They often have graphics designed to attract your attention to "important" elements like new content, specially-priced merchandise, or other things. Good designs draw your visual attention to the content and use visual cues to help you understand the site's organization almost instantaneously.

Screen reader users cannot survey the entirety of a web page with such immediacy. Web content is linear and text-based. They don't often think in terms of left and right or position on the page. It is irrelevant to them whether the most important content is visually in the middle with the boldest colors and most artistic design. Positioning and visual artistry do not inherently help or hinder the accessibility of web content for screen reader users. This type of information is simply useless to them.

Then how do screen reader users experience web pages?

There are two primary methods for exploring web content in a screen reader - reading the content of the page or navigating the elements of the page.

When reading the content of a page, the screen reader users hear the title of the web page (assuming there is one), then each text element in the order that it appears in the source code of the document. Now, they don't hear the source code directly, but all text content within the page markup and any relevant structure and semantics. The order of these is important.

For a crude approximation of this linearized effect, you could copy the entire web page (not the source code, but the regular web page in the browser), then paste everything into a text editor that does not support text formatting, styling, or images. You can also use WAVE to view the underlying, un-styled content of the document.

Screen reader users have multiple ways of navigating within a page. Screen reader users can navigate from heading to heading, get a list of links organized alphabetically, use the tab key to navigate to links and form controls, and search within the page for keywords, among other methods. Screen readers allow users to navigate through data tables, going from cell to cell and (assuming the table has been marked up correctly) informing the user what the row or column headers are for each cell. So even though the content is linear, screen reader users have many options to navigate through that content, and everyone has their own favorite methods.

It is rare that a screen reader user would listen to an entire web page from start to finish without skipping some portion, such as the navigation links at the top of the page, the copyright information at the bottom, or other parts in-between. Users are more likely to listen to an entire page if they are unfamiliar with a site and if the content is very important to them, but most of the time they attempt to find what they are looking for as quickly as possible. Headings, landmarks, searches within pages, and lists of links (or tabbing from link to link) are among the most common methods used to find content quickly.

Some screen reader users, especially those that are deaf-blind, may receive the content of a page via a refreshable braille display.

So how do I know which of all these methods screen reader users are going to use on my web content?

You don't. You can probably assume that someone will use all of those methods at some point or another. You can't control how users will interact with your content. The only thing you can do is make sure that your design doesn't make it hard to use the various kinds of interactions available.

What kinds of things would make it hard?

You can't navigate through the headings of a web page if the page doesn't have headings. You can't hear the meaning of a graphic unless it has alternative text. Non-descriptive link text such as "click here" and "more" offer little or no clues as to what will happen when the user selects them. These are some of the barriers. Following the principles of the Web Content Accessibility Guidelines (WCAG) will definitely help remove barriers, though they are not fool-proof. As a matter of fact, no method is fool-proof. You'll have the most success by paying attention to principles and guidelines and testing your content.

How can I learn all the screen reader keyboard shortcuts?

WebAIM has compiled a list of keyboard shortcuts for JAWS and keyboard shortcuts for NVDA. Other screen readers typically provide keyboard shortcut directions in their help files. Learning just a handful of keyboard shortcuts will allow you to conduct screen reader testing.

Wait a minute! How many screen reader programs do I need to learn?

Realistically, no one should be expected to learn all of these programs to expert proficiency. In fact, if you become an expert screen reader user, it's likely that you may overlook potential barriers to your users. Many screen reader users use multiple screen readers for different situations or types of content. The problem, of course, is that several brands of screen readers do exist, and there are slight differences between them in terms of how they read and access content.

How big are these "slight differences," and do designers have to take these differences into account when they design content?

Many of the differences are more superficial, and don't matter much from a content author's perspective. For example, JAWS says "link" before each link. Most of the differences between screen readers fall into the "interesting but not important" category. On the other hand, there are sometimes differences between screen readers in terms of the technologies they support, or the bugs they have, or some other substantial area of difference. For example, when Adobe first included accessibility features in Adobe Reader, only a couple of screen readers supported these features. Now this capability is more widespread among screen readers.

Do you really expect developers to keep track of all of the differences like that?

As new web technologies come out, it is important to keep track of the large developments in the field. One way to keep up-to-date is to periodically visit sites like WebAIM or subscribe to email discussion lists on the subject. Good developers keep up with new browser versions, new web technologies, and web standards. Screen readers should be added to the list. Developers should at least be aware of the different brands of screen readers and the technologies each supports.

On the other hand, overall it is better for developers to pay more attention to the principles, guidelines, and best practices of accessibility, rather than to the differences between screen readers. We wouldn't want developers to design specifically for one brand of screen reader, for example, if doing so would make the content unfriendly to users of other brands of screen readers.

So should I test my content with all the screen readers?

You certainly could, and you may learn quite a bit if you do. This would be especially true of JavaScript or PDF content. It would be wise to test it in as many technologies as possible, including a wide range of screen readers. For less complex content, though, testing in one or two screen readers is usually enough. If you follow guidelines, you can be reasonably assured that your content will be screen reader friendly.

Which "one or two?"

JAWS is currently the most popular screen reader, though VoiceOver, NVDA, and others are becoming much more popular due to their features and low cost. Test your content in at least one of these. Most screen readers are about equal in terms of capabilities with web content, so you can't go wrong with any of them. It's just a matter of which ones you feel most comfortable using.

Wouldn't it just be easier to let a blind user test my web content, so I don't have to learn all this stuff?

Blind users are part of your real audience, so there's a lot you can learn from them. Many of them would be happy to share their expertise and opinions with you. Large organizations, especially, should consider hiring blind experts to do just that.

There are, however, some caveats that you need to take into consideration.

First of all, not all blind people are proficient screen reader users. Inexperienced users can offer a valuable perspective (the perspective of novice), but they may not be able to give you the broader perspective of screen reader users in general. In some cases, inexperienced blind users may give you bad advice as a result of their lack of expertise. On the other hand, expert screen reader users may not be able to give you the full perspective of less experienced users. In other words, as with any user test, you have to make sure that your users accurately reflect the target audience. You don't want to fall into the trap of saying "Fred, our blind screen reader user, said we should do it this way." Fred may or may not be representative of the majority of screen reader users. He may also have customized settings that render content differently than the vast majority of other screen reader users. Ideally you would have a user group consisting of several individuals of varying levels of expertise. It may not be feasible or practical to always have such a group available, but it wouldn't be a bad idea to occasionally invest in user group studies, especially before launching new versions of complex web sites.

Second, blind users may or may not be able to recognize when content is inaccessible to them. For example, if the main content of a web page consists of complex, inaccessible JavaScript, the user may not even know that the main content is there at all. If the screen reader doesn't read it to them, the content may as well not exist. A sighted user, on the other hand, could look at the content and listen to it. This person could recognize that none of the main content is being read. Of course, having sight can also be a disadvantage in this testing process. Sighted users may rely too much on what they see, and not realize that not everything they see is being read by the screen reader.

The lesson to be learned here is that you should have a variety of people with a variety of abilities test your content. It is best to not rely on just one person's interpretation. One way to involve blind users is to solicit their advice on the WebAIM discussion list and other online forums. Sometimes they'll offer their advice for free. Other times it may be more appropriate to contract with them to provide testing or evaluation services.

Isn't that a lot of work to get that many opinions from that many different kinds of users?

Sometimes, yes, it can be a lot of work. Maybe you won't have the time or resources to do this for every piece of web content. Most people would have a hard time getting a user group of any kind together on a regular basis, and it can sometimes be a challenge to find a users with a variety of types of disabilities.

Still, when implementing big, site-wide changes, or when designing a new interface, or when performing a comprehensive evaluation of the accessibility of existing content, this is still good advice. Even expert designers who have been designing with accessibility in mind for a long time can benefit from user groups like this.

On a practical level, there is more than one way to get this kind of feedback. The expensive, time-consuming option is to carry out formal studies. Such studies can be very helpful and in-depth. But you can't do these studies very often, unless you happen to have a lot of money and time at your disposal. The cheaper method is to periodically have a few people—colleagues, associates, people on discussion forums, etc.—check pages and give you informal feedback.

How expensive are screen readers?

JAWS is usually the most expensive, typically priced at over US $1000. JAWS can also read word processors, spreadsheets, personal finance programs, the operating system, and many other applications. VoiceOver comes with any Mac or iOS operating system. NVDA is free, but does not provide full system access - it is best for accessing web content and basic documents.

So I'm going to have to spend a lot of money, right?

Actually, no. If you have a Mac with OS X or an iOS device, you already have VoiceOver. NVDA is free for use on Windows. Windows Narrator, though currently not used frequently by users with visual disabilities, comes with Windows and can be a viable option for testing.

In the end, is it all worth it?

If you're implying that it's inconvenient to have to learn how to make content accessible to screen reader users, then I'll answer in the form of a question: How important is it to you to be able use the internet? I don't know about you, but the internet has become an integral and important part of my life. For me, the real "inconvenience" would be not being able to take advantage of what the internet has to offer. For screen reader users, this unfortunate inconvenience is all too real.