In the earlier years of web accessibility (pre-1999 to 2004), screen readers were buggy and temperamental. They would sometimes respond erratically to even simple content. Several of the out-dated recommendations of WCAG 1.0 were put in place to address these screen reader-specific issues (see almost any checkpoint that starts with “Until user agents…”), but problems remained and screen reader testing was often a necessity, even for relatively basic pages.
Since then, screen readers have improved. They are far from perfect, but they have fewer bugs and tend to reliably read correctly-structured content. If you build a simple web page following solid accessibility guidelines, you can be relatively confident that your page would be accessible to a screen reader without actually testing it with a screen reader. I think this was one of the main reasons that, between around 2004 and 2007, I relied less on screen reader testing as part of my accessibility evaluations. It was still important in some cases, but many pages could be evaluated using a good evaluation tool like WAVE (provided by WebAIM) and by performing a few other manual checks, such as testing keyboard accessibility.
The web is much more dynamic and interactive than it was just a few years ago. Some of these changes have promising implications for accessibility, but they also introduce new accessibility challenges. Highly dynamic content, which is becoming more common, needs to be tested in a screen reader. This means screen reader testing as part of a larger accessibility evaluation is becoming more common as well. Today, it is a key part of almost all my accessibility evaluation work. Not only is my screen reader testing more frequent, it is more focused. In addition to listening to a page being read from start to finish, I usually concentrate my testing on certain areas, such as forms and navigation.
I have heard, and likely said, that testing with a screen reader very difficult. That it takes weeks or months to learn even the basics. I don’t agree with this statement anymore. Basic screen reader testing really only requires familiarity with a few simple keyboard commands and an understanding of what to listen for. Resources are available (e.g., our articles on testing with JAWS and testing with NVDA) to help users learn the basic shortcuts. In my opinion, the knowledge required to implement accessibility is much more difficult to acquire than knowledge of how to actually use the screen reader. If anyone feels like screen reader testing is an unnecessary or overly difficult part of accessibility testing, I hope you give it another chance.