Internet Delivery of Real-time Captioning

Internet Delivery of Real-time Captioning

Aaron Andersen and Jonathan Whiting

WebAIM: Web Accessibility in Mind

http://webaim.org/presentations/2007/csun/captioning.html
or view this presentation JavaScript and CSS removed
Credits

WebAIM is a project of

WebAIM's captioning system sponsored by

Funded by the U.S. Department of Education, Project #P116Z050043
Popularity of Web-based Multimedia
- According to statistics generated by Alexa Internet, the following are, in order, the five most visited websites in the world (as of March 2007).
  1. Yahoo! - http://www.yahoo.com
  2. Microsoft Network - http://www.msn.com
  3. Google - http://www.google.com
  4. YouTube - http://www.youtube.com
  5. Myspace - http://www.myspace.com
Source: http://www.alexa.com/site/ds/top_sites?ts_mode=global&lang=none, Accessed 5 Mar 2007.
Popularity of Web-based Multimedia
- Of those websites, all five offer web-based video services!
  1. Yahoo Video! - http://video.yahoo.com/
  2. MSN Video - http://video.msn.com/
  3. Google Video - http://video.google.com
  4. YouTube - http://www.youtube.com
  5. Myspace Videos - http://videos.myspace.com
Popularity of Web-based Multimedia
- Of those five, only one currently has closed captioning functionality.
  1. Google Video - http://video.google.com
The only other major video site with advertised captioning functionality is AOL Video. However, since neither Google nor AOL offers a way to filter a search or view for only captioned videos, and since the number of captioned videos on both sites represent only a fraction of a percent of the total videos available, actually finding captioned videos on either site is nearly impossible.
What's the Problem?
- With web-based video (and audio) becoming so common, why is captioning still so rare?
  - Not (necessarily) legally required
    - Telcom Section 255: Mandates closed captions for television broadcasts. Does not cover web-based media.
    - Rehab Section 508: Mandates synchronized captions (including web-based media). Only applies to the Federal Government.
    - Rehab Section 504 and ADA would treat web captions as part of a reasonable accommodation, not required a priori.
  - Expensive
    - Hardware closed caption encoder: $1,000 - $9,000
    - Professional stenographer: $75-$150/hr
    - Complete live captioning service: $120 - $300/hr
  - Difficult
    - Majority of existing technology designed for television. Web-based media is not TV.
    - Multiple formats, systems, and languages.
Captioning Software
- MAGpie
  - Developed by the National Center for Accessible Media at WGBH. Free
  - Aids in captioning archived media into a number of formats. No support for live captions.
- Hi-Caption Studio
  - From HiSoftware Inc. $495.00
  - Another suite for captioning archived media, also without support for live captions.
Caption Formats
- All major media players include captioning capabilities.
  - Windows Media Player: SAMI
  - Apple Quicktime: QTtext
  - RealPlayer: RealText
- Almost all internet video now uses Adobe's Falsh technology.
  - Flash does not (yet) have a native captioning system.
  - Many people have written their own caption-enabled video players in flash.
Difficulties Specific to Live Media
- Live media requires live captains. After-the-fact or delayed captions are not sufficient.
  - Caption text must be obtained and processed in real-time.
  - Caption text must be distributed and made available to viewers along with the original media.
Obtaining Live Captions
- Manual Typing
  - Cheap, easy.
  - Impossible to keep up with most verbal communication; meaning for meaning not necessarily sufficient.
- Professional stenographer
  - Can be expensive.
  - May have to work remotely.
  - Extremely fast, very high quality (but not perfect) text.
  - Resulting data requires a hardware or software decoder.
- Voice recognition systems
  - Can be very accurate, but only if the speaker has trained the system and the environment has little background noise.
  - Doesn't work with unknown or multiple speakers.
The various places I looked online for information about speaking and typing speed differ and conflict each other in the values they give, but the general consensus is that a normal conversation is usually at a rate of 150-250 words per minute. Keyboard typing speeds for most people are probably 40-60 words per minute, with an advanced skilled typist often able to reach 100-120 wpm (very fast, but still short of being able to keep up with the spoken word). A trained user of a stenotype machine, one the other hand, can reach speeds of 200 to 300 words per minute with very high accuracy.
Solutions?
- Several groups working on software or hardware tools to help.
- Isolated systems exist, but no single fix for all issues and steps involved.
WebAIM's CaptionCaster
- Unable to locate an existing system that met our needs, we created our own captioning software for internet broadcasts sponsored or produced by us.
- CaptionCaster suite consists of four software components:
  1. CaptionCaster - Obtains, decodes, and broadcasts streaming captions to web clients.
  2. CaptionClient - Flash client for receiving captions from CaptionCaster.
  3. CaptionConnect - Synchronizing, correcting, and archiving captions.
  4. CaptionConvert - Online tool for conversion between caption file formats.
CaptionCaster
- Multiple input modes.
  - Local file
  - Network or modem connection
  - Second CaptionCaster unit
- Decodes common stenographer formats and codes.
- Broadcasts live captions to internet clients.
  - CaptionClient
  - Instant messenger networks
  - Time-encoded log
CaptionClient
- Flash technology installed on 95% of internet-enabled computers.
- Customizable display for optimal viewing environment.
CaptionConnect
- Reads and writes CaptionCaster XML format and several commonly used caption formats.
  - RealText (RealPlayer)
  - QTtext (QuickTime)
  - SAMI (Windows Media Player)
  - SubViewer (Google )
- Caption text can be edited and corrected.
- Individual captions can be added, merged, or removed.
- Can shift entire file or a single caption by specified time interval.
CaptionConvert
- Web-based interface to format conversion functionality of CaptionConnect
- Retains basic formatting information between formats.
- Can be configured to automatically convert an entire batch of caption files from one format to another.
Putting it all together
CaptionCaster Demo
CaptionCaster as a Solution
- Benefits of using CaptionCaster
  - Low-cost.
  - Simple and easy to use.
  - An integrated solution for all steps in the process.
Future Plans for CaptionCaster
- WebAIM been using it internally for some time, and it all works for our needs.
- Isn't ready to be distributed publically.
- Still Needed
  1. Development of features not used by WebAIM, but required by others.
  2. Compatibility with more input modes and formats.
  3. User and client documentation.
  4. Extensive testing and benchmarking.
  5. Money to pay for 1-4.
- WebAIM is seeking funding to help bring CaptionCaster to the masses.
Questions?

Internet Delivery of Real-time Captioning

Aaron Andersen and Jonathan Whiting

WebAIM: Web Accessibility in Mind

Credits

WebAIM is a project of

WebAIM's captioning system sponsored by

Popularity of Web-based Multimedia

Popularity of Web-based Multimedia

Popularity of Web-based Multimedia

What's the Problem?

Captioning Software

Caption Formats

Difficulties Specific to Live Media

Obtaining Live Captions

Solutions?

WebAIM's CaptionCaster

CaptionCaster

CaptionClient

CaptionConnect

CaptionConvert

Putting it all together

CaptionCaster Demo

CaptionCaster as a Solution

Future Plans for CaptionCaster

Questions?