There are a growing number of software products available that allow web developers to evaluate the accessibility of their web pages and sites. Many tools also prompt the developer to make specific repairs. The Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C) maintains an extensive list of such tools in their document Evaluation, Repair, and Transformation Tools for Web Content Accessibility. There are presently dozens of products to choose from, and many of the leading vendors offer multiple products varying considerably in features, functionality, and price. This vast selection leaves web administrators at educational entities with the challenging task of determining which product is the best for supporting their school's web accessibility initiative.

Some independent third parties have conducted reviews and tests of several available tools; however, these resources quickly become outdated as new products are released. Nevertheless, these resources do serve as excellent examples of how one might objectively evaluate such products. The following resources can be particularly helpful:

  • WebTango Project
    WebTango is a research project of the Information School at the University of Washington. Its empirical research includes numerous studies on the effectiveness of automated website evaluation tools.
  • Evaluation and Repair: Web Accessibility
    Accessibility consultant Jim Thatcher provides detailed reviews of two products, Watchfire Bobby and Usablenet LIFT, in this online tutorial.
  • Government Computer News (GCN)
    This August 2001 online journal edition presents a comparison of six web accessibility products.

In addition to consulting the W3C list of tools and reading product reviews such as those listed above, the following steps are recommended for selecting web accessibility software:

  • Review the promotional literature for vendors' products.
  • View online demos of products, if available.
  • Install and test evaluation versions of products, if available.
  • Ask around. Ask vendors for references. Read the archives of discussion lists related to web design or web accessibility. One particularly active and helpful list is the mailing list of the W3C Web Accessibility Initiative (WAI) Interest Group. Subscription information is available on the WAI Interest Group home page.
  • Plan ahead. Know what features you would like in a product, and ask vendors about their support for those features.

Following are some questions to ask when planning ahead for a purchase of web accessibility software:

How automatic is an automatic accessibility checker?

Be careful of product marketing literature. Some vendors make bold claims about their numbers of automatic checks. Many aspects of web accessibility evaluation are inherently subjective and require human judgment. For example, all products are technically able to check for the presence of alternate text for graphics (HTML "alt" attributes). However, no product can ascertain whether the "alt" attribute is meaningful.

What operating system is supported?

Most web accessibility desktop software has been developed for Microsoft Windows. A limited number of products are also available for Macintosh operating systems. Some companies produce higher-end products designed to run on web servers. Some of these products support Windows, some support Linux or UNIX, and some support all of the above.

Which sets of accessibility checkpoints are predefined?

Most if not all available products have built-in support for W3C's Web Content Accessibility Guidelines 1.0 (WCAG 1.0) and the Section 508 Electronic and Information Technology Accessibility Standards. However, some states, educational entities, and other organizations have implemented their own web accessibility guidelines or standards, sometimes based on select and/or combined checkpoints from WCAG 1.0 and 508. Some products allow users to customize their evaluations by manually selecting individual checkpoints.

Is the tool able to automate repairs?

Some products are only able to evaluate websites and do not include functionality that automates repairs. Many products have repair wizards that walk users through the process of repairing basic accessibility problems such as missing alternate text for graphics. Some products offer site-wide repair, in which repairs can be implemented across multiple pages within a site. For example, if a graphic is used repeatedly throughout a site, a user can add alternate text to that graphic and choose to apply it to all pages on which that graphic is displayed. Also of interest, HiSoftware™ AccVerify includes wizards that walk users through the process of repairing inaccessible forms and tables.

Is the user interface accessible and usable?

As obvious as this question may seem, the answer is not universally "yes." Some products do an excellent job of evaluating web content for accessibility but cannot themselves be operated by users with disabilities. For more information about determining whether software products are accessible, see the AccessIT Knowledge Base article How can I tell whether a software application is accessible?

Does the tool assess site quality issues beyond accessibility?

Some educational entities might gain stronger support from their administration and faculty if accessibility efforts are perceived to be a benefit to them, rather than a burden. Sometimes web accessibility is more likely to be embraced by an institution if it is presented as an issue of web quality, along with other quality issues unrelated to accessibility. Some of the web accessibility products support this approach by including checks for broken links, orphan pages, missing meta tags (which will affect positioning within search results), link depth from the home page, download time, and other web quality issues.

Does the tool check spelling?

Spelling is a web quality issue, but it can also be an accessibility issue, since misspelled words are likely to be mispronounced by screen reader software. Some web accessibility tools conduct a spelling check as part of their evaluation process.

Does the tool include page preview filters?

An important step in the web design process is reviewing your web content with a variety of browsers and configurations. Current web accessibility software provides a variety of tools for simulating how users might perceive your site. Some products allow users to view their content linearized (as a screen reader user might perceive it), with objects and applets off, with images off, with scripts off, without color, and with various types of color blindness.

Can the product spider or crawl an entire website or domain?

Spidering (aka crawling) is the process by which software (in this case, web accessibility software) evaluates a starting page for accessibility but is automatically able to follow links from that page and evaluate the linked pages. Because educational entities typically purchase web accessibility evaluation products as part of a systematic effort toward improving their school's web accessibility, spidering or crawling pages throughout an entire website is a desirable feature. Most products today have this ability, though some are limited in scope; for example, they may only spider within the current domain or evaluate a certain number of pages.

Can the tool follow non-HTML links?

As the web expands beyond HTML, a growing number of pages are coded with Javascript and/or present content using proprietary formats such as Macromedia Flash, Adobe PDF, Microsoft Word, or other multimedia formats. If such content includes links to other documents on the site, web accessibility software might be unable to process the links, and site spidering could come to a sudden halt. Vendors are increasingly updating their products to support scripted and other nonstandard links, though products vary in their ability to do this and in which formats they support.

Can the tool assess the accessibility of any non-HTML content?

Though web accessibility software has gotten better at following links within non-HTML content, few products are able to evaluate that same content for accessibility. There are specific documented techniques for making some non-HTML content accessible, including Adobe PDF and Macromedia Flash content (see the AccessIT Knowledge Base articles Is PDF accessible? and Is Flash content accessible?). However, few evaluation tools are currently able to assess whether these techniques have been practiced. One exception is HiSoftware™ AccRepair for Macromedia Flash (no longer available).

How does the tool handle potential obstacles such as passwords, cookies, required form fields, and session IDs?

Most products now have mechanisms for addressing these obstacles, though typically they require manual intervention and pose interruptions to an otherwise automated process. No product exists that can automatically handle all possible variations of web security. The only way to ascertain whether a product will work in your environment is to identify a select few products that meet all of your other requirements and then to obtain and test evaluation versions of these products. Evaluation versions vary by product: some are fully functional products with expiration dates, some are limited by number of evaluated pages, and some simply offer a money-back return policy.

Does the tool have built-in features to facilitate web accessibility project management?

Web accessibility project management is the domain of the high-end products. These products allow project administrators to monitor and track accessibility site-wide over time, and they provide a variety of database features to support this effort.

How are the results presented?

Most products present an individual report for each web page evaluated. Products that are capable of spidering a site additionally present a summary report for the entire site. The detail within the individual page reports is typically customizable, so users can select the level of detail that they want presented. This customizability is particularly desirable if reports are to be distributed to individual web authors, since many authors might be overwhelmed by full-detail reports. Some products include line numbers when identifying accessibility errors, which makes it easier for web developers to correct the errors. Error correction is further facilitated by some products' ability to communicate with web authoring tools. With these products, users can jump directly from the accessibility report to the inaccessible content loaded within their web authoring software.

Is the results data exportable?

Some educational entities may desire flexibility in how report data is presented. Most of the high-end products provide export functionality, whereas the low-end products do not. Some of the low-end products, though they don't formally support export functionality, store raw evaluation data in XML format, so educational entities with in-house programming expertise can easily write programs to process this data and build custom reports.

Can the tool be run from a command prompt or as part of a batch process?

Educational entities may want to automate their accessibility assessments in order to run them at regular intervals, or to run them overnight when web server demand is lowest. High-end products support this functionality. Most low-end products do not.

Is the tool available with a software development kit (SDK) or open application program interface (API)?

Those educational entities with in-house programming expertise may want to develop their own programs for customizing reports, querying the results database, and integrating the web accessibility software's functionality into other applications. Vendors of high-end products approach this need in different ways, from distributing open-source APIs for interfacing with select popular applications to providing an SDK that allows users to develop all-new applications with the vendor's accessibility evaluation engine.