Digital accessibility testing

Accessibility is part of usability and should be part of usability testing. A best practice for your accessibility testing approach includes the 3-tier approach.

Include accessibility into each phase of your product development cycle: design, build, test.

The W3C offers this great resource Evaluating Web Accessibility Overview.

Check out Siteimprove's Accessibility 101 webinars (MyAccess login required) for a primary introduction on testing and the importance of combining automated testing with manual testing.

The 3-tier testing approach

This 3-tier approach for testing a web page/site for accessibility compliance involves different kinds of testing methods, tools and testers for each tier. You need a combination of all to get a feel for the whole picture.

Automated testing
Automated and manual checks
Testing with users from the disability community

1) Automated testing

Siteimprove is a website monitoring tool used at UCSF that scans and scores websites on accessibility. The Siteimprove contract with the University of California Office of the President (UCOP) represents UC's commitment to provide accessible and inclusive websites to our external audiences. Visit the UCOP Siteimprove page.

Automated testing will help you understand the basics of Web Content Accessibility Guidelines (WCAG). Most tools have suggestions for correcting, links to the criteria, and general guidelines. Even the better testing tools catch less than 25% of the WCAG 2.0 AA guidelines. False positives are common and take time to weed out, hence the manual checks that come into play in this tiered approach.

There is more to the accessibility of your site beyond the Siteimprove score as the score calculation currently does not include issues with lacking or inaccurate video captions or issues with PDFs, but the speed and convenience of automated testing make it an integral part of testing.

It is a good start but not a reliable evaluation of true accessibility. Some tools do a better job of presenting the findings. It is a matter of personal preference. Also, there are many tools that are improving rapidly so keep checking to see what is new. Additional tools can be found toward the bottom of this page under the heading "Evaluation tools".

2) Automated and manual checks

Manual checks

Automated testing results flag issues that require manual checks. Manual checks require an understanding of the goals of the web page/site being tested and an understanding of the full context of the content to judge the accessibility.

Basic expectations of manual testing for UCSF web pages

Enlarge text size up to 200%. Using Firefox browser go to Options or Menu bar > View > Zoom text only > 200%.
- Verify there is no loss of content or functionality (e.g. no parts of the text are clipped, boxes do not overlap, controls are not obscured or separated from their labels, etc.).
- Is relevant information close enough together to give context? This is especially important with forms.
- Learn more in the video "Web Accessibility Perspectives: Customizable Text".
Navigate by using keyboard only. Look for focus and tabbing order. Can you get to all interactive objects? What you experience on a particular website may be different for each browser. Learn more in the video "Web Accessibility Perspectives: Keyboard Compatibility".
- Click in the browser address bar
- Take your hand off your mouse and use only your keyboard
- Start tabbing around
- Use Shift+Tab to navigate backward
- Spacebar activates checkboxes and buttons
Use automated tools to catch low-hanging fruit.
- Free browsers plugin suggestions include WAVE or Siteimprove Accessibility Checker.
- Check out the color contrast-analyzing tools on our Accessible Color Best Practices page.
Responsive design and content scaling:
- Verify there is no loss of content or functionality on the small screen of a mobile phone like iPhone or Android.
- Verify the same on a laptop/desktop by expanding and shrinking the size of the browser window to simulate different sized monitors.
Captions: Test the quality and accuracy of the captioning. Watch your video with the closed captioning on.
One in 20 Americans is deaf or hard of hearing. Half of those are in the very fast-growing age 65+ demographic.
- Are you reading what you are hearing?
- How is the quality?

People with standard hearing enjoy the benefits of captioning, too:

Increases comprehension of dialogue that is spoken very quickly, with accents, mumbling, or background noise
Clarity for full names, medical terminology, etc.
Helps maintain concentration
Easier to follow along with the speech for English-as-a-second-language viewers
Viewing in sound-sensitive environments, such as libraries and public spaces

Check out our Accessible Videos Best Practices page for more on captioning.

More examples of manual testing

For color testing, interact with your site with the color grayed out. Grayscale is a very simplistic view of color perception, but it is a good starting point. Meaning conveyed by color alone will be missed by many.

In Windows, use the Grayscale Color Filter.
In Apple OSX go to System Preferences > Accessibility > Display and check the box labeled “Use Grayscale”.

More information following in the Checklists and Evaluation tools sections.

Testing with assistive technology (AT)

Assistive technology (AT) is a broad term for hardware and software that supports people with disabilities. One of the best ways to ensure your website works for everyone, regardless of ability, is to use it with one or more types of AT.

People without disabilities are often introduced to the world of AT through screen readers. Using a screen reader will help you understand how your website’s information is conveyed to blind or low-vision users.

A screen reader is a software that allows a blind or visually impaired person to use a computer. It includes commands that allow the user to move the cursor, read the text, determine formatting information and to perform other essential computing tasks, while also supporting the operating system’s built-in keyboard shortcuts. Commands can be issued from a keyboard or Braille display, which eliminates the need for a mouse. The information is conveyed using either synthetic speech or Braille output.

5-minute screen reader demo

Non-visual display access (NVDA) is an open-source screen reader that is free to download and use on Windows. JAWS is another 3rd-party screen reader for Windows. Macs include Voiceover as part of the Apple operating system.

Assistive technology checklists and reference sheets provide guides for testing with screen readers in general and using NVDA specifically.

There is controversy about sighted users testing with screen readers. Our colleagues at UC-Berkeley have an article on why they advise against it. See the heading "That sounds great, Pecan. Should I learn how to use a screen reader?".

For example, a sighted user may think there is an accessibility issue when it may simply be a matter of not knowing the right command to use, and the time spent learning how to use a screen reader could be better spent considering other assistive technologies and learning how to create content that conforms to WCAG and other accessibility best practices.

Use automated and manual testing together

When you use automated testing, eliminate false positives first. Then manually check the rest of the elements using a checklist developed specifically for your website page to get a clearer picture of its accessibility. You should be able to fix many issues on the spot. For example, adding alt text to images is a simple fix.

Mobile testing

This mobile accessibility resource is compiled by our colleagues at UC-Berkeley. The note that while addressing accessibility may at first feel like a lot of work, once you master the learning curve, it will become part of your standard development process.

3) Test with users from the disability community

We can provide a list of vendors that provide this service, typically called usability testing (UT). Since a cost is involved, do your due diligence and get your content accessible to the best of your capability before paying for this type of service.

If you are on a shoestring budget, start with about five pages that represent your site. A common set is:

Home
Contact us
Forms,
Most-visited pages
Pages in a path critical to a process, such as registering for a course, getting directions, or making a donation.

Checklists

The WCAG 2.0 Guidelines Level A and AA list more than 50 success criteria. Maintaining a checklist of satisfied criteria will help you manage your accessibility review process. The lists include short descriptions of each success criteria and several include links to reference documentation.

The checklist will get you started, but it's best to customize your list to address what you are testing, and only what you need makes the process a bit more manageable.

The A11Y Project's Web Accessibility checklist (a beginner’s guide to accessibility)
McMaster University Web Page checklist (Interactive checklist for WCAG 2.0 Level AA)
Paul J Adam's WCAG 2.0 checklist (Interactive checklist with option to remove AAA checks from the list, all links go to the W3C WCAG 2.0 checklist)
W3C WCAG 2.0 checklist (HTML)
WebAIM WCAG 2.0 checklist (HTML and PDF)
Wuhcag checklists by level (HTML)

Test early, test often

These simple tools will go a long way in teaching you about accessibility. Checking and ensuring accessibility is for every stage, but is most cost effective in development.

A reiterative approach to testing and correcting can give you balance. Trying to test and fix all at once is not practical. The biggest misconception is that Accessibility is applied – it is an integral part of the creation and website management process.

Evaluation tools

Bookmarklets

ANDI (Accessible Name & Description Inspector) is a free tool to test web content for accessibility. It will flag issues and reveal what a screen reader is expected to say for interactive elements.
HTML CodeSniffer – JS bookmarklet that works with all browsers. Provides clear results with links to appropriate WCAG 2.0 documentation and suggested fixes.
tota11y Accessibility Visualization Toolkit – Overlays indicators on your page where improvements are needed, along with explanations.

Browser extensions

Accessibility Developer Tools Chrome Extension – Gives you an accessibility audit and allows you to see accessibility properties for any element in the page.
aXe Firefox Extension – Deque’s open source accessibility extension for Firefox allows you to perform accessibility testing in the Firefox browser.
Lighthouse Accessibility Tool – Lighthouse is an open-source, automated tool for improving the performance, quality, and correctness of your web apps.
Siteimprove Accessibility Checker - This free Google Chrome extension lets you check any multi-step form, dynamic content, or non-public page for accessibility issues. All analysis is done entirely within the Chrome browser, which means you can securely check password-protected pages, too.
WAVE Web Accessibility Evaluation Tool – The online version of the popular WAVE tool. It checks single HTML pages and returns a detailed report highlighting areas of concern on the page.
Web Developer Toolbar – Extension for Firefox and Chrome with tools that make manual accessibility testing easier.

GitHub tool

AccessLint - A GitHub app that finds accessibility issues in your pull request. Test in your development workflow before code goes live.

Digital Accessibility Wrap Up for 2022

Content Restricted