Automated and manual testing

Automated accessibility testing can be carried out by developers, quality assurance (QA) testers, or anyone involved in the website project lifecycle. They're quick tests that can be completed using browser extensions and online tools. The tests provide immediate feedback on a sub-set of Web Content Accessibility Guidelines (WCAG) 2.2 success criteria.

Benefits

The main benefit of automated accessibility testing is it's quick and easy to do at any point during development, QA or maintenance. Tools can check webpages, or a set of webpages, in a matter of minutes, which is much faster than a human can. Some services will check your site at regular intervals automatically and provide a report.

Automated accessibility testing is also beneficial when maintaining a website. You can carry out tests at regular intervals to check updates are accessible.

Limitations

Automated accessibility tests can't test all WCAG 2.2 success criteria. As the tests are automated, they can only verify content which can be checked in the code. You can't check for context or relevance, which means you'll miss some issues if you only do automated checks.

For example, an automated test can confirm that an image correctly has an alt attribute, but it can't tell you if the text description is appropriate or not.

The true value of automated accessibility testing is when it's combined with manual testing and testing with people with disabilities. This ensures both WCAG compliance and the best possible usability for colleagues with disabilities.

When to carry out automated tests

Automated accessibility tests can be carried out by anyone in the project lifecycle. They're particularly useful for developers during development and for testers during QA. Automated tests should also be included as part of ongoing maintenance.

The earlier and more often tests are done, the easier it is to identify and fix issues before they become a serious problem. Regular automated testing reduces the risk of fixing content towards the end of a project or post-launch. It also helps maintain the accessibility of a site as it is updated.

Particularly useful times to include automated testing are:

• as part of your definition of done for a new feature or page
• when you update your site content
• at regular intervals for accessibility maintenance
• before launching a new site
• after conducting a site redesign

Automated testing tools 

There are many automated testing tools. What you use will depend on what tools you can access, what tools you like and if they fit into your process. 

Accessibility insights

Accessibility insights is a free browser extension for Chrome and Edge available on Windows and macOS. It can run two testing scenarios:

1. Fastpass, which identifies accessibility issues without you having to check the HyperText Markup Language (HTML) .
2. Assessment, which produces a report pointing to issues in the HTML and provides step-by-step manual checks.

ARC Toolkit

The ARC Toolkit is a free browser extension for Chrome available on Windows and macOS. It's integrated with the Chrome browser inspect tools and produces reports highlighting errors both within the rendered page and in the HTML.

axe DevTools

Another free browser extension for Chrome and Edge available on Windows and macOS is axe DevTools. It's integrated with the browser inspect tools and produces reports highlighting errors both within the rendered page and in the HTML.

WAVE

WAVE from WebAIM is a free online tool and browser extension for Chrome and Firefox available on Windows and macOS. Both the online tool and extension will provide findings in a report.

Other types of testing

A combination of approaches to testing will ensure your content meets WCAG 2.2 success criteria, assistive technologies can access it and people with disabilities can use it.

Manual testing

Manual testing is done by a human and ideally in combination with automated testing. It can be carried out by anyone in the project lifecycle, but in particular by developers during development and testers during QA.

Manual testing assesses some of the things automated testing can't. Automated testing can test the presence of an alt attribute, but when you do a manual test this can assess the appropriateness of the text description.

You can do manual testing in combination with automated testing tools, as well as testing with a variety of assistive technologies. These include keyboard, screen readers, screen magnification software and browser preferences for colour and zoom.

Testing with people with disabilities

Testing with people with disabilities helps identify barriers that may go undetected by manual and automated testing. By including people with disabilities in usability testing, you and your product team learn first-hand what people need to use your products. For example, you might do usability testing with screen readers, screen magnification and speech recognition software, or customise text, colour and zoom settings. All of this helps you build an understanding of how colleagues who use assistive technologies interact with content. 

Tips

Follow these tips to improve user experience for your colleagues:

• include passing automated testing as part of your definition of done
• ensure you schedule time to include automated accessibility testing during development
• take the time to explore the tools to find out which ones work best for you
• combine automated testing with manual testing and testing with people with disabilities