1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297---
id: accessibility-testing
title: "Accessibility testing"
---
## Introduction
Playwright can be used to test your application for many types of accessibility issues.
A few examples of problems this can catch include:
- Text that would be hard to read for users with vision impairments due to poor color contrast with the background behind it
- UI controls and form elements without labels that a screen reader could identify
- Interactive elements with duplicate IDs which can confuse assistive technologies
The following examples rely on the [`@axe-core/playwright`](https://npmjs.org/@axe-core/playwright) package which adds support for running the [axe accessibility testing engine](https://www.deque.com/axe/) as part of your Playwright tests.
:::note[Disclaimer]
Automated accessibility tests can detect some common accessibility problems such as missing or invalid properties. But many accessibility problems can only be discovered through manual testing. We recommend using a combination of automated testing, manual accessibility assessments, and inclusive user testing.
For manual assessments, we recommend [Accessibility Insights for Web](https://accessibilityinsights.io/docs/web/overview/?referrer=playwright-accessibility-testing-js), a free and open source dev tool that walks you through assessing a website for [WCAG 2.1 AA](https://www.w3.org/WAI/WCAG21/quickref/?currentsidebar=%23col_customize&levels=aaa) coverage.
:::
## Example accessibility tests
Accessibility tests work just like any other Playwright test. You can either create separate test cases for them, or integrate accessibility scans and assertions into your existing test cases.
The following examples demonstrate a few basic accessibility testing scenarios.
### Scanning an entire page
This example demonstrates how to test an entire page for automatically detectable accessibility violations. The test:
1. Imports the `@axe-core/playwright` package
1. Uses normal Playwright Test syntax to define a test case
1. Uses normal Playwright syntax to navigate to the page under test
1. Awaits `AxeBuilder.analyze()` to run the accessibility scan against the page
1. Uses normal Playwright Test [assertions](./test-assertions) to verify that there are no violations in the returned scan results
```js tab=js-ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright'; // 1
test.describe('homepage', () => { // 2
test('should not have any automatically detectable accessibility issues', async ({ page }) => {
await page.goto('https://your-site.com/'); // 3
const accessibilityScanResults = await new AxeBuilder({ page }).analyze(); // 4
expect(accessibilityScanResults.violations).toEqual([]); // 5
});
});
```
```js tab=js-js
const { test, expect } = require('@playwright/test');
const AxeBuilder = require('@axe-core/playwright').default; // 1
test.describe('homepage', () => { // 2
test('should not have any automatically detectable accessibility issues', async ({ page }) => {
await page.goto('https://your-site.com/'); // 3
const accessibilityScanResults = await new AxeBuilder({ page }).analyze(); // 4
expect(accessibilityScanResults.violations).toEqual([]); // 5
});
});
```
### Configuring axe to scan a specific part of a page
`@axe-core/playwright` supports many configuration options for axe. You can specify these options by using a Builder pattern with the `AxeBuilder` class.
For example, you can use [`AxeBuilder.include()`](https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md#axebuilderincludeselector-string--string) to constrain an accessibility scan to only run against one specific part of a page.
`AxeBuilder.analyze()` will scan the page *in its current state* when you call it. To scan parts of a page that are revealed based on UI interactions, use [Locators](./locators.md) to interact with the page before invoking `analyze()`:
```js
test('navigation menu should not have automatically detectable accessibility violations', async ({
page,
}) => {
await page.goto('https://your-site.com/');
await page.getByRole('button', { name: 'Navigation Menu' }).click();
// It is important to waitFor() the page to be in the desired
// state *before* running analyze(). Otherwise, axe might not
// find all the elements your test expects it to scan.
await page.locator('#navigation-menu-flyout').waitFor();
const accessibilityScanResults = await new AxeBuilder({ page })
.include('#navigation-menu-flyout')
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```
### Scanning for WCAG violations
By default, axe checks against a wide variety of accessibility rules. Some of these rules correspond to specific success criteria from the [Web Content Accessibility Guidelines (WCAG)](https://www.w3.org/TR/WCAG21/), and others are "best practice" rules that are not specifically required by any WCAG criterion.
You can constrain an accessibility scan to only run those rules which are "tagged" as corresponding to specific WCAG success criteria by using [`AxeBuilder.withTags()`](https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md#axebuilderwithtagstags-stringarray). For example, [Accessibility Insights for Web's Automated Checks](https://accessibilityinsights.io/docs/web/getstarted/fastpass/?referrer=playwright-accessibility-testing-js) only include axe rules that test for violations of WCAG A and AA success criteria; to match that behavior, you would use the tags `wcag2a`, `wcag2aa`, `wcag21a`, and `wcag21aa`.
Note that automated testing cannot detect all types of WCAG violations.
```js
test('should not have any automatically detectable WCAG A or AA violations', async ({ page }) => {
await page.goto('https://your-site.com/');
const accessibilityScanResults = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```
You can find a complete listing of the rule tags axe-core supports in [the "Axe-core Tags" section of the axe API documentation](https://www.deque.com/axe/core-documentation/api-documentation/#axecore-tags).
## Handling known issues
A common question when adding accessibility tests to an application is "how do I suppress known violations?" The following examples demonstrate a few techniques you can use.
### Excluding individual elements from a scan
If your application contains a few specific elements with known issues, you can use [`AxeBuilder.exclude()`](https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md#axebuilderexcludeselector-string--string) to exclude them from being scanned until you're able to fix the issues.
This is usually the simplest option, but it has some important downsides:
* `exclude()` will exclude the specified elements *and all of their descendants*. Avoid using it with components that contain many children.
* `exclude()` will prevent *all* rules from running against the specified elements, not just the rules corresponding to known issues.
Here is an example of excluding one element from being scanned in one specific test:
```js
test('should not have any accessibility violations outside of elements with known issues', async ({
page,
}) => {
await page.goto('https://your-site.com/page-with-known-issues');
const accessibilityScanResults = await new AxeBuilder({ page })
.exclude('#element-with-known-issue')
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```
If the element in question is used repeatedly in many pages, consider [using a test fixture](#using-a-test-fixture-for-common-axe-configuration) to reuse the same `AxeBuilder` configuration across multiple tests.
### Disabling individual scan rules
If your application contains many different preexisting violations of a specific rule, you can use [`AxeBuilder.disableRules()`](https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md#axebuilderdisablerulesrules-stringarray) to temporarily disable individual rules until you're able to fix the issues.
You can find the rule IDs to pass to `disableRules()` in the `id` property of the violations you want to suppress. A [complete list of axe's rules](https://github.com/dequelabs/axe-core/blob/master/doc/rule-descriptions.md) can be found in `axe-core`'s documentation.
```js
test('should not have any accessibility violations outside of rules with known issues', async ({
page,
}) => {
await page.goto('https://your-site.com/page-with-known-issues');
const accessibilityScanResults = await new AxeBuilder({ page })
.disableRules(['duplicate-id'])
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```
### Using snapshots to allow specific known issues
If you would like to allow for a more granular set of known issues, you can use [Snapshots](./test-snapshots.md) to verify that a set of preexisting violations has not changed. This approach avoids the downsides of using `AxeBuilder.exclude()` at the cost of slightly more complexity and fragility.
Do not use a snapshot of the entire `accessibilityScanResults.violations` array. It contains implementation details of the elements in question, such as a snippet of their rendered HTML; if you include these in your snapshots, it will make your tests prone to breaking every time one of the components in question changes for an unrelated reason:
```js
// Don't do this! This is fragile.
expect(accessibilityScanResults.violations).toMatchSnapshot();
```
Instead, create a *fingerprint* of the violation(s) in question that contains only enough information to uniquely identify the issue, and use a snapshot of the fingerprint:
```js
// This is less fragile than snapshotting the entire violations array.
expect(violationFingerprints(accessibilityScanResults)).toMatchSnapshot();
// my-test-utils.js
function violationFingerprints(accessibilityScanResults) {
const violationFingerprints = accessibilityScanResults.violations.map(violation => ({
rule: violation.id,
// These are CSS selectors which uniquely identify each element with
// a violation of the rule in question.
targets: violation.nodes.map(node => node.target),
}));
return JSON.stringify(violationFingerprints, null, 2);
}
```
## Exporting scan results as a test attachment
Most accessibility tests are primarily concerned with the `violations` property of the axe scan results. However, the scan results contain more than just `violations`. For example, the results also contain information about rules which passed and about elements which axe found to have inconclusive results for some rules. This information can be useful for debugging tests that aren't detecting all the violations you expect them to.
To include *all* of the scan results as part of your test results for debugging purposes, you can add the scan results as a test attachment with [`testInfo.attach()`](./api/class-testinfo#test-info-attach). [Reporters](./test-reporters) can then embed or link the full results as part of your test output.
The following example demonstrates attaching scan results to a test:
```js
test('example with attachment', async ({ page }, testInfo) => {
await page.goto('https://your-site.com/');
const accessibilityScanResults = await new AxeBuilder({ page }).analyze();
await testInfo.attach('accessibility-scan-results', {
body: JSON.stringify(accessibilityScanResults, null, 2),
contentType: 'application/json'
});
expect(accessibilityScanResults.violations).toEqual([]);
});
```
## Using a test fixture for common axe configuration
[Test fixtures](./test-fixtures) are a good way to share common `AxeBuilder` configuration across many tests. Some scenarios where this might be useful include:
* Using a common set of rules among all of your tests
* Suppressing a known violation in a common element which appears in many different pages
* Attaching standalone accessibility reports consistently for many scans
The following example demonstrates creating and using a test fixture that covers each of those scenarios.
### Creating a fixture
This example fixture creates an `AxeBuilder` object which is pre-configured with shared `withTags()` and `exclude()` configuration.
```js tab=js-ts title="axe-test.ts"
import { test as base } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
type AxeFixture = {
makeAxeBuilder: () => AxeBuilder;
};
// Extend base test by providing "makeAxeBuilder"
//
// This new "test" can be used in multiple test files, and each of them will get
// a consistently configured AxeBuilder instance.
export const test = base.extend<AxeFixture>({
makeAxeBuilder: async ({ page }, use) => {
const makeAxeBuilder = () => new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.exclude('#commonly-reused-element-with-known-issue');
await use(makeAxeBuilder);
}
});
export { expect } from '@playwright/test';
```
```js tab=js-js title="axe-test.js"
const base = require('@playwright/test');
const AxeBuilder = require('@axe-core/playwright').default;
// Extend base test by providing "makeAxeBuilder"
//
// This new "test" can be used in multiple test files, and each of them will get
// a consistently configured AxeBuilder instance.
exports.test = base.test.extend({
makeAxeBuilder: async ({ page }, use) => {
const makeAxeBuilder = () => new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
.exclude('#commonly-reused-element-with-known-issue');
await use(makeAxeBuilder);
}
});
exports.expect = base.expect;
```
### Using a fixture
To use the fixture, replace the earlier examples' `new AxeBuilder({ page })` with the newly defined `makeAxeBuilder` fixture:
```js
const { test, expect } = require('./axe-test');
test('example using custom fixture', async ({ page, makeAxeBuilder }) => {
await page.goto('https://your-site.com/');
const accessibilityScanResults = await makeAxeBuilder()
// Automatically uses the shared AxeBuilder configuration,
// but supports additional test-specific configuration too
.include('#specific-element-under-test')
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```