Testing Accessibility Features With Playwright

In one of my previous posts, I mentioned the importance of having e2e tests to verify that your keyboard navigation works properly. I also suggested Playwright as a tool to write those tests.

I owe you a little tutorial on how to do that. And because keyboard navigation is not the only accessibility feature you should test and since Playwright has some nice built-in tools for accessibility testing, I will also show you how to test other aspects of your website.

What is Playwright?

Let's start with a short introduction for those who don't know Playwright yet. Playwright is an open-source automation library developed by Microsoft, designed to enable developers and testers to write scripts that can control a web browser's behavior. It is a Node.js library that provides a high-level API to control headless Chrome, Firefox, and WebKit with a single interface. This means you can write your tests once and run them in multiple browsers, ensuring cross-browser compatibility.

If you're new to Playwright, you can check out Dane's article, which nicely introduces the basics such as configuration or selectors, and presents some nice test examples.

Keyboard Accessibility Testing

Now, as promised, let's dive into keyboard accessibility testing with Playwright. As I already mentioned, keyboard navigation is one of the most important accessibility features to test, given that it can break anytime you add new functionality to your application and the problem can easily go unnoticed. It is also one of the easiest to test with Playwright because it provides a neat keyboard API that allows you to simulate keyboard input and verify that the correct elements are focused or activated.

Verifying Element Focus

To check which element is selected after a series of keyboard inputs, you can use the toBeFocused() method to check whether it is focused or not, like this:

await page.keyboard.press('Tab');
await page.keyboard.press('Tab');
await page.keyboard.press('Tab');

const textbox = page.getByRole('textbox');

// Verify that the textbox is focused
await expect(textbox).toBeFocused();

// Tab out of the textbox
await page.keyboard.press('Tab');

// And verify that the textbox is not focused
await expect(textbox).not.toBeFocused();

This code snippet simulates pressing the Tab key three times and then checks that an input is focused. It then presses the Tab key again, verifying that the input is no longer focused.

If you read my previous article, you might remember that I mentioned that many websites have a bug that prevents you from tabbing out of a textbox. This simple test can catch this bug and prevent headaches for your users.

Simulating Complex Keyboard Interactions

Playwright's keyboard class provides a keyboard.press() method to simulate pressing a key and releasing it immediately. It allows you to simulate pressing a single key, a combination of keys, or a key combination with a modifier key:

await page.keyboard.press('Shift+KeyA'); // Presses uppercase 'A'
await page.keyboard.press('Control+A'); // Select all text (on Windows/Linux)

You can find the full list of keys you can pass to the keyboard.press function on MDN.

If you need finer control over the keyboard, you can use the keyboard.down() and keyboard.up() methods to simulate pressing and releasing a key separately. One use case for this is to simulate pressing a key and holding it down for a certain amount of time, such as holding the Shift key to select multiple elements or lines of text:

// Press Shift and hold it down
await page.keyboard.down('Shift');

//Press the down arrow 5 times
for (let i = 0; i < 5; i++) {
  await page.keyboard.press('ArrowDown');
}

//Release the Shift key
await page.keyboard.up('Shift');

// and verify that the first 5 list items are selected
for (let i = 0; i < 5; i++) {
    // construct the selector for the ith list item
    const listItemSelector = `ul#myList > li:nth-child(${i + 1})`;

    // use the `toHaveClass` assertion to check if the item has the 'selected' class
    await expect(page.locator(listItemSelector)).toHaveClass(/.*selected.*/);
}

You can also use the keyboard.insertText() method to insert text into an element, but note that modifier keys do not affect this method (e.g. holding down the Shift key will not type the text uppercase) and it only dispatches the input event and does not emit the keydown, keyup or keypress events.

// press Tab to focus the textbox
await page.keyboard.press('Tab');
// insert text into the textbox
await page.keyboard.insertText('Hello World!');
// expect the textbox to contain the text
await expect(page.getByRole('textbox')).toHaveValue('Hello World!');

The keyboard API allows you to test keyboard input, keyboard shortcuts; and ensure that your application responds correctly to user input without a mouse. Although this might seem unnecessary, believe me when I say it is not. As I mentioned in my previous article, many applications try to implement keyboard navigation but contain a nasty bug that renders the effort useless. And it is worth spending some time writing a few simple e2e tests to avoid invalidating your efforts.

Testing Other Accessibility Features

This article focuses mostly on keyboard accessibility, but I have good news! You can easily test other accessibility aspects with Playwright too! It conveniently integrates with the axe accessibility testing engine using the @axe-core/playwright package to run automated accessibility tests. These tests can catch various issues, such as poor color contrast, missing labels on UI controls, and duplicate IDs that can confuse assistive technologies.

Example Test for Entire Page Accessibility

Here's how you can scan your entire page for accessibility issues using Playwright:

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test.describe('homepage', () => {
  test('should not have any automatically detectable accessibility issues', async ({ page }) => {
    await page.goto('https://your-site.com/');
    const accessibilityScanResults = await new AxeBuilder({ page }).analyze();
    expect(accessibilityScanResults.violations).toEqual([]);
  });
});

Scanning Specific Page Sections

You can also target specific parts of your page by using AxeBuilder.include() to focus the accessibility scan:

test('navigation menu should not have automatically detectable accessibility violations', async ({ page }) => {
  await page.goto('https://your-site.com/');
  await page.getByRole('button', { name: 'Navigation Menu' }).click();
  await page.locator('#navigation-menu-flyout').waitFor();
  const accessibilityScanResults = await new AxeBuilder({ page })
    .include('#navigation-menu-flyout')
    .analyze();
  expect(accessibilityScanResults.violations).toEqual([]);
});

Handling Known Accessibility Issues

When you have known accessibility issues that you cannot immediately fix, Playwright allows you to exclude certain elements from scans or disable specific rules:

const accessibilityScanResults = await new AxeBuilder({ page })
  .exclude('#element-with-known-issue')
  .analyze();

Reporting Accessibility Issues

If you want to report accessibility issues in your test results, you can use the testInfo.attach() method to attach the scan results to your test report:

test('example with attachment', async ({ page }, testInfo) => {
  await page.goto('https://your-site.com/');

  const accessibilityScanResults = await new AxeBuilder({ page }).analyze();

  await testInfo.attach('accessibility-scan-results', {
    body: JSON.stringify(accessibilityScanResults, null, 2),
    contentType: 'application/json'
  });

  expect(accessibilityScanResults.violations).toEqual([]);
});

You can then use reporters to embed or link the scan results in your test output.

Conclusion

While your e2e tests should not serve as a replacement for manual testing and you should still check your app yourself to catch any issues that automated tests won't reveal, Playwright provides a convenient way to automate checks whether all of your website's accessibility features work as intended. It allows you to test keyboard navigation and other accessibility aspects such as color contrast or missing labels to make sure you aren't shipping a broken experience.

Testing accessibility with Playwright requires little effort, and in my opinion, is very much worth it. So, if you haven't done so yet, I encourage you to give it a try! And if you're still unconvinced, read about how I broke my hand to appreciate the importance of accessibility.

About the author

Jan Kaiser

Senior Software Engineer

Software engineer passionate about everything web-related. Particularly loves Angular, SCSS, TypeScript, soccer and dachshunds

@honzikec@honzikec