Vercel previews offer a great way to test PRs for a project. They have a predefined environment and don’t require any additional setup work from the reviewer to test changes quickly. Many projects also use end-to-end tests with Playwright as part of the review process to ensure that no regressions slip uncaught.
Usually, workflows configure Playwright to run against a project running on the GitHub action worker itself, maybe with dependencies in Docker containers as well, however, why bother setting that all up and configuring yet another environment for your app to run in when there’s a working preview right there? Not only that, the Vercel preview will be more similar to production as it’s running on the same infrastructure, allowing you to be more confident about the accuracy of your tests.
In this article, I’ll show you how you can run Playwright against the Vercel preview associated with a PR.
Setting up the Vercel Project
To set up a project in Vercel, we first need to have a codebase. I’m going to use the Next.js starter, but you can use whatever you like. What technology stack you use for this project won’t matter, as integrating Playwright with it will be the same experience.
You can create a Next.js project with the following command:
npx create-next-app@latest
If you’ve selected all of the defaults, you should be able to run npm run dev and navigate to the app at http://localhost:3000.
Setting up Playwright
We will set up Playwright the standard way and make a few small changes to the configuration and the example test so that they run against our site and not the Playwright site. Setup Playwright in our existing project by running the following command:
npm init playwright@latest
Install all browsers when prompted, and for the workflow question, say no since the one we’re going to use will work differently than the default one. The default workflow doesn’t set up a development server by default, and if that is enabled, it will run on the GitHub action virtual machine instead of against our Vercel deployment.
To make Playwright run tests against the Vercel deployment, we’ll need to define a baseUrl in playwright.config.ts and send an additional header called X-Vercel-Protection-Bypass where we'll pass the bypass secret that we generated earlier so that we don’t get blocked from making requests to the deployment. I’ll cover how to add this environment variable to GitHub later.
export default defineConfig({
...
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: process.env.DEPLOYMENT_URL ?? "http://127.0.0.1:3000",
extraHTTPHeaders: {
"X-Vercel-Protection-Bypass":
process.env.VERCEL_AUTOMATION_BYPASS_SECRET ?? "",
},
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: "on-first-retry",
},
...
}
Our GitHub workflow will set the DEPLOYMENT_URL environment variable automatically.
Now, in tests/example.spec.ts let’s rewrite the tests to work against the Next.js starter that we generated earlier:
import { test, expect } from "@playwright/test";
test("has title", async ({ page }) => {
await page.goto("/");
await expect(page).toHaveTitle(/Create Next App/);
});
test("has deploy button", async ({ page }) => {
await page.goto("/");
await expect(page.getByRole("link", { name: "Deploy now" })).toBeVisible();
});
This is similar to the default test provided by Playwright. The main difference is we’re loading pages relative to baseURL instead of Playwright’s website. With that done and your Next.js dev server running, you should be able to run npx playwright test and see 6 passing tests against your local server. Now that the boilerplate is handled let’s get to the interesting part.
The Workflow
There is a lot going on in the workflow that we’ll be using, so we’ll go through it step by step, starting from the top. At the top of the file, we name the workflow and specify when it will run.
name: E2E Tests (Playwright)
on:
pull_request:
branches:
- main
push:
branches:
- main
This workflow will run against new PRs against the default branch and whenever new commits are merged against it. If you only want the workflow to run against PRs, you can remove the push object.
Be careful about running workflows against your main branch if the deployment associated with it in Vercel is the production deployment. Some tests might not be safe to run against production such as destructive tests or those that modify customer data. In our simple example, however, this isn’t something to worry about.
Installing Playwright in the Virtual Machine
Workflows have jobs associated with them, and each job has multiple steps. Our test job takes a few steps to set up our project and install Playwright.
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: 'npm'
- name: Install npm dependencies
run: npm ci
- name: Install system dependencies needed by Playwright
run: sudo npx playwright install-deps
- name: Install all supported Playwright browsers
run: npx playwright install
The actions/checkout@v4 step clones our code since it isn’t available straight out of the gate. After that, we install Node v22 with actions/setup-node@v4, which, at the time of writing this article, is the latest LTS available. The latest LTS version of Node should always work with Playwright. With the project cloned and Node installed, we can install dependencies now. We run npm ci to install packages using the versions specified in the lock file.
After our JS dependencies are installed, we have to install dependencies for Playwright now. sudo npx playwright install-deps installs all system dependencies that Playwright needs to work using apt, which is the package manager used by Ubuntu. This command needs to be run as the administrative user since higher privilege is needed to install system packages. Playwright’s dependencies aren’t all available in npm because the browser engines are native code that has native library dependencies that aren’t in the registry.
Vercel Preview URL and GitHub Action Await Vercel
The next couple of steps is where the magic happens. We need two things to happen to run our tests against the deployment. First, we need the URL of the deployment we want to test. Second, we want to wait until the deployment is ready to go before we run our tests. We have written about this topic before on our blog if you want more information about this step, but we’ll reiterate some of that here.
Thankfully, the community has created GitHub actions that allow us to do this called zentered/vercel-preview-url and UnlyEd/github-action-await-vercel. Here is how you can use these actions:
jobs:
test:
runs-on: ubuntu-latest
steps:
...
- name: Get the Vercel preview url
id: vercel_preview_url
uses: zentered/vercel-preview-url@v1.4.0
env:
VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
with:
vercel_app: 'playwright-vercel-preview-demo'
- uses: UnlyEd/github-action-await-vercel@v2.0.0
env:
VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
with:
deployment-url: ${{ format('https://{0}', steps.vercel_preview_url.outputs.preview_url) }}
timeout: 420
poll-interval: 15
There are a few things to take note of here. Firstly, some variables need to be set that will differ from project to project. vercel_app in the zentered/vercel-preview-url step needs to be set to the name of your project in Vercel that was created earlier.
The other variable that you need is the VERCEL_TOKEN environment variable. You can get this by going to Vercel > Account Settings > Tokens and creating a token in the form that appears. For the scope, select the account that has your project.
To put VERCEL_TOKEN into GitHub, navigate to your repo, go to Settings > Secrets and variables > Actions and add it to Repository secrets.
We should also add VERCEL_AUTOMATION_BYPASS_SECRETl. In Vercel, go to your project then navigate to Settings > Deployment Protection > Protection Bypass for Automation. From here you can add the secret, copy it to your clipboard, and put it in your GitHub action environment variables just like we did with VERCEL_TOKEN.
With the variables taken care of, let’s take a look at how these two steps work together. You will notice that the zentered/vercel-preview-url step has an ID set to vercel_preview_url. We need this so we can pass the URL we receive to the UnlyEd/github-action-await-vercel action, as it needs a URL to know which deployment to wait on.
Running Playwright
After the last steps we just added, our deployment should be ready to go, and we can run our tests! The following steps will run the Playwright tests against the deployment and save the results to GitHub:
jobs:
test:
runs-on: ubuntu-latest
steps:
...
- name: Run E2E tests
run:
npx playwright test
env:
DEPLOYMENT_URL: ${{ format('https://{0}', steps.vercel_preview_url.outputs.preview_url) }}
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
- name: Upload the Playwright report
uses: actions/upload-artifact@v4
if: always() # Always run regardless if the tests pass or fail
with:
name: playwright-report
path: ${{ format('{0}/playwright-report/', github.workspace) }}
retention-days: 30
In the first step, where we run the tests, we pass in the environment variables needed by our Playwright configuration that’s stored in playwright.config.ts. DEPLOYMENT_URL uses the Vercel deployment URL we got in an earlier step, and VERCEL_AUTOMATION_BYPASS_SECRET gets passed the secret with the same name directly from the GitHub secret store.
The second step uploads a report of how the tests did to GitHub, regardless of whether they’ve passed or failed. If you need to access these reports, you can find them in the GitHub action log. There will be a link in the last step that will allow you to download a zip file.
Once this workflow is in the default branch, it should start working for all new PRs! It’s important to note that this won’t work for forked PRs unless they are explicitly approved, as that’s a potential security hazard that can lead to secrets being leaked. You can read more about this in the GitHub documentation.
One Caveat
There’s one caveat that is worth mentioning with this approach, which is latency. Since your application is being served by Vercel and not locally on the GitHub action instance itself, there will be longer round-trips to it. This could result in your tests taking longer to execute. How much latency there is can vary based on what region your runner ends up being hosted in and whether the pages you’re loading are served from the edge or not.
Conclusion
Running your Playwright tests against Vercel preview deployments provides a robust way of running your tests against new code in an environment that more closely aligns with production. Doing this also eliminates the need to create and maintain a 2nd test environment under which your project needs to work.
About the author
