CircleCI Setup for Playwright Testing

Setting up Playwright tests in CircleCI can be a complex challenge for many development teams, especially when scaling test suites across multiple environments. This comprehensive guide will walk you through parallelizing and splitting Playwright tests in CircleCI, enabling faster and more efficient continuous integration workflows.

Prerequisites#

• Node.js (v16.x or later)
• CircleCI account
• GitHub or Bitbucket repository
• Basic understanding of TypeScript
• Playwright installed in project
• Estimated setup time: 45-60 minutes

Configuring CircleCI for Playwright#

Project Structure Setup#

Before diving into CircleCI configuration, organize your Playwright test suite strategically. A typical structure might look like:

├── tests/
│   ├── unit/
│   ├── integration/
│   └── e2e/
├── playwright.config.ts
└── .circleci/config.yml

├── tests/
│   ├── unit/
│   ├── integration/
│   └── e2e/
├── playwright.config.ts
└── .circleci/config.yml

CircleCI Configuration File#

Create a .circleci/config.yml with comprehensive test parallelization:

version: 2.1
orbs:
  playwright: circleci/[email protected]

jobs:
  test:
    docker:
      - image: cimg/node:16.13.0-browsers
    parallelism: 4
    steps:
      - checkout
      - node/install-packages
      - run:
          command: |
            # Split tests across parallel executors
            TESTS=$(circleci tests glob "tests/**/*.spec.ts" | circleci tests split)
            npx playwright test $TESTS

version: 2.1
orbs:
  playwright: circleci/[email protected]

jobs:
  test:
    docker:
      - image: cimg/node:16.13.0-browsers
    parallelism: 4
    steps:
      - checkout
      - node/install-packages
      - run:
          command: |
            # Split tests across parallel executors
            TESTS=$(circleci tests glob "tests/**/*.spec.ts" | circleci tests split)
            npx playwright test $TESTS

Test Splitting Strategies#

CircleCI offers multiple test splitting approaches:

1. Timing-Based Splitting: Automatically distributes tests based on historical execution times
2. File-Based Splitting: Divides tests evenly across available executors
3. Custom Splitting: Implement manual test group definitions

Advanced Parallelization Techniques#

Dynamic Test Configuration#

Implement dynamic test configuration to optimize parallel execution:

import { defineConfig } from "@playwright/test";

export default defineConfig({
  workers: process.env.CI ? 4 : 2,
  timeout: 60000,
  reportSlowTests: { max: 3, threshold: 15000 },
});

import { defineConfig } from "@playwright/test";

export default defineConfig({
  workers: process.env.CI ? 4 : 2,
  timeout: 60000,
  reportSlowTests: { max: 3, threshold: 15000 },
});

Performance Monitoring#

Track test performance and identify bottlenecks:

import { test, expect } from "@playwright/test";

test.describe("Performance Critical Tests", () => {
  test("login flow", async ({ page }) => {
    const startTime = Date.now();
    // Test implementation
    const duration = Date.now() - startTime;
    expect(duration).toBeLessThan(5000); // Fail if > 5s
  });
});

import { test, expect } from "@playwright/test";

test.describe("Performance Critical Tests", () => {
  test("login flow", async ({ page }) => {
    const startTime = Date.now();
    // Test implementation
    const duration = Date.now() - startTime;
    expect(duration).toBeLessThan(5000); // Fail if > 5s
  });
});

Troubleshooting CircleCI Playwright Configurations#

Best Practices#

• Use deterministic test ordering
• Implement comprehensive error logging
• Configure test retries
• Monitor test execution times
• Use environment-specific configurations
• Implement caching for dependencies
• Set up comprehensive reporting

Next Steps#

• Explore advanced Playwright testing patterns
• Investigate container-based test execution
• Learn about browser compatibility testing
• Implement comprehensive monitoring
• Dive deeper into CI/CD optimization techniques