Testrail

What This Skill Does

This skill provides bidirectional synchronization between Playwright tests and TestRail test management. It allows users to import test cases from TestRail to generate Playwright tests, push test results from Playwright to TestRail, create test runs in TestRail, and sync the status between the two systems. It streamlines the testing workflow by integrating test management and automation.

When to Use

• Import test cases from TestRail to generate Playwright tests.
• Push Playwright test results to TestRail.
• Create a new test run in TestRail.
• Sync the status between TestRail and Playwright tests.
• Update test cases in TestRail.
• Manage test cases in TestRail and automate their execution.

Key Features

• Imports test cases from TestRail.
• Pushes test results to TestRail.
• Creates test runs in TestRail.
• Syncs status between TestRail and Playwright.
• Requires TestRail URL, user, and API key.
• Uses test annotations to map Playwright tests to TestRail cases.

Manual Installation

TestRail Integration

Bidirectional sync between Playwright tests and TestRail test management.

Prerequisites

Environment variables must be set: TESTRAIL_URL — e.g., https://your-instance.testrail.io TESTRAIL_USER — your email TESTRAIL_API_KEY — API key from TestRail

If not set, inform the user how to configure them and stop.

Capabilities

1. Import Test Cases → Generate Playwright Tests

/pw:testrail import --project <id> --suite <id>

Steps: Call testrail_get_cases MCP tool to fetch test cases For each test case:

• Read title, preconditions, steps, expected results
• Map to a Playwright test using appropriate template
• Include TestRail case ID as test annotation: test.info().annotations.push({ type: 'testrail', description: 'C12345' }) Generate test files grouped by section Report: X cases imported, Y tests generated

2. Push Test Results → TestRail

/pw:testrail push --run <id>

Steps: Run Playwright tests with JSON reporter: npx playwright test --reporter=json > test-results.json Parse results: map each test to its TestRail case ID (from annotations) Call testrail_add_result MCP tool for each test:

• Pass → status_id: 1
• Fail → status_id: 5, include error message
• Skip → status_id: 2 Report: X results pushed, Y passed, Z failed

3. Create Test Run

/pw:testrail run --project <id> --name "Sprint 42 Regression"

Steps: Call testrail_add_run MCP tool Include all test case IDs found in Playwright test annotations Return run ID for result pushing

4. Sync Status

/pw:testrail status --project <id>

Steps: Fetch test cases from TestRail Scan local Playwright tests for TestRail annotations Report coverage: TestRail cases: 150 Playwright tests with TestRail IDs: 120 Unlinked TestRail cases: 30 Playwright tests without TestRail IDs: 15

5. Update Test Cases in TestRail

/pw:testrail update --case <id>

Steps: Read the Playwright test for this case ID Extract steps and expected results from test code Call testrail_update_case MCP tool to update steps

MCP Tools Used

Test Annotation Format

All Playwright tests linked to TestRail include:

test('should login successfully', async ({ page }) => { test.info().annotations.push({ type: 'testrail', description: 'C12345', }); // ... test code });

This annotation is the bridge between Playwright and TestRail.

Output

Operation summary with counts Any errors or unmatched cases Link to TestRail run/resultsTestRail Integration

Bidirectional sync between Playwright tests and TestRail test management.

Capabilities

1. Import Test Cases → Generate Playwright Tests

/pw:testrail import --project <id> --suite <id>

Steps: Call testrail_get_cases MCP tool to fetch test cases For each test case:

• Read title, preconditions, steps, expected results
• Map to a Playwright test using appropriate template
• Include TestRail case ID as test annotation: test.info().annotations.push({ type: 'testrail', description: 'C12345' }) Generate test files grouped by section Report: X cases imported, Y tests generated

2. Push Test Results → TestRail

/pw:testrail push --run <id>

Steps: Run Playwright tests with JSON reporter: npx playwright test --reporter=json > test-results.json Parse results: map each test to its TestRail case ID (from annotations) Call testrail_add_result MCP tool for each test:

• Pass → status_id: 1
• Fail → status_id: 5, include error message
• Skip → status_id: 2 Report: X results pushed, Y passed, Z failed

3. Create Test Run

/pw:testrail run --project <id> --name "Sprint 42 Regression"

Steps: Call testrail_add_run MCP tool Include all test case IDs found in Playwright test annotations Return run ID for result pushing

4. Sync Status

/pw:testrail status --project <id>

Steps: Fetch test cases from TestRail Scan local Playwright tests for TestRail annotations Report coverage: TestRail cases: 150 Playwright tests with TestRail IDs: 120 Unlinked TestRail cases: 30 Playwright tests without TestRail IDs: 15

5. Update Test Cases in TestRail

/pw:testrail update --case <id>

Steps: Read the Playwright test for this case ID Extract steps and expected results from test code Call testrail_update_case MCP tool to update steps

MCP Tools Used

Test Annotation Format

All Playwright tests linked to TestRail include:

test('should login successfully', async ({ page }) => { test.info().annotations.push({ type: 'testrail', description: 'C12345', }); // ... test code });

This annotation is the bridge between Playwright and TestRail.

Output

Operation summary with counts Any errors or unmatched cases Link to TestRail run/results