TestZeus Execute · Composite GitHub Action

Generate detailed CTRF reports from your TestZeus test runs directly from a GitHub Workflow.

The action generates a CTRF report that will be saved to the specified filename (defaults to ctrf-report.json) after execution completes.

---

📥 Inputs

Name	Description	Required	Default
test_ids_file	Path to a JSON file that lists the test IDs to execute.	Yes	-
email	TestZeus account email used to authenticate via the CLI.	Yes	-
password	TestZeus account password used to authenticate via the CLI.	Yes	-
execution_mode	Execution mode for the test run ("lenient" or "strict").	No	"lenient"
name	Name for the test run group.	No	"Smoke action suite"
filename	Filename for the generated CTRF report.	No	"ctrf-report.json"

---

🗂️ Test IDs JSON Format

Your test_ids_file must follow this shape:

{
  "test_ids": [
    "TC-101",
    "TC-102",
    "TC-103"
  ]
}

• test_ids — array of TestZeus test IDs.

The action validates the file and will fail fast if the structure is invalid.

---

🚀 Quick Start

First Add pyproject.toml file using poetry init --no-interaction to your repo then replace the python version to ">=3.11,<3.14" using sed -i "s/>=3.11/>=3.11,<3.14/" pyproject.toml

name: "Run TestZeus Suite"

on: [workflow_dispatch]

jobs:
  testzeus:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Run image
        uses: abatilo/actions-poetry@v3

      - name: Install dependencies
        run: |
          poetry install --no-root

      - name: Execute TestZeus Tests
        uses: test-zeus-ai/testzeus-execute@v1
        with:
          test_ids_file: "test_ids.json"
          email: ${{ secrets.TESTZEUS_EMAIL }}
          password: ${{ secrets.TESTZEUS_PASSWORD }}
          # execution_mode: "strict"    # optional, defaults to "lenient"
          # name: "My Test Suite"       # optional, defaults to "Smoke action suite"
          # filename: "my-report.json"  # optional, defaults to "ctrf-report.json"

      # Publish a beautiful GitHub summary using CTRF-io
      - name: Publish Test Report
        uses: ctrf-io/github-test-reporter@v1
        with:
          report-path: "ctrf-report.json"  # Use the same filename as specified in the action
          template-path: "templates/testzeus-report.hbs"
          custom-report: true
          artifact-name: 'ctrf_report'
          upload-artifact: true
        if: always()

---

📝 Handlebars Template (templates/testzeus-report.hbs)

Copy the template below into your repo (for example inside templates/testzeus-report.hbs) or create your own custom template.
Then reference the file in your workflow step with the template-path input (see the Publish Test Report example above).

To create you own custom template then refer the following repos:

• build custom CTRF template using ctrf-io repo
• Learn how to write handlebar for github actions

# 🧪 Test Results Summary

| **Tests** | **Passed** | **Failed** | **Skipped** | **Other** | **Flaky** | **Duration** |
|----------|------------|------------|-------------|-----------|-----------|--------------|
| {{ctrf.summary.tests}} | {{ctrf.summary.passed}} | {{ctrf.summary.failed}} | {{add ctrf.summary.skipped ctrf.summary.pending}} | {{ctrf.summary.other}} | {{countFlaky ctrf.tests}} | {{formatDuration ctrf.summary.start ctrf.summary.stop}} |

---

## 📊 Overview
- ✅ Passed: {{ctrf.summary.passed}} / {{ctrf.summary.tests}}
- ❌ Failed: {{ctrf.summary.failed}}

---

## ⚙️ Execution Details
{{#if ctrf.tool.name}}![tool](https://ctrf.io/assets/github/tools.svg) **Tool**: {{ctrf.tool.name}}{{/if}}  
🔍 **Branch**: `{{github.branchName}}`  
👤 **Triggered by**: `{{github.actor}}`

---

{{#if ctrf.summary.failed}}
## ❌ Failed Tests

{{#each ctrf.tests}}
  {{#if (eq this.status "fail")}}
  ### 🔴 {{this.extra.feature_name}} - {{this.extra.scenario_name}}
  - ⏱️ Duration: {{formatDurationMs this.duration}}
  - 🔗 TestZeus Run: [View](https://prod.testzeus.app/test-runs/{{this.extra.test_run_id}})
  {{#if this.extra.test_data_id}}
  - 🧾 Test Data: [View](https://prod.testzeus.app/test-data/{{this.extra.test_data_id.[0]}})
  {{/if}}

  {{#if (getCollapseLargeReports)}}
  <details>
    <summary><strong>View Steps</strong></summary>

    {{#each this.steps}}
    - {{#if (eq this.status "fail")}}❌{{else if (eq this.status "pass")}}✅{{/if}} {{this.name}}
    {{/each}}

  </details>
  {{else}}
  - **Steps**:
    {{#each this.steps}}
    - {{#if (eq this.status "fail")}}❌{{else if (eq this.status "pass")}}✅{{/if}} {{this.name}}
  {{/each}}
  {{/if}}

  {{/if}}
{{/each}}

{{/if}}

---

🔔 Slack Notification (Optional)

Add a Slack message once the tests finish:

- name: Notify Slack
  uses: slackapi/slack-github-action@v1
  with:
    payload: |
      {
        "text": "🧪 TestZeus run finished – see details: ${{ github.run_url }}"
      }
  env:
    SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
  if: always()

---

📦 Upload Artifacts (Optional)

- name: Upload CTRF report
  uses: actions/upload-artifact@v4
  with:
    name: ctrf-report
    path: ctrf-report.json  # Use the same filename as specified in the action

---

CTRF Schema

The generated CTRF report follows the Common Test Report Format (CTRF) v1.0.0 specification. The schema includes:

• Report metadata: Format version, specification version, and tool information
• Test summary: Aggregate counts (total, passed, failed, pending, skipped, other) and execution timing
• Individual test results: Each test includes:
  • Test identification (name, status, duration, timing)
  • Thread/execution context information
  • File attachments (screenshots, logs, artifacts)
  • Step-by-step execution details with individual step status
  • Extended metadata (tenant IDs, test run identifiers, feature/scenario names)

This standardized format ensures compatibility with CTRF-compliant tools and enables consistent test reporting across different testing frameworks.

Schema Example

{
  "reportFormat": "CTRF",
  "specVersion": "1.0.0",
  "results": {
    "tool": {
      "name": "testzeus",
      "version": "1.0.0"
    },
    "summary": {
      "tests": 5,
      "passed": 4,
      "failed": 1,
      "start": 1640995200,
      "stop": 1640995800
    },
    "tests": [
      {
        "name": "Login Test",
        "status": "pass",
        "duration": 2500,
        "steps": [
          {
            "name": "Enter credentials",
            "status": "pass"
          }
        ],
        "attachments": [
          {
            "name": "image.png",
            "contentType": "png",
            "path": "<path/to/image>.png"
          }
        ],
        "extra": {
          "tenantid": "abcd",
          "test_run_id": "abcd",
          "test_run_dash_id": "abcd",
          "agent_config_id": "abcd",
          "feature_name": "Authentication",
          "scenario_name": "Login to google"
        }
      }
    ]
  }
}

---

ℹ️ Tips

• Keep your test_ids_file under source control so reviewers can see exactly what is being executed.
• Store credentials (TESTZEUS_EMAIL, TESTZEUS_PASSWORD) as encrypted repository secrets.
• Use the execution_mode parameter to control test execution behavior ("lenient" or "strict").
• Customize the name parameter to give meaningful names to your test runs, or let it default to "Smoke action suite".
• Pair this action with schedule triggers to run nightly regression suites.
• The action installs dependencies with Poetry and invokes the CLI with poetry run.
  If you use a self-hosted runner, make sure Poetry is available on the machine (GitHub-hosted runners already include it).

Happy testing! 🚀