Skip to content

Conversation

MintsInc
Copy link
Member

@MintsInc MintsInc commented Sep 2, 2025

Context

This PR implements the CI workflow modernization for the python client by converting all CI jobs to reusable workflows. This aligns the python client with the established pattern from the Go clients (see PR) and enables centralized CI management for the datadog-api-spec repository.

This work enables:

  • Complete CI reusability: The datadog-api-spec repo can now call a single reusable workflow and get the full CI pipeline (pre-commit, unit tests, examples, integration tests)
  • MergeQueue compatibility: Centralizing all CI jobs in reusable workflows is essential for enabling the MergeQueue functionality
  • Consistent local development: The local test.yml maintains the familiar individual job structure while leveraging reusable components
  • Cross-repository validation: Generated code from datadog-api-spec can be validated with the same comprehensive test suite

Changes

New Files

  • .github/workflows/reusable-ci.yml: Complete CI workflow that orchestrates all validation steps - the single entry point for external repos
  • .github/workflows/reusable-pre-commit.yml: Pre-commit checks with configurable auto-commit behavior and fork handling
  • .github/workflows/reusable-examples.yml: Example code validation workflow

Modified Files

  • .github/workflows/test.yml: Refactored to call individual reusable workflows (maintains familiar structure)
  • .github/workflows/reusable-python-test.yml: Stripped to contain only core testing logic, enhanced with target-branch support

Key Design Decisions

  • Dual architecture: Local repo uses individual job calls for clarity; external repos use single reusable-ci.yml for simplicity
  • Target branch control: Centralized branch checkout logic with fallbacks for cross-repo usage
  • Behavioral preservation: All original CI conditions, triggers, and logic maintained exactly
  • Modular workflows: Each workflow component has single responsibility and can be called independently
  • Status reporting: No reporting on the reusable CI as it is called by the spec PR directly

Tests

This PR maintains 100% functional equivalence with the original CI behavior:

Local Repository (test.yml)

  • Individual job visibility: Each job appears separately in PR status checks as before (pre-commit, test, examples, report)
  • Identical triggers: Same conditions for draft PRs, ci/skip labels, and scheduled runs
  • Auto-commit behavior: Pre-commit fixes are automatically committed as before
  • All original functionality: Multi-version testing (Python 3.8, 3.12), matrix exclusions for platform optimization maintained exactly

Reusable Workflows

  • Modular design: Core testing logic isolated in reusable-python-test.yml
  • Cross-repo compatibility: All workflows can be called from datadog-api-spec with proper secrets
  • Fork handling: Pre-commit workflow handles external contributors correctly
  • Branch flexibility: Supports testing arbitrary branches from external repos

After merging, this will be tested end-to-end by calling the reusable workflows from the datadog-api-spec repository to validate generated client code, ensuring full MergeQueue compatibility.

@MintsInc MintsInc requested review from a team as code owners September 2, 2025 08:53
Copy link
Member Author

MintsInc commented Sep 2, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

jack-edmonds-dd
jack-edmonds-dd previously approved these changes Sep 8, 2025
@MintsInc MintsInc force-pushed the ulysse.mavrocordatos/AAWF-562/make-python-test-workflow-reusable branch 3 times, most recently from da1040b to 18393d4 Compare September 16, 2025 14:19
@MintsInc MintsInc dismissed jack-edmonds-dd’s stale review September 16, 2025 14:21

Big changes since review

@MintsInc MintsInc force-pushed the ulysse.mavrocordatos/AAWF-562/make-python-test-workflow-reusable branch 4 times, most recently from c8b86aa to 58e40f9 Compare September 18, 2025 09:19
@MintsInc MintsInc force-pushed the ulysse.mavrocordatos/AAWF-562/make-python-test-workflow-reusable branch 2 times, most recently from b4a5f40 to a722b00 Compare September 19, 2025 12:58
This PR is part of a larger initiative to standardize CI workflows across all
Datadog API client repositories. By converting our Python testing workflow to a reusable
workflow, we can:

- **Enable centralized CI management**: The datadog-api-spec repo will be able to use
the same testing workflow
- **Prepare for MergeQueue**: Centralizing the CI in the datadog-api-spec repo is necessary to enable the MergeQueue

## Changes

### New Files
- **`.github/workflows/reusable-python-test.yml`**: A reusable workflow that contains the same logic as the original test workflow

### Modified Files
- **`.github/workflows/test.yml`**: Simplified to call the reusable workflow

### Key Design Decisions
- **Behavior preservation**: Every aspect of the original workflow is maintained through input parameters
- **Configurability**: The reusable workflow accepts inputs for all major parameters (Python versions, platforms, matrix exclusions)
- **Environment variables**: Git author information is inherited from the caller workflow
@MintsInc MintsInc force-pushed the ulysse.mavrocordatos/AAWF-562/make-python-test-workflow-reusable branch from a722b00 to 13b1da6 Compare September 19, 2025 13:05
@MintsInc MintsInc merged commit 4041c20 into master Sep 19, 2025
13 checks passed
@MintsInc MintsInc deleted the ulysse.mavrocordatos/AAWF-562/make-python-test-workflow-reusable branch September 19, 2025 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants