Skip to content

Conversation

F-WRunTime
Copy link
Member

@F-WRunTime F-WRunTime commented Sep 12, 2025

Overview

This PR adds macOS testing to the pull request workflow to monitor macOS-specific build issues without blocking development. The approach uses continue-on-error: true to collect failure data while maintaining a working CI/CD pipeline.

Changes Made

PR Workflow (test-pr.yml)

  • Added test-macos-build job that runs on macos-15
  • Non-blocking execution: Uses continue-on-error: true to prevent PR failures
  • Comprehensive testing:
    • Homebrew formula validation using brew audit --strict
    • Formula installation from source using brew install --build-from-source
    • Basic functionality testing with kompile for both LLVM and Haskell backends
    • Formula debugging artifacts uploaded for troubleshooting

Release Workflow (release.yml)

  • Made macOS jobs non-blocking: Added continue-on-error: true to both:
    • macos-build job (Build MacOS Package)
    • macos-test job (Test MacOS Package)
  • Prevents release blocking: macOS failures won't prevent releases from completing

Key Benefits

  • Data Collection: Monitor macOS build issues without breaking the pipeline
  • Early Detection: Catch macOS-specific problems in PRs before they reach releases
  • Incremental Fixes: Address issues over time without pressure to fix immediately
  • Working CI/CD: Maintain functional builds for all other platforms
  • Debugging Support: Upload artifacts for troubleshooting failed builds

Current Status

  • PR testing: macOS tests run but don't block PR merges
  • Release testing: macOS builds don't block releases
  • Clean codebase: No complex workarounds or modifications to core build files
  • Monitoring: Can collect data on macOS-specific failures

Remaining TODOs

After further investigation of the collected failure data, the following items need to be addressed:

High Priority

  • Java 17 Compatibility: Resolve Spotless/Google Java Format compatibility issues on macOS
  • Maven Dependencies: Fix dependency resolution problems in Homebrew build environment
  • Build Environment: Address macOS-specific build toolchain issues

Medium Priority

  • Homebrew Formula: Optimize formula for better macOS compatibility
  • Error Handling: Improve error reporting and debugging capabilities
  • Performance: Optimize build times for macOS runners

Future Enhancements

  • CloudRepo Integration: Enable proper Maven artifact publishing for PR testing
  • Staging Repositories: Integration with staging Maven repositories
  • Enhanced Monitoring: Better failure analysis and reporting

Testing Strategy

The macOS testing job will:

  1. Run on all PRs targeting the develop branch
  2. Execute comprehensive Homebrew formula testing
  3. Upload debugging artifacts on failure
  4. Not block PR merges if it fails (due to continue-on-error: true)

Related Issues

  • Addresses macOS build reliability issues in the release workflow
  • Provides foundation for incremental macOS build improvements
  • Establishes monitoring system for macOS-specific problems
  • Complements existing non-blocking macOS testing in release workflow

Next Steps

  1. Monitor macOS test results over several PR cycles
  2. Analyze failure patterns and root causes
  3. Implement targeted fixes based on collected data
  4. Gradually improve macOS build reliability
  5. Eventually remove continue-on-error when issues are resolved

@F-WRunTime F-WRunTime requested a review from a team as a code owner September 12, 2025 05:31
@F-WRunTime F-WRunTime self-assigned this Sep 18, 2025
Copy link
Contributor

@palinatolmach palinatolmach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm other than the failing job, but it'd also be good for @Robertorosmaninho to look through the diffs. @F-WRunTime could you please document the Spotless changes, and clarify why we're skipping it in the Test PR workflow?

@F-WRunTime F-WRunTime force-pushed the feature/pr-macos-testing branch from 4dccb41 to 3ea08e8 Compare October 6, 2025 21:31
- Add macOS Build & Test job to test-pr.yml with continue-on-error: true
- Update release.yml to make macOS build and test jobs non-blocking
- Keep macOS tests in place to monitor failures without blocking PRs/releases
- Clean approach: test the failures without breaking the build pipeline

This allows us to:
- Monitor macOS build issues without blocking development
- Collect data on macOS-specific failures
- Fix issues incrementally without pressure
- Maintain working CI/CD pipeline for other platforms
@F-WRunTime F-WRunTime force-pushed the feature/pr-macos-testing branch from 3ea08e8 to ecf07c2 Compare October 6, 2025 22:30
F-WRunTime added a commit that referenced this pull request Oct 7, 2025
- Pull Changes from PR 4857
- Specify java 17 for macos teting
@F-WRunTime F-WRunTime merged commit ec6b3c2 into develop Oct 7, 2025
18 of 19 checks passed
@F-WRunTime F-WRunTime deleted the feature/pr-macos-testing branch October 7, 2025 19:05
F-WRunTime added a commit that referenced this pull request Oct 8, 2025
Resolving TODO items from PR #4857

---------

Co-authored-by: Roberto Rosmaninho <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants