Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix: enable junit reports #4030

Closed
wants to merge 5 commits into from
Closed

nix: enable junit reports #4030

wants to merge 5 commits into from

Conversation

dougch
Copy link
Contributor

@dougch dougch commented Jun 1, 2023

Resolved issues:

none

Description of changes:

In the nix devshells, we have a newer cmake/ctest that will output junit reports and an html rendered version. This PR adds the junit output flags to ctest when running under nix.

Example test reports:

  1. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_buffered_send_report.html
  2. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_client_authentication_report.html
  3. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_dynamic_record_sizes_report.html
  4. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_early_data_report.html
  5. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_external_psk_report.html
  6. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_fragmentation_report.html
  7. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_happy_path_report.html
  8. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_hello_retry_requests_report.html
  9. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_key_update_report.html
  10. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_npn_report.html
  11. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_ocsp_report.html (shows a failure too)
  12. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_pq_handshake_report.html
  13. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_record_padding_report.html
  14. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_renegotiate_report.html
  15. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_session_resumption_report.html
  16. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_signature_algorithms_report.html
  17. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_sni_match_report.html
  18. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_version_negotiation_report.html
  19. https://d3fqnyekunr9xg.cloudfront.net/dougch/integrationv2_well_known_endpoints_report.html (shows a rerun)

Call-outs:

Brings the nix ctest command in line with our codebuild scripts by adding --no-tests=error --output-on-failure

This doesn't handle publishing the reports; by doing all the data in single html per test, they're quite large, so we'll need to sort out some of the presentation details.

The ability to sort by duration is kinda nifty.

pytest-metadata now knows about CodeBuild

Testing:

How is this change tested (unit tests, fuzz tests, etc.)? locally

Is this a refactor change? no

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions github-actions bot added the s2n-core team label Jun 1, 2023
@dougch dougch force-pushed the nix_junit branch 2 times, most recently from e67f491 to 497585f Compare June 2, 2023 21:35
@dougch dougch changed the title add junit reports to tests under nix nix: enable junit reports Jun 7, 2023
@dougch dougch requested review from goatgoose and lrstewart June 7, 2023 23:07
@dougch dougch marked this pull request as ready for review June 7, 2023 23:10
@dougch dougch marked this pull request as draft June 7, 2023 23:11
@dougch dougch marked this pull request as ready for review June 7, 2023 23:12
@dougch dougch marked this pull request as draft June 27, 2023 21:17
@github-actions
Copy link

This PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Copy link

This PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@dougch dougch requested a review from goatgoose February 6, 2024 00:27
@dougch dougch marked this pull request as ready for review February 6, 2024 00:27
Copy link

This PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@dougch
Copy link
Contributor Author

dougch commented Jun 18, 2024

Use Cloudwatch instead.

@dougch dougch closed this Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants