Skip to content

[no-relnote] Enable CodeCov #958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ArangoGutierrez
Copy link
Collaborator

This patch enables CodeCov to track or Go unit test coverage

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This PR enables CodeCov integration to track Go unit test coverage. It adds a new CodeCov upload step under the "test" job and makes minor adjustments to the GitHub Actions workflow configuration.

  • Added CodeCov action in the "test" job.
  • Configured environment variables for Go version extraction.
  • Maintained setup consistency across testing and build steps.

Reviewed Changes

File Description
.github/workflows/golang.yaml Added CodeCov step for coverage reporting and adjusted version setup

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

.github/workflows/golang.yaml:96

  • The variable substitution operator differs from the one in the test job (':='). Consider using the consistent syntax ':=' for extracting GOLANG_VERSION unless the differing behavior is intentional.
echo "GOLANG_VERSION=${GOLANG_VERSION##GOLANG_VERSION ?= }" >> $GITHUB_ENV

@codecov-commenter
Copy link

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@ArangoGutierrez ArangoGutierrez marked this pull request as draft March 4, 2025 11:21
@ArangoGutierrez
Copy link
Collaborator Author

umm let's leave this for later on, we might need to move Go actions to hosted runners if we want to pass the secret during PR

- name: Install Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GOLANG_VERSION }}

- run: make test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change this to make coverage instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it actually makes sense from the perspective of a Makefile file functionality; make coverage would actually generate an output artifact. So I am not against it, I'll leave it as your call

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need the coverage.out artifact?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it actually makes sense from the perspective of a Makefile file functionality; make coverage would actually generate an output artifact. So I am not against it, I'll leave it as your call

Sorry, I don't understand what you're saying here. We have two targets:

  • make test
  • make coverage

The first generates coverage.out and the second filters out coverage related to mocks to generate a .no-mocks file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I was not aware of the make coverage but for codecov we need the output file from make test

with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.out
fail_ci_if_error: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to block CI on an external system failing?

@elezar
Copy link
Member

elezar commented Mar 6, 2025

umm let's leave this for later on, we might need to move Go actions to hosted runners if we want to pass the secret during PR

That's not true. Secrets are not available on PRs from forks, but should be available from the pull-request branches. (or is this not essentially the same issue as publishing images). Another note, we only need to run the "upload to codecove" step in the internal runners consuming the coverage file generated in a previous step.

@ArangoGutierrez
Copy link
Collaborator Author

umm let's leave this for later on, we might need to move Go actions to hosted runners if we want to pass the secret during PR

That's not true. Secrets are not available on PRs from forks, but should be available from the pull-request branches. (or is this not essentially the same issue as publishing images). Another note, we only need to run the "upload to codecove" step in the internal runners consuming the coverage file generated in a previous step.

ture, I forgot about copy-pr-bot I was thinking in the old way o passing secrets. still will retake this PR after we cut the release

@ArangoGutierrez ArangoGutierrez marked this pull request as ready for review March 6, 2025 14:28
@ArangoGutierrez
Copy link
Collaborator Author

/ok to test

Signed-off-by: Carlos Eduardo Arango Gutierrez <[email protected]>
@ArangoGutierrez
Copy link
Collaborator Author

Here's a look of how we would be able to visualize the results

https://app.codecov.io/gh/NVIDIA/nvidia-container-toolkit

image

@ArangoGutierrez ArangoGutierrez requested a review from elezar March 6, 2025 14:40
@ArangoGutierrez
Copy link
Collaborator Author

@tariq1890 if you like this, we could replace the coversall implementation on the other repos

@elezar
Copy link
Member

elezar commented Mar 7, 2025

@ArangoGutierrez as a general question, how does CodeCov compare to coversall that @tariq1890 was using? I was under the impression that CodeCov is a paid service.

@ArangoGutierrez
Copy link
Collaborator Author

@ArangoGutierrez as a general question, how does CodeCov compare to coversall that @tariq1890 was using? I was under the impression that CodeCov is a paid service.

It is free for OSS repos.

@ArangoGutierrez
Copy link
Collaborator Author

@elezar, any thoughts on this?

@elezar
Copy link
Member

elezar commented Apr 2, 2025

@elezar, any thoughts on this?

Please confirm whether this can be run from a PR through the copy bot? You also have not answered by question as to how this compares to coversall (or other tools).

@ArangoGutierrez
Copy link
Collaborator Author

coversall

Here's a detailed comparison

Comparison: Coveralls vs. Codecov (GitHub Actions – Code Coverage)

Feature Coveralls Codecov
GitHub Integration Native support via GitHub Actions, simple setup Native GitHub Actions support, rich status checks
Language Support Broad (C, C++, Go, Python, etc.) Very broad (Go, Python, JVM, Swift, LLVM, etc.)
Coverage Format Support LCOV, Cobertura, SimpleCov LCOV, Cobertura, Jacoco, GCOV, LLVM, etc.
UI & Reporting Basic UI, focuses on coverage delta Advanced UI: trends, per-line coverage, PR annotations
Monorepo Support Limited Strong monorepo support via flags and path-based filters
Test File Upload GitHub Action or manual upload Robust CLI and GitHub Action
Security & Compliance Basic OAuth / GitHub App SOC2, audit logs, role-based permissions (better enterprise fit)
Free for Open Source ✅ Yes ✅ Yes
Private Repo Pricing Limited free tier, paid plans available Free tier for small teams, paid plans for more features

Summary:

  • Coveralls: Good for simple projects needing straightforward coverage tracking.
  • Codecov: Better suited for complex or large codebases (e.g., monorepos), with more detailed insights and security options.

References:

@ArangoGutierrez ArangoGutierrez self-assigned this Apr 10, 2025
@ArangoGutierrez
Copy link
Collaborator Author

@elezar PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants