GitHub Actions: Publish Code Coverage Summary to Pull Requests | josh-ops

[utterances-bot]

GitHub Actions: Publish Code Coverage Summary to Pull Requests | josh-ops

Using GitHub Actions to add a code coverage summary report comment to a pull request

https://josh-ops.com/posts/github-code-coverage/

[ardalis]

Note that the irongut/codecoveragesummary only works on linux builds.

[joshjohanning]

@ardalis that is a great point, because it is a Docker action.

I have definitely been preferring non-Docker actions over Docker because they're typically a little faster, and of course run on non-linux machines (and do not require Docker configured on self-hosted machines!).

Perhaps try the 5monkeys/cobertura-action action as an alternative; this is a node-based action so it will run on Windows/Mac.

Here's an example of this in practice.

[ardalis]

Thanks, I'm fine with using an ubuntu agent for most of my repos, I just added the comment to save others some time. Cheers.

[ardalis]

FYI when I run this:
name: Copy Coverage To Predictable Location
run: cp coverage/*/coverage.cobertura.xml coverage/coverage.cobertura.xml

I get:

0s
Run cp coverage//coverage.cobertura.xml coverage/coverage.cobertura.xml
cp coverage//coverage.cobertura.xml coverage/coverage.cobertura.xml
shell: /usr/bin/bash -e {0}
env:
DOTNET_ROOT: /home/runner/.dotnet
cp: target 'coverage/coverage.cobertura.xml' is not a directory
Error: Process completed with exit code 1.

[ardalis]

https://github.com/ardalis/Result/runs/5336761228?check_suite_focus=true

[joshjohanning]

@ardalis thanks for the build link!

Ahh - it looks like you have multiple unit test projects so it's creating two coverage.cobertura.xml files, and that cp command with the wildcard doesn't work when it finds more than 1 file. See your logs:

2022-02-25T17:01:55.0619291Z Attachments:
2022-02-25T17:01:55.0622478Z /home/runner/work/Result/Result/coverage/c34d87ff-fa78-4509-b5e2-347a9d076404/coverage.cobertura.xml
2022-02-25T17:01:55.0674534Z Passed! - Failed: 0, Passed: 8, Skipped: 0, Total: 8, Duration: 52 ms - /home/runner/work/Result/Result/sample/Ardalis.Result.SampleWeb.FunctionalTests/bin/Debug/net6.0/Ardalis.Result.SampleWeb.FunctionalTests.dll (net6.0)
...
2022-02-25T17:01:56.1963720Z Attachments:
2022-02-25T17:01:56.1967088Z /home/runner/work/Result/Result/coverage/0df8280e-d59d-4c5f-8c89-04ca738acb67/coverage.cobertura.xml
2022-02-25T17:01:56.2019135Z Passed! - Failed: 0, Passed: 6, Skipped: 0, Total: 6, Duration: 31 ms - /home/runner/work/Result/Result/sample/Ardalis.Result.Sample.UnitTests/bin/Debug/net6.0/Ardalis.Result.Sample.UnitTests.dll (net6.0)

I believe you would want to use the ReportGenerator action to combine the cobertura files and then upload the combined code coverage result.

You might have to play with it a bit, but something like this:

    - name: Test
      run: dotnet test --no-build --verbosity normal --collect:"XPlat Code Coverage" --logger trx --results-directory coverage
      
    - name: Copy Coverage To Predictable Location
      run: cp coverage/*/coverage.cobertura.xml coverage/coverage.cobertura.xml

    - name: ReportGenerator
      uses: danielpalme/ReportGenerator-GitHub-Action@5.1.4
      with:
        reports: 'coverage/*/coverage.cobertura.xml' # REQUIRED # The coverage reports that should be parsed (separated by semicolon). Globbing is supported.
        targetdir: 'coveragereport' # REQUIRED # The directory where the generated report should be saved.
        reporttypes: 'HtmlInline;Cobertura' # The output formats and scope (separated by semicolon)

    - name: Code Coverage Summary Report
      uses: irongut/CodeCoverageSummary@v1.0.2
      with:
        filename: coveragereport/coverage.cobertura.xml # double check path with targetdir parameter above
        badge: true
        format: 'markdown'
        output: 'both'

Let me know how that works!

[dozer75]

The ReportGenerator section.. It states it is GitHub Action, but that code is Azure DevOps related with variables and syntax that is based on Azure DevOps?

While the report generator syntax is easy to figure out, which way would be best to add it to PR/Job summary?

[joshjohanning]

The ReportGenerator section.. It states it is GitHub Action, but that code is Azure DevOps related with variables and syntax that is based on Azure DevOps?

@dozer75 oh you're right! I totally missed changing those variables when translating that to a run command. Here's what it should be:

    - name: Create code coverage report
      run: |
        dotnet tool install -g dotnet-reportgenerator-globaltool
        reportgenerator -reports:coverage/*/coverage.cobertura.xml -targetdir:CodeCoverage -reporttypes:'Cobertura'

In this post, the ${{ github.workspace }} (alternatively referred to as $GITHUB_WORKSPACE if referencing directly in run command) is implied, assuming otherwise the default working directory hasn't been modified.

Now to generate a markdown report and upload to the GitHub job summary, this is what you would use:

    - name: Create code coverage report
      run: |
        dotnet tool install -g dotnet-reportgenerator-globaltool
        reportgenerator -reports:coverage/*/coverage.cobertura.xml -targetdir:CodeCoverage -reporttypes:'MarkdownSummaryGithub,Cobertura'

    - name: Write to Job Summary
      run: cat CodeCoverage/SummaryGithub.md >> $GITHUB_STEP_SUMMARY

Adding in MarkdownSummaryGithub to the reporttypes parameter.

The ReportGenerator site is really helpful for creating the command 😄

This is a comparison between using the irongut/CodeCoverageSummary action (top) and reportgenerator (bottom):

You can see the entire build workflow here.

If you wanted to post this as a PR comment instead of the job summary, then simply use the marocchino/sticky-pull-request-comment action as shown in the post 😄. Hope this helps!

I updated the post to reflect all of this, and simplified a few other areas as well (like removing the cp step!). 🚀

[dozer75]

Thanks! :)