Currently, I’m working on my new open source project which hopefully will be announced by the end of October. One thing that I really missed during my work was integrating my GitHub repository with some kind of service which would present me how each Pull Request increases/decreases coverage and what’s the overall coverage on develop and master branch. After few hours I decided to try Codecov since it’s quite easy to setup and has a very clear visualization.
Generating Codecov token
The very first step is generating a token which will be required to send the reports to the Codecov. We need to sign up using our GitHub credentials, allow Codecov to scan our repositories and select one we want to integrate with. After it’s done, you should see the following screen which contains your token:
Generating coverage report in AppVeyor pipeline
Having the token, we need to produce the actual report somehow. I decided to use OpenCover since it’s one of the most popular tools. But here’s one important thing to note. Currently, it supports only Windows OS which unfortunately limits the number of Continuous Integration providers which can be integrated with. Therefore I decided to use AppVeyor to build my project and then send generated reports to Codecov. Below, you can find the YAML file, which does the job:
init: - git config --global core.autocrlf true image: Visual Studio 2015 before_build: - choco install opencover.portable - choco install codecov build_script: - ps: .\build.ps1 - OpenCover.Console.exe -register:user -target:"C:/Program Files/dotnet/dotnet.exe" -targetargs:"test --logger:trx;LogFileName=results.trx /p:DebugType=full <PATH_TO_TESTS_PROJECT>.csproj" -filter:"+[MyApp*]* -[MyApp.Tests*]*" -output:".\my_app_coverage.xml" - codecov -f .\valit_coverage.xml -t $(codecov_token) test: off branches: only: - develop - master cache: - tools -> build.cake - packages -> build.cake
First, notice that before the build I have required dependencies which need to be installed. These are mentioned OpenCover for generating reports and Codecov which will allow sending reports to the server. Then we have the actual build. The first line runs the Cake script (build.ps1) which is responsible for several things like nuget restoring, building, running unit tests etc. What’s interesting is that Cake supports OpenCover, but this extension wasn’t updated for last 6 months and it’s not compatible with the newest version of Cake. Bummer. Therefore I had to put the script directly inside YAML:
- OpenCover.Console.exe -register:user -target:"C:/Program Files/dotnet/dotnet.exe" -targetargs:"test --logger:trx;LogFileName=results.trx /p:DebugType=full <PATH_TO_TESTS_PROJECT>.csproj" -filter:"+[MyApp*]* -[MyApp.Tests*]*" -output:".\my_app_coverage.xml"
This might look complicated but it’s not that bad. Let’s start with the first part:
This is simply using the OpenCover tool inside the „terminal”. Then we have the following construction:
As the documentation says „register asks OpenCover to register the code coverage profiler under HKEY_CURRENT_USER, which doesn’t require the user to have administrative permissions.”. Without this line, the whole generating will fail. Then we’ve got:
This part points the place where we can find dotnet CLI. Keep in mind that in order to generate the coverage, we need to run the tests. How do we do it in .NET Core? We run „dotnet test” command! So, this part calls the dotnet CLI using the full path. Moving to next part:
-targetargs:"test --logger:trx;LogFileName=results.trx /p:DebugType=full <PATH_TO_TESTS_PROJECT>.csproj"
This is the second half of the „dotnet test” command which have some extra parameters. The most important part is „/p:DebugType=full” which will force dotnet CLI to produce PDB files required to generate the coverage file. After this declaration, we have the full path to the project which contains the actual tests. Next, we have:
These are the filters we apply to coverage file. In this particular example, we say we’re interested in generating coverage file from the tests target app, and not from test project itself. Then we have last part:
This one informs what’s the name of the output file and where to save it. At this point, the test coverage file should be generated. The only thing we want to do is to send it to the Codecov server. That’s the responsibility of the third line in build_script:
- codecov -f .\valit_coverage.xml -t $(codecov_token)
Notice that instead of putting Codecov token directly inside the script I used AppVeyor environment variable.
That’s it! Now, each time some new changes will be pushed to the GitHub repository, the AppVeyour will be triggered and new coverage file will be reported to Codecov. I think we should see this in action 😉
Testing Codecov on GitHub
So, I added some unit test to project and created a new pull request on GitHub. After a while, Codecov received the new report from AppVeyor and presented the results in pull request message:
You can see that it presents code coverage both on target (so develop branch) and my feature branch. The green square represents one class that was covered by my unit tests. We can click on it and see more details:
As you see it shows all the lines that have been covered by my pull request changes. It also shows code coverage for this particular class which in this case is 100%. One thing which is worth mentioning is that Codecov automatically adds status checks on your pull request. This means that pull request which decreases code coverage will fail these checks like in the below example:
Besides the reports on PR, you can also put the badge with your result inside README.ms. Simply, go to the Codecov -> Your Project -> Settings -> Badge and copy the markdown. After adding this you should see the badge on GitHub:
Of course, you can customize your reports and integrate Codecov with other applications like Slack. In order to do that create a codecov.yml file and put it into your projects. All available options can be found here.