Unit Testing of Embedded Firmware – Part 4 – Code Coverage with Gcov & LCOV

Uncannier Software December 24, 2018

This article is part 4 of a series. Return to part 1 here: https://uncannier.com/unit-testing-of-embedded-firmware-part-1-software-confucius/

When unit testing, it’s helpful to have some way to measure how much of your code is being tested. In this article I’ll show how to use Gcov and LCOV to produce code coverage reports for embedded firmware that is unit tested on-host.

The techniques shown here apply to any unit test build that is performed using GCC or a variant (e.g. MinGW). I will demonstrate using the Uncannier Thunderboard projects. Hence the example shown here is for x86 GCC under Ubuntu 18.04, with managed make inside Simplicity Studio (a YACE – Yet Another Customized Eclipse). However, the techniques are easily adaptable to any OS, to any IDE, or to makefile projects.

Prerequisites

Gcov is included in Ubuntu by default. LCOV needs to be installed.

sudo apt install lcov

Gcov

For the Uncannier Thunderboard projects, the unit tests are built through the Test build configuration as x86 executables, using x86 GCC. The Test build configuration needs to be altered to enable Gcov instrumentation. It’s of no interest to add Gcov instrumentation to the Debug and Release build configurations because coverage measurement is not needed on target.

C Compiler

We don’t need to enable Gcov instrumentation for the C++ compiler since we only want to measure code coverage for the embedded firmware, and it’s all C. As can be seen from the screenshot, if you’re working with makefiles, the compilation switches you need to add are -ftest-coverage and -fprofile-arcs.

Linker

Somewhat bizarrely, there’s no Gcov option for the linker in Simplicity Studio (nor in Eclipse or any YACE in my experience). The -fprofile-arcs linker switch needs to be added. I’ve done it through the Miscellaneous settings

gcno Files

A rebuild of the Test configuration now produces gcno files along with the object file for each C file. These gcno files are described by GNU as “information to reconstruct the basic block graphs and assign source line numbers to blocks“. I.e. Instrumentation.

gcda Files

With the Gcov profiling information embedded into the compilation output, running of the unit test executable now produces Gcov data files for each C file. These data files are gcda files. They record what lines of code were executed, and how many times

Reports

You can use Gcov to generate reports from the gcno and gcda files, but you only get uninspired coverage reports on the command line. We deserve better!

File '../uncannier/ota_service.c'
 Lines executed:100.00% of 12
 Creating 'ota_service.c.gcov'

LCOV

LCOV is the means by which we can create inspired coverage reports. Coverage reports worthy of showing to management, and worthy of saving as artifacts on continuous integration severs.

I’m not going to go to detail on LCOV usage and options. Dr Google can help you with that. Or you can check my references at the end of this article. Let’s just cut to the chase. I’ve created a postbuild-Test.sh script that is called as a post-build step for the Test build configuration. The comments tell the story.

#!/bin/sh
 
###-----------------------------------------------------------------------------
###
### Post-build step to execute unit tests and produce a coverage report
###
### Copyright (c) Uncannier Software 2018
###
###-----------------------------------------------------------------------------
 
# Set the directories where the coverage report will go
OUTDIR=Test
COVDIR=$OUTDIR/Coverage
 
# Clean any past runs
find . -name '*.gcda' -delete
rm -rf $COVDIR
 
# Somewhere to put the coverage report
mkdir -p $COVDIR
 
# Run the executable to run the tests, and produce the test coverage data
$OUTDIR/${1}
 
echo Generating test coverage report in $COVDIR
 
# Capture the coverage data
lcov -c --quiet --directory $OUTDIR -o $COVDIR/${1}.info
 
# Remove header files data - we don't want to know about coverage of inline functions by Silabs
lcov --quiet --remove $COVDIR/${1}.info "*.h" -o $COVDIR/${1}.info
 
# Now generate the HTML report from the coverage data
genhtml --quiet -o $COVDIR $COVDIR/${1}.info

So my unit tests run as a matter of course for each build, and likewise the coverage report is produced on each build. Compared to Gcov, it’s a thing of qualified beauty. As can be seen, the coverage report can be viewed within Simplicity Studio.

The bad news is the test coverage is very low. The good news is it’s 100% on the Uncannier parts of the project; I’m not so interested in testing Silicon Labs’ code.

Drilling into the report, we can see more detail, such as how many times each line of code was executed.

Warning

Any useful source of information on code coverage will make this warning: code coverage is really only useful for identifying under-tested code. High coverage, even 100%, really tells you next to nothing about the quality of the testing. Don’t let high coverage give you false confidence in a project, and don’t make it your mission to hit 100%.

Conclusion

If you already have your code building with GCC or MinGW, it’s quite trivial to generate code coverage reports using Gcov and LCOV. Do it.

Uncannier Pull Requests

References

Tagged embedded, tdd, thunderboard, unit testing

Uncannier Software