This article is part 2 of a series. Read part 1 here: https://uncannier.com/unit-testing-of-embedded-firmware-part-1-software-confucius/

In this article I’ll cover the hardest part: getting the firmware building on-host with an x86 toolchain. The goal is to perform Test Driven Development (TDD) and unit testing, on-host, not on-target.

First we’ll setup the tools, then we’ll do the code. I’ll use the Uncannier Thunderboard projects as an example. This article therefore concentrates on an ARM32 system within Simplicity Studio (YACE – Yet Another Customized Eclipse), using GCC under Ubuntu 18.04. I’ll use CppUTest as the unit test harness, but I could have just as easily used CppUnit, Unity, Google Test etc.

I have done this in the past for many embedded systems and toolchains; ARM32, 8051, ATmega328p, CodeWarrior, Atollic TrueSTUDIO, Keil uVision, GCC, MinGW, Atollic PC Tools, Windows, Ubuntu and more. The process I follow here is adaptable to projects for different targets, different IDEs, different toolchains, different unit test harnesses and different development host operating systems.

Strategy

Before covering the mechanics, let’s cover the strategy. With Test Driven Development (TDD) and unit testing, you want to test your code. The embedded application. You don’t want to test third-party code. So the aim is to isolate the application code, and surround it with a harness, tests, fakes and mocks so that you can exercise it thoroughly.

Per the diagram, the application becomes part of an x86 executable, instead of part of an embedded binary. The APIs of the OS, the vendor device drivers and the third party libraries constitute a test “seam”. You have to replace these components with test doubles: stubs, fakes and mocks that let you inject input vectors into the application, and intercept output vectors from the application. The unit test harness provides a mocking framework that allows the test cases to communicate with the test doubles below the seam.

The creation of the test seam is the most burdensome part of setting up a unit test environment. Although I’m only interested in unit testing the Uncannier parts of the Uncannier Thunderboard application, I still take the time in this article to get the entire application into the harness.

Prerequisites

You need to install some tools to make all this magic happen. I’m doing this development within Ubuntu 18.04 LTS, so the steps I give are appropriate for that.

If you’re doing this under some other OS, you’ll need to adapt the steps. I have been through this unit test process many times in the past with MinGW under Windows, so I can vouch for it. I don’t recommend it though. If you’re on a team, it’s not easy to get other developers to come with you on this journey if they have to install all the tools and re-do all this configuration. Do yourself a favour, go Linux, and share a VM or Docker image with your team.

CppUTest

You could get the ZIP from https://cpputest.github.io/ and build from source. However Ubuntu 18.04 LTS already packages the most recent release (3.8), so I’ll take the path of least resistance. I recommend the 32-bit edition, for reasons that will soon become clear.

dpkg --add-architecture i386
sudo apt update
sudo apt install libcpputest-dev:i386

g++

As the name suggests, the CppUTest harness is C++. If you’re a C-only developer, fear not. CppUTest is designed with minimal exposure to C++, wrapping most of the complexity in easy-to-use macros. Nonetheless, you’ll need a C++ toolchain to be able to build the unit tests and link against CppUTest.

sudo apt install g++

gcc Multilib

Here’s a RED HOT TIP: building and running your ARM32 embedded code with an x86 toolchain will generally be easier if you do 32-bit builds rather than 64-bit. That way, your integer size matches your pointer size, as it does on the ARM32 embedded target. Thus you should install GCC Multilib to allow cross-compilation to 32-bit.

sudo apt-get install gcc-multilib g++-multilib

Of course, if your embedded application is super portable, as it probably should be, you won’t have any code that tries to convert between integers and pointers. Then this won’t matter. In the real world though, some clown (maybe even you) on your project will make code that does such a conversion. Then this RED HOT TIP is a time saver.

I focus here on ARM32 because it’s relevant to my example. And also because I will be retired, and probably dead, before ARM loses its market stranglehold. If you’re working with a target that isn’t 32-bit, you will have a little more work to do.

Simplicity Studio

We could use basically any tool for the unit test development. However, if we need Simplicity Studio to build the embedded binaries, it sure makes a lot of sense to do the unit testing within the same IDE.

Convert The Project To C++

Click on the project. Then from the Simplicity Studio menu, select Project->Convert->Convert MCU Project to C or C++… and convert the project to C++. This adds C++ nature to your project and exposes C++ toolchain settings in the project properties.

The conversion also causes Use library file circular dependency to become unchecked in the GNU ARM C++ Linker settings for both the Debug and Release build configurations. This needs to be re-checked.

Create A New Build Configuration

Simplicity Studio is all about Managed Make builds, rather than Makefile builds. Generally I prefer Makefiles, but I think it would be “Fighting The Fed” in this case.

Create a Test build configuration by cloning the Release build configuration. By cloning, the Test build configuration will inherit all of the embedded system includes paths. This is what you want; to compile against all of the production code includes, except for those you choose to stub/replace when wrangling the code onto an x86 environment.

Add The Linux GCC Toolchain

Silicon Labs have tried to lock Simplicity Studio down such that users can’t go wrong. They want you to create AppBuilder projects for the embedded target, and that’s it. So you have to apply some Eclipse Jedi mind tricks to unlock the Eclipse CDT x86 capabilities that Silicon Labs have deliberately nobbled.

Window->Preferences->General->Capabilities and enable CDT – C/C++ Managed Builder UI

This makes the Tool Chain Editor available under the project properties. Thus it becomes possible to select the x86 toolchains.

As I’m developing under Ubuntu 18.04 LTS, I’ll use Linux GCC and Gnu Make Builder. This of course only changes the tools for the Test build configuration. Debug and Release continue to use Si32 GNU ARM and Si32 GNU ARM Builder.

Exclude Third Party Code

Now you need to exclude those source files that are below the test seam. In this case it’s the hardware and platform directories, to exclude the Ember libraries, Ember drivers, CMSIS, the BSP and a few other bits. This will become clearer when you create the test seam.

Right click each->Resource Configurations->Exclude From Build…

This will give the following pop-up. Naturally, these resources should only be excluded from the Test build configuration.

Create A Unit Test Source Directory

You need a directory to hold the unit tests, and also the associated stubs. In my case, I’ve created a directory called utest. This directory needs to be excluded from the Debug and Release build configurations, so that the code is only built for the Test configuration.

Right click utest->Resource Configurations->Exclude From Build…

The utest directory needs to be populated with a main.cpp so that the unit test executable can be built and linked. The CppUTest harness requires main() to call the RUN_ALL_TESTS() function-like macro.

//-----------------------------------------------------------------------------
///
/// @file main.cpp
///
/// @brief Test runner for the Thunderboard test suite
///
/// @copyright Copyright (c) Uncannier Software 2018
///
///-----------------------------------------------------------------------------
 
#include <iostream>
#include <CppUTest/TestHarness.h>
#include <CppUTest/CommandLineTestRunner.h>
#include <CppUTest/TestRegistry.h>
#include <CppUTestExt/MockSupportPlugin.h>
 
int main( int argc, char** argv )
{
    std::cout << "Thunderboard Test" << std::endl; 
 
    // Install a mock plugin to save us a lot of boilerplate in the test cases 
    MockSupportPlugin mockPlugin;
    TestRegistry::getCurrentRegistry()->installPlugin( &mockPlugin );
 
    return RUN_ALL_TESTS( argc, argv );
}

Exclude main.c From The Test Configuration

You can’t have two mains in the build of the Test configuration. Exclude the embedded application main.c from the Test configuration.

Right click each->Resource Configurations->Exclude From Build…

Include Paths

The Test configuration needs some additional include paths.

The C++ compiler needs a path to the CppUTest includes.

And the C compiler needs a path to any include files you might create as part of the stubs. This should be the first include path so that the build will choose stub include files first. This is necessary if you seek to replace any production includes to create the test seam.

Link CppUTest

Obviously the Test configuration needs to link the CppUTest harness.

32-bit Build

To achieve a 32-bit build, it’s necessary to add the -m32 option to both compilers and the linker. This invokes the Multilib.

Other Settings

Some other settings to consider:

• Dialects: I selected ISO C++11 and ISO C99.
• Preprocessor/Symbols: I deleted all the stack and heap settings, but I retained the processor selection (EFR32MG12P332F1024GL125) so that the production includes resolve correctly.
• Optimization: -O0 since I’m only interested in running this application for debugging and testing purposes.
• Warnings: I turned on -Wextra.
• Miscellaneous: Selective disable of warnings, specifically -Wno-format and -Wno-unused-parameter.

Test Seam

Now we get down to the nitty gritty of the test seam. This is primarily about code, not about tools.

If you attempt to build the Test configuration at this moment, everything should compile. The linker will complain about lots of undefined functions – these more or less tell you the functions that will comprise your seam. That is, all the functions that need to be stubbed, faked or mocked. I won’t lie; it will take you a few hours.

In the case of the Uncannier Thunderboard projects, it’s pretty clear that the seam is comprised primarily of:

• The Ember device driver and library APIs.
• The hardware kit BSP and drivers.
• The Blue Gecko stack library functions.

As CppUTest is C++, I recommended stubbing, faking and mocking in C++. That is, all C functions you replace should be replaced by C++ functions. This allows you to most easily use the mocking framework supplied by CppUTest.

I also recommend you don’t create mocks for the entire seam up front. Do the simplest stubs and fakes you can get away with to get the program to link. Only implement mocks (and sophisticated fakes) for those functions you need for the unit test cases you have written. Do the least you need to do. Expand the number of mocks as the number of test cases grow.

Ember Library

The Ember library is straightforward to stub, fake or mock. As my initial test cases will seek only to test ota_service.c, there are no Ember functions that need to be mocked. They can be minimally stubbed or faked, such as em_emu.cpp below.

All of the Ember library header file have the extern “C” wrappers around the function prototypes, thus all stubs or fakes will be linked as C functions even though they are defined in C++ source files. This is exactly what you want.

///-----------------------------------------------------------------------------
///
/// @file em_emu.cpp
///
/// @brief Stubs for platform/emlib/src/em_emu.c
///
/// @copyright Copyright (c) Uncannier Software 2018
///
///-----------------------------------------------------------------------------
 
#include <cstdint>
#include <em_emu.h>
 
 
bool EMU_DCDCInit( const EMU_DCDCInit_TypeDef *dcdcInit )
{
    return true;
}

Hardware BSP

As with the Ember library, I have no desire to mock any hardware BSP functions yet. Thus they can be minimally stubbed or faked.

Unlike the Ember library, the hardware BSP headers do not wrap their function prototypes in extern “C” statements. This must done in the CPP files to ensure correct linkage. Here’s hall_si7210.cpp as an example.

///-----------------------------------------------------------------------------
///
/// @file hall_si7210.cpp
///
/// @brief Stubs for hardware/kit/common/bsp/thunderboard/hall_si7210.c
///
/// @copyright Copyright (c) Uncannier Software 2018
///
///-----------------------------------------------------------------------------
 
#include <cstdint>
 
extern "C"
{
#include <hall.h>
}
 
 
uint32_t HALL_measure( uint32_t scale, float *result )
{
    *result = 0;
 
    return HALL_OK;
}
 
float HALL_getTamperLevel( void )
{
    return 0;
}

Blue Gecko Library

The Bluetooth library is the most problematic part of the test seam. This is because Silicon Labs have made heavy use of inline functions to implement the API. The native_gecko.h file is full of inline definitions like this:

static inline void* gecko_cmd_system_reset(uint8 dfu)
{
    struct gecko_cmd_packet *gecko_cmd_msg = (struct gecko_cmd_packet *)gecko_cmd_msg_buf;
 
    gecko_cmd_msg->data.cmd_system_reset.dfu=dfu;
    gecko_cmd_msg->header=((gecko_cmd_system_reset_id+((1)<<8)));
    sli_bt_cmd_handler_delegate(gecko_cmd_msg->header, sli_bt_cmd_system_reset, &gecko_cmd_msg->data.payload);
    return 0;
}

I could choose to implement the seam by faking and mocking the sli_bt_cmd_handler_delegate() calls (and other sli_bt_foobar() calls). However, this would mean that the test cases would have to parse the gecko_cmd_packet structures to figure out if the Code Under Test made correct Bluetooth library calls or not. Yuck.

Essentially there is no value in mocking at the sli_bt_foobar() level. The application code never makes sli_bt_foobar() calls directly. It only makes gecko_cmd_foobar() calls. I only want to know if these calls are correctly made.

Thus I make the design decision that my seam will replace native_gecko.h entirely with my own native_gecko.h. This is a common approach when you want to create seams at inline functions.

I create my own native_gecko.h by copying the existing one, then converting all of the inline functions to inline function prototypes using a regular expression search and replace.

Then use another search and replace to obliterate all instances of static inline. This then leaves a native_gecko.h that merely declares gecko_cmd_foobar() function prototypes. A native_gecko.cpp can now be created to define and implement gecko_cmd_foobar() function fakes and mocks. Here are the mocks needed to implement unit tests for ota_service.c.

void* gecko_cmd_system_reset( uint8 dfu )
{
    mock().actualCall( "gecko_cmd_system_reset()" ).withParameter( "dfu", (int)dfu );
 
    return 0;
}
 
struct gecko_msg_endpoint_close_rsp_t* gecko_cmd_endpoint_close( uint8 endpoint )
{
    mock().actualCall( "gecko_cmd_endpoint_close()" ).withParameter( "endpoint", endpoint );
 
    return &gecko_rsp_msg->data.rsp_endpoint_close;
}
 
struct gecko_msg_gatt_server_send_user_write_response_rsp_t* gecko_cmd_gatt_server_send_user_write_response( uint8 connection, uint16 characteristic,
                                                                                                             uint8 att_errorcode )
{
    mock().actualCall( "gecko_cmd_gatt_server_send_user_write_response()" )
            .withParameter( "connection", connection )
            .withParameter( "characteristic", characteristic )
            .withParameter( "att_errorcode", att_errorcode );
 
    return &gecko_rsp_msg->data.rsp_gatt_server_send_user_write_response;
}

Unit Test Cases

With the seam built, all that’s left to do is to create test cases in ota_service_test.cpp, and execute them. Here’s a minimal set of test cases for ota_service.cpp, making use of the mocks created in native_gecko.cpp.

TEST( ota_service, HandleServiceControlWrite )
{
    mock().expectOneCall( "gecko_cmd_gatt_server_send_user_write_response()" )
            .withParameter( "connection", conGetConnectionId() )
            .withParameter( "characteristic", gattdb_ota_control )
            .withParameter( "att_errorcode", bg_err_success );
    mock().expectOneCall( "gecko_cmd_endpoint_close()" ).withParameter( "endpoint", conGetConnectionId() );
 
    otaServiceControlWrite( NULL );
}
 
TEST( ota_service, EnterOtaDfuMode )
{
    mock().expectOneCall( "gecko_cmd_system_reset()" ).withParameter( "dfu", 2 );
    mock().ignoreOtherCalls();
 
    otaServiceControlWrite( NULL );
    otaServiceConnectionClosed();
}
 
TEST( ota_service, DoNotEnterOtaDfuModeOnRegularClose )
{
    // The harness will complain if any mocks are called (since no expectations have been set)
    otaServiceConnectionClosed();
}

And executing a humble total of 3 unit test cases on the command line:

Success!

Pitfalls

Inline Functions

As shown with native_gecko.h, inline functions in header files can be problematic. To stub, fake or mock those, it’s necessary to create a replacement header file, not just replacement functions.

Evolving Seam

As you add more tests, more of the application will be executed. This will sometimes introduce code that can’t be executed on the host, even though it has been building fine. Thus, in adding new test cases, it’s not uncommon for the x86 executable to start crashing.

One example would be the battMeasure() function. If I were to write a unit test case that makes a call to that, the executable will suffer a segmentation fault. This is because of the battMeasure()-> appHwBatteryLevel()-> measureBatteryVoltage()-> measureOneAdcSample()-> ADC_IntGet() call chain. The ADC_IntGet() function is an inline function defined in em_adc.h, and it attempts to access the IF register offset from the ADCo register address. This is an invalid address on the host.

__STATIC_INLINE uint32_t ADC_IntGet(ADC_TypeDef *adc)
{
  return adc->IF;
}

Thus the seam would need to be evolved. Probably by replacing em_adc.h or perhaps by replacing the efr32mg12p332f1024gl125.h file that defines ADC0.

Hard Dependencies

As you just saw, it would be helpful if the application were developed in such a way that it doesn’t have tightly coupled hardware dependencies. e.g. If you set a pointer to a fixed address to access a register, that’s not going to be unit testable. The address needs to be a dependency that can be injected at run-time, or the hard-coding of the address needs to be stubbed out, and hence be below the test seam. The dependency injection approach is better, if you have control over it.

C/C++ Linkage

Per the discussion on the Hardware BSP header files, care needs to be taken that correct linkage is specified when mixing C and C++ source. “Undefined functions” from the linker, due to C++ name mangling, will result otherwise.

Compiler Extensions

Compiler extensions, such as those used to mark functions as ISRs, can give you a headache. However, clever and judicious use of the GCC pre-processor can usually spirit the extensions away, converting your code to seemingly regular and portable C code.

Threading

In systems using an RTOS, you will likely end up having to do non-trivial work to fake the OS calls. If your application uses active objects, you will have private threads in your modules, and thus task functions that are not callable from unit test cases. In this situation, your fakes will need to capture the task function pointer from the thread creation, and offer a backdoor API for you to call the task function. You should use strategies like that, rather than take the soft option of making otherwise-private functions public.

You could start doing horrible things, like including C source files into the CPP test files, but that is, like I just said, horrible.

Conclusion

I warned at the start of the article that wrangling your embedded code into an x86 build is the hardest part of setting up unit testing. It’s certainly non-trivial. However, if you make this investment early in your project, the effort is smaller, and the productivity and quality pay off will be bigger.

Uncannier Pull Requests

• https://github.com/gregbreen/uncannier-thunderboard-sense2/pull/11
• https://github.com/gregbreen/uncannier-thunderboard-react/pull/11

References

• My brain, shaped by years of bitter experience.
• https://www.amazon.com/Driven-Development-Embedded-Pragmatic-Programmers/dp/193435662X