Testing

To ensure high code quility each release is tested extensively by many test suites. The test suites are executed both on native Linux and on many of the supported boards. See Test suites for a list of all test suites that are executed before each release.

The native Linux test suites are executed automatically on each commit.

Test result: https://travis-ci.org/eerimoq/simba

Code coverage: https://codecov.io/gh/eerimoq/simba

Unit tests

Each module shall have unit tests to verify that the implementation works as expected and that future refactoring does not break legacy.

All unit tests except low level drivers and networking are hardware independent. This makes it possible to use common Linux tools (gcov, valgrind, gdb, etc.) to debug and gather statistics of a module, which is very useful.

For low level drivers where the majority of the code is hardware specific (ports folder), testing on real hardware is important. It’s preferable to have a hardware independent test suite with stubbed interfaces for drivers without any port specific code, and having an example application in examples to test on real hardware.

All unit tests are found in the tst folder.

Hardware setup

Below is a picture of all supported boards connected to a USB hub. The USB hub is connected to a linux PC (not in the picture) that executes test suites on all boards.

A short description of the setup:

  • The DS3231 device (on the breadboard to the left) is connected over i2c to the Arduino Mega.
  • CAN0 is connected to CAN1 on the Arduino Due. The CAN driver is tested by sending frames between the two CAN devices.
  • The UART of the STM32VLDISCOVERY board is connected to a serial to USB adaptor. DTR on the adaptor is used to reset the board.
  • The ESP-12E Development Board also has a serial to USB adaptor connected. RTS is used to set the board in flashing mode (GPIO0) and DTR is used to reset the board (REST).

Test suites

Below is a list of all test suites that are executed before every release. They are listed per board.

Arduino Uno

ESP-01

ESP-12E Development Board

ESP32-DevKitC

Adafruit HUZZAH ESP8266 breakout

Maple-ESP32

nRF52840-PDK

STM32F3DISCOVERY

WEMOS D1 mini