7.1. harness — Test harness

In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the test execution engine and the test script repository.

This module implements the test execution engine.

The test scripts are part of the build system.

7.1.1. Example test suite

Below is an example of a test suite using the harness. It has three test cases; test_passed, test_failed and test_skipped.

The test macro BTASSERT(condition) should be used to validate conditions.

#include "simba.h"

static int test_passed(struct harness_t *harness_p)
{
    /* Return zero(0) when a test case passes. */
    return (0);
}

static int test_failed(struct harness_t *harness_p)
{
    /* Return a negative integer when a test case fails. BTASSERT
       will return -1 when the condition is false. */
    BTASSERT(0);

    return (0);
}

static int test_skipped(struct harness_t *harness_p)
{
    /* Return a positive integer when a test case is skipped. */
    return (1);
}

int main()
{
    /* Test harness and NULL terminated list of test cases.*/
    struct harness_t harness;
    struct harness_testcase_t harness_testcases[] = {
        { test_passed, "test_passed" },
        { test_failed, "test_failed" },
        { test_skipped, "test_skipped" },
        { NULL, NULL }
    };

    sys_start();

    harness_init(&harness);
    harness_run(&harness, harness_testcases);

    return (0);
}

The output from the test suite is:

app:    test_suite-7.0.0 built 2016-07-25 17:38 CEST by erik.
board:  Linux
mcu:    Linux

enter: test_passed
exit: test_passed: PASSED

enter: test_failed
exit: test_failed: FAILED

enter: test_skipped
exit: test_skipped: SKIPPED

            NAME        STATE  PRIO   CPU  LOGMASK
            main      current     0    0%     0x0f
                        ready   127    0%     0x0f
harness report: total(3), passed(1), failed(1), skipped(1)

There are plenty of test suites in the tst folder on Github.


Source code: src/debug/harness.h, src/debug/harness.c


Defines

BTASSERTN(cond, res, ...)
BTASSERT(cond, ...)

Assert given condition in a testcase. Print an error message and return -1 on error.

Typedefs

typedef

The testcase function callback.

Return
zero(0) if the testcase passed, a negative error code if the testcase failed, and a positive value if the testcase was skipped.
Parameters

Functions

int harness_init(struct harness_t *self_p)

Initialize given test harness.

Return
zero(0) or negative error code.
Parameters
  • self_p: Test harness to initialize.

int harness_run(struct harness_t *self_p, struct harness_testcase_t *testcases_p)

Run given testcases in given test harness.

Return
zero(0) or negative error code.
Parameters
  • self_p: Test harness.
  • testcases_p: An array of testcases to run. The last element in the array must have callback and name_p set to NULL.

struct

Public Members

harness_testcase_cb_t harness_testcase_t::callback
const char *harness_testcase_t::name_p
struct

Public Members

struct uart_driver_t harness_t::uart