Embedded Software Unit Testing, De-facto Standard for Coverage Measurement Tools

CoverageMaster winAMS

CoverageMaster winAMS

Embedded Software Unit Testing, De-facto Standard for Coverage Measurement Tools

This is a unit test tool for embedded C/C++ software, compliant with the functional safety standard ISO 26262/IEC 61508. It can measure code coverage for statements, branches, MC/DC, function calls, etc. A built-in microcontroller simulator runs the unit tests by executing compiled object code. In comparison to the (more common) unit testing at the logic level of the C source, this provides unit testing that is the most reliable and faithful to the final embedded microcontroller code (Assembly code). (*C++11/14 supported. See below for details.)

Running unit tests at "machine code level" for the most faithful results, leading to improved efficiency and overall lead times.

"CoverageMaster winAMS" runs unit tests on a Microcontroller Simulator using the "Compiled Object Code" generated by the cross-compiler as it is. Compared to unit testing at the logic level of the C source, which is the common approach in unit testing tools, it provides Test results that are more reliable and faithful to the final microcontroller code, including the detection of issues that depend on the implementation of the specific embedded microcontroller.

In addition to C0, C1, and MC/DC Coverage, Function/Call Coverage Measurements for the Integration Testing phase are also supported.

Fully supports the code coverage measurement standards: C0 (Statement Coverage), C1 (Branch Coverage), and MC/DC(Modified Condition Decision Coverage) required for functional safety certification (ISO 26262), as well as the function/call (Function/Call) coverage measurement also required for functional safety during Integration Testing.

Certified for Automotive Functional Safety ISO 26262/IEC 61508

CoverageMaster winAMS, together with CasePlayer2, has been certified by the third-party certification organization TUV SUD for the ISO 26262 Automotive Functional Safety standard and its meta-standard IEC 61508. It is therefore recognized as a valid tool to generate all the Functional Safety Test Evidence required in a variaety of industries. All the TCL and ASIL levels for Automotive functional safety are applicable.

A "Test Data Analysis Editor" to efficiently creates white/black box test data

CoverageMaster is equipped with a "Test Data Analysis Editor," an editor for efficiently creating test cases based on requirements and specifications. This feature efficiently designs test data from the required specifications of C functions using the "Test Data Analysis method"established by GAIO through the experience of its Unit Tests Outsourcing services.
By looking at the Specifications Requirements while re-using the auto-generated Test Data, both Black-box (=Requirement level) testing and White-box (C source level) testing can be done efficiently.

Test Flow using actual Machine Code and CSV Test Cases

This section shows the flow of executing real machine code generated by the cross-compiler and creating Unit test cases.

"Code Coverage Measurement" shows the amount of code that was covered by a specific Unit Test

The Test Data (CSV file) created for each function holds the Input Test Data but also the results of the unit Tests and the automatically measured code coverage for that function.

 ● C0: Statement coverage
 ● C1: Branch coverage
 ● MC/DC
 ● Function coverage
 ● Call coverage

"Test Data Analysis Editor," to design test cases based on requirements specifications

CoverageMaster includes the "Test Data Analysis Editor," an editor for efficiently creating test cases based on requirements specifications. The Unit test casse are derived from boundary values, maximum/minimum values for data types, and typical values per equivalence class, which are all displayed in one table.
When a function is selected, its code structure is automatically displayed based on static code analysis, and the boundary values of conditions and maximum/minimum values are automatically extracted from the code.
By looking at the Specifications Requirements while re-using this auto-generated Test Data, both Black-box (=Requirement level) testing and White-box (C source level) testing can be done efficiently.

A CLI to automate the test run process with scripts

CLI (Command Line Interface) functionality is supported for controlling and operating CoverageMasters using scripts (standard Windows batch command files) that describe the test steps. Almost all CoverageMaster's operations can be done through scripts, making test management and execution easier.
When the source code is updated, a "regression test" is required to check the effect of the modification through the whole function. In addition, the same source code is often used in a wide variety of versions (ex: vehicle models) thanks to general-purpose development platforms, like AUTOSAR in the automotive field. With the CLI function, a batch command can be used to run the whole test process, including compilation, in a single batch file. Repetitions of the same test are fully automated with no need for human interactions on the GUI.

Link with the Static Analysis tool "CasePlayer2" to efficiently browse and analyze Sources files

The CasePlayer2-generated flowcharts and documents (Module/Structure graphs, Function Call graphs etc...) can be linked to be used to review the source code during Unit Tests. The automatic detection of external variables used by the function makes it easy to identify the input variables for the test, and prevents human errors.

"Function Coverage and Call Coverage" measurement function

Structural coverage measurement for Integration level Testing is also supported. "Function coverage", which indicates whether all functions in a specified source file have been executed, and "Call coverage", which indicates whether all sub-function calls in a specified function have been covered, can be measured, as required by the Integration Test Part (6 to 10) of the Automotive Functional Safety standard ISO 26262.

 [Function Coverage]
 Test cases are run for the top-level function of a component
 to measure whether the sub-functions that make up
 the functional component were all executed or not.

 [Call coverage]
 Test cases are run for the function under test to measure
 whether all the sub-function calls in the function were  
 executed or not.

Automated test case generation for structural coverage measurement (C1, MC/DC)

With the information from the Static Analysis tool CasePlayer2, CoverageMaster can analyze the conditions of IF, SWITCH, FOR, and WHILE branches, and automatically generate test cases that cover 100% of C1 and MC/DC.
In the data creation window the function's conditional statements (IF, SWITCH etc...)are listed. Clicking each condition automatically searches the variables and boundary values associated with that conditional statement and generates the data required to cover all branches.

Unit test software kit for C++ 11/14 and AUTOSAR-AP

C++11 (ISO/IEC 14882:20119) and C++14 (IEC 14882:2014) language standards are also supported in addition to the conventional C++ 03 to support testing AUTOSAR Adaptive Platform Software.
In addition, a software kit can be used to support STLs and POSIX API environments. Using the results of test frameworks ( Google Test) as they were created by developers,
and by linking with the Windows DLL, it is possible to execute accesses to common files, for example so that Image data can also be imported.

Main features

Code coverage measurementStatement (C0), Branch (C1), MC/DC, Functions, and Calls
Microcontroller simulatorSupports a large number of embedded microcontrollers such as Altera, ARM, Cypress, Infineon, JRC, Melexis, NXP, Panasonic, Renesas, Sony, ST Microelectronics, Texas Instruments, and Toshiba.
Automatic test data generationC0 and MC/DC coverage data based on Branch Condition analysis
Command line executionTest execution through batch files (Command Line Interface)
Functional Safety CertificationISO 26262 (TCL1 to TCL3) and IEC 61508 Tool Certifications, and related Safety manual ("Reference Workflow") can be provided to our users

Associated link

scroll back to page top