This is a unit test tool for embedded C/C++ software, compliant with the functional safety standard ISO 26262/IEC 61508. It can measure code coverage for statements, branches, MC/DC, function calls, etc. A built-in microcontroller simulator runs the unit tests by executing compiled object code. In comparison to the (more common) unit testing at the logic level of the C source, this provides unit testing that is the most reliable and faithful to the final embedded microcontroller code (Assembly code). (*C++11/14 supported. See below for details.)
"CoverageMaster winAMS" runs unit tests on a Microcontroller Simulator using the "Compiled Object Code" generated by the cross-compiler as it is. Compared to unit testing at the logic level of the C source, which is the common approach in unit testing tools, it provides Test results that are more reliable and faithful to the final microcontroller code, including the detection of issues that depend on the implementation of the specific embedded microcontroller.
Fully supports the code coverage measurement standards: C0 (Statement Coverage), C1 (Branch Coverage), and MC/DC(Modified Condition Decision Coverage) required for functional safety certification (ISO 26262), as well as the function/call (Function/Call) coverage measurement also required for functional safety during Integration Testing.
CoverageMaster winAMS, together with CasePlayer2, has been certified by the third-party certification organization TUV SUD for the ISO 26262 Automotive Functional Safety standard and its meta-standard IEC 61508. It is therefore recognized as a valid tool to generate all the Functional Safety Test Evidence required in a variaety of industries. All the TCL and ASIL levels for Automotive functional safety are applicable.
CoverageMaster is equipped with a "Test Data Analysis Editor," an editor for efficiently creating test cases based on requirements and specifications. This feature efficiently designs test data from the required specifications of C functions using the "Test Data Analysis method"established by GAIO through the experience of its Unit Tests Outsourcing services.
By looking at the Specifications Requirements while re-using the auto-generated Test Data, both Black-box (=Requirement level) testing and White-box (C source level) testing can be done efficiently.
This section shows the flow of executing real machine code generated by the cross-compiler and creating Unit test cases.
The Test Data (CSV file) created for each function holds the Input Test Data but also the results of the unit Tests and the automatically measured code coverage for that function.
● C0: Statement coverage
● C1: Branch coverage
● Function coverage
● Call coverage
CoverageMaster includes the "Test Data Analysis Editor," an editor for efficiently creating test cases based on requirements specifications. The Unit test casse are derived from boundary values, maximum/minimum values for data types, and typical values per equivalence class, which are all displayed in one table.
When a function is selected, its code structure is automatically displayed based on static code analysis, and the boundary values of conditions and maximum/minimum values are automatically extracted from the code.
By looking at the Specifications Requirements while re-using this auto-generated Test Data, both Black-box (=Requirement level) testing and White-box (C source level) testing can be done efficiently.
CLI (Command Line Interface) functionality is supported for controlling and operating CoverageMasters using scripts (standard Windows batch command files) that describe the test steps. Almost all CoverageMaster's operations can be done through scripts, making test management and execution easier.
When the source code is updated, a "regression test" is required to check the effect of the modification through the whole function. In addition, the same source code is often used in a wide variety of versions (ex: vehicle models) thanks to general-purpose development platforms, like AUTOSAR in the automotive field. With the CLI function, a batch command can be used to run the whole test process, including compilation, in a single batch file. Repetitions of the same test are fully automated with no need for human interactions on the GUI.
The CasePlayer2-generated flowcharts and documents (Module/Structure graphs, Function Call graphs etc...) can be linked to be used to review the source code during Unit Tests. The automatic detection of external variables used by the function makes it easy to identify the input variables for the test, and prevents human errors.
Structural coverage measurement for Integration level Testing is also supported. "Function coverage", which indicates whether all functions in a specified source file have been executed, and "Call coverage", which indicates whether all sub-function calls in a specified function have been covered, can be measured, as required by the Integration Test Part (6 to 10) of the Automotive Functional Safety standard ISO 26262.
Test cases are run for the top-level function of a component
to measure whether the sub-functions that make up
the functional component were all executed or not.
Test cases are run for the function under test to measure
whether all the sub-function calls in the function were
executed or not.
With the information from the Static Analysis tool CasePlayer2, CoverageMaster can analyze the conditions of IF, SWITCH, FOR, and WHILE branches, and automatically generate test cases that cover 100% of C1 and MC/DC.
In the data creation window the function's conditional statements (IF, SWITCH etc...)are listed. Clicking each condition automatically searches the variables and boundary values associated with that conditional statement and generates the data required to cover all branches.
C++11 (ISO/IEC 14882:20119) and C++14 (IEC 14882:2014) language standards are also supported in addition to the conventional C++ 03 to support testing AUTOSAR Adaptive Platform Software.
In addition, a software kit can be used to support STLs and POSIX API environments. Using the results of test frameworks ( Google Test) as they were created by developers,
and by linking with the Windows DLL, it is possible to execute accesses to common files, for example so that Image data can also be imported.
|Code coverage measurement||Statement (C0), Branch (C1), MC/DC, Functions, and Calls|
|Microcontroller simulator||Supports a large number of embedded microcontrollers such as Altera, ARM, Cypress, Infineon, JRC, Melexis, NXP, Panasonic, Renesas, Sony, ST Microelectronics, Texas Instruments, and Toshiba.|
|Automatic test data generation||C0 and MC/DC coverage data based on Branch Condition analysis|
|Command line execution||Test execution through batch files (Command Line Interface)|
|Functional Safety Certification||ISO 26262 (TCL1 to TCL3) and IEC 61508 Tool Certifications, and related Safety manual ("Reference Workflow") can be provided to our users|