Testing embedded systems is extremely different from classical testing. First of all, there are quite a few non-standard platforms for embedded systems. Secondly, frequently no test tools or the tests themselves cannot be installed on the device and/or embedded platform. Additionally, the system under test doesn’t wait while its tested. The time to execute a series of operations takes less than a hundred of a second, and a number of tests must be run in this time.

To solve these challenges, we use external boards (carrier-boards) or various IP-KVM devices to send commands to the system under test. Generally, we develop a framework and “harness” for each embedded device in order to run automated tests.

We have two areas for automated testing of embedded systems:

TESTING OF THE EMBEDDED OS AND APPLICATION FOR EMBEDDED OS

• Model-based testing
• Functional testing of the OS/embedded software
• Regression of the OS/embedded software
• Unit testing
• Mocking (emulation of external data sources and signals)
• Testing of the command stack (including testing floating point operations)
• Testing components based on protocols (CAN, I2C, RS232, RS245, TCP, REST)
• Testing of drivers for elements of embedded systems with respect to specific OSes (Linux, RTOS, QNX)

DIRECT TESTING OF SYSTEMS (HARDWARE TESTING)

• Model-based testing using a model for PLD and microcontrollers
• Spec-based testing
• Test-bench testing (testing systems on a test bench using external carrier boards and sensors)
• Testing communications with external systems at the level of interfaces and signals (sampling and analysis of input/output signals)
• Testing based on modeling external signals (black box – SCPI, VXI-11, or other wave form generator)
• White-box testing based on a component’s specification
• Load testing using an input stream of prepared data (measurement of the performance of an MCU or cryptographic coprocessors)
• Fault testing (supplying edge-case data , modeling crosstalk and voltage surges for power circuits – from a carrier board)

When testing we use our internal tools to store and analyze data (a Neuron-R system), but we use enterprise systems to generate data. Storing and analyzing the test results using a Neuron-R allows us to generate reports, view past results, and assess aggregate metrics in real-time on a single chart, which makes it possible to see the correlation between input and output parameters. This is an important ability when testing embedded systems and devices.