Testing embedded systems is extremely different from classical testing. First of all, there are quite a few non-standard platforms for embedded systems. Secondly, frequently no test tools or the tests themselves cannot be installed on the device and/or embedded platform. Additionally, the system under test doesn’t wait while its tested. The time to execute a series of operations takes less than a hundred of a second, and a number of tests must be run in this time.
To solve these challenges, we use external boards (carrier-boards) or various IP-KVM devices to send commands to the system under test. Generally, we develop a framework and “harness” for each embedded device in order to run automated tests.
We have two areas for automated testing of embedded systems:
Testing of the Embedded OS and Application for Embedded OS
- Model-based testing
- Functional testing of the OS/embedded software
- Regression of the OS/embedded software
- Unit testing
- Mocking (emulation of external data sources and signals)
- Testing of the command stack (including testing floating point operations)
- Testing components based on protocols (CAN, I2C, RS232, RS245, TCP, REST)
- Testing of drivers for elements of embedded systems with respect to specific OSes (Linux, RTOS, QNX)
Direct Testing of systems (Hardware Testing)
- Model-based testing using a model for PLD and microcontrollers
- Spec-based testing
- Test-bench testing (testing systems on a test bench using external carrier boards and sensors)
- Testing communications with external systems at the level of interfaces and signals (sampling and analysis of input/output signals)
- Testing based on modeling external signals (black box – SCPI, VXI-11, or other wave form generator)
- White-box testing based on a component’s specification
- Load testing using an input stream of prepared data (measurement of the performance of an MCU or cryptographic coprocessors)
- Fault testing (supplying edge-case data , modeling crosstalk and voltage surges for power circuits – from a carrier board)
When testing we use our internal tools to store and analyze data (a Neuron-R system), but we use enterprise systems to generate data. Storing and analyzing the test results using a Neuron-R allows us to generate reports, view past results, and assess aggregate metrics in real-time on a single chart, which makes it possible to see the correlation between input and output parameters. This is an important ability when testing embedded systems and devices.
