Real-Time Software Verification and Validation

1. Real-Time Software Verification and Validation

2. Background • Two major difficulties of building real-time embedded applications • temporal dependence in program behavior • asynchronous events • Test or formal method? Implementation or model? • Software quality analysis and test coverage • Instrumentation at source programs • program behavior may be changed due to timing intrusion • test a robotic controller in the target system – hardware and human-in-the loop operations • some solutions : • hardware-based trace collection (Applied Microsystems) • special datalogging, monitoring, and test facility (SVF for NASA ISS)

3. Our Approach -- A Two-stage Instrumentation • Extract task/system interactions at RTOS and device drivers: events, input signals, timing • Program execution trace during replay Source programs Program analyzer program model system & task instrumentation program instrumentation Human- and hardware- in-the-loop test runs Timing analysis real-time events traces & input signals Emulated test runs Coverage analysis program traces

4. Our Approach (continued) • Instrumentation based on RTOS -- for context switches, interrupts, events, and task communication • Annotation for device drivers • A challenging task -- to emulate the execution during program instrumentation • synchronize program execution with external events • cannot rely on program counter • an interrupt during a loop (need loop count and program counter) • simulated time • must be adjusted to match with the real execution time • determine when an event occurs • if no data dependence, it can occur at any instance during a block execution • else, need to know the corresponding statement

5. Execution/ Instrumentation Execution D. replay/ Instrumentation Execution/ Observation/ Assertion Execution D. replay/ Observation/ Assertion Deterministic Replay • If the intrusion cannot be tolerated • Exact execution in a controlled environment for multiple purposes: • Test analysis • Debugging • Recovery Execution/ Checkpointing/ Msg logging Rollback/ D. replay

6. Software Instruction Counter • Exact instance in program execution • specified by program counter (PC) I/O status changed read I/O check value read I/O check value • Software instruction counter (SIC) -- • incremented when backward jump or procedure call • software or hardware implemented • Has been applied to recovery and debugging

7. Research Issues • Enhancements – • insertion of software instruction counting operations based on program dependence analysis • RTOS and hardware support for recording and replay • Multi-thread environment • Software components for test analysis, recording and replay • Program analysis + (PC, SIC) + time  temporal dependence • Two test inputs T1 and T2 that have the same data values, but different arrival instances. Should both be tested? • Useful parameters for performance measurement ?

8. Current Status source program code instrumentation code analyzer execution trace ESIC and replay instrumentation ESIC, system, and event instrumentation target - record environment target - replay environment instrumented program instrumented program time stamp converter event trace event trace

9. Testing Analysis in vxWork with Suds • Instrument C codes and run in vxWorks-based targets • Trace collected in the host • Portable libraries independent of target • Multiple threads

10. Current Research Tasks • Time stamp converter and tool integration • Experiments • joystick program with input and timer • air instrument for altitude and velocity (DAQ and multitasking) • applications in JSC • Porting • vxWorks and Suds on MBX860 embedded controller • porting to RT-linux and other platforms • Test of temporal behavior • automatic changes of event occurrence instances • Documentation and dissemination