Constructing automated test oracle for low observable software

Abstract

Using machine learning techniques for constructing automated test oracles have been successful in recent years. However, existing machine learning based oracles have deficiencies when applied to software systems with low observability, such as embedded software, cyber-physical systems, multimedia software programs, and computer games. This paper proposes a new black box approach to construct automated oracles which can be applied to software systems with low observability. The proposed approach employs an Artificial Neural Network (ANN) algorithm which uses input values as well as corresponding pass/fail outcomes of the program under test, as the training set. To evaluate the performance of the proposed approach, we have conducted extensive experiments on several benchmarks. The results manifest the applicability of the proposed approach to software systems with low observability as well as its higher accuracy in comparison to a well-known machine learning based method. We have also assessed the effect of different parameters on the accuracy of the proposed approach.