Experimental Validation of the Instrumentation and Control Design Evaluation Method for Predicting and Estimating Common Cause Failures in Software Development

Employing diverse development teams may still result in common faults being introduced in various stages of software development that will cause failures if the developers in those teams possess common incorrect knowledge or have common gaps in their knowledge. The Instrumentation and Control Design Evaluation method can predict and estimate the probabilities of the occurrence of common faults in software developed by diverse teams during the different stages of software development, such as requirements specification, design, and implementation. Common knowledge possessed by the teams is evaluated, and certain performance-shaping factors are assessed using predefined question sets to serve as the input for the Instrumentation and Control Design Evaluation method. This paper presents the design of an experiment to validate the Instrumentation and Control Design Evaluation method during software requirements specification. A level control system is selected to serve as the subject of the experiment, and the development teams will be asked to develop the software requirements specification for the level controller. The experiment instrumentation consists of a control system question set and a software requirements question set devised to evaluate the domain-specific and software engineering knowledge of the participating development teams. In addition, another question set is developed to assess the performance-shaping factors used by the COCOMO II model. A detailed user requirements document is developed to be provided to the development teams, and a large language model-based tool is developed to trace the user requirements to the developed software requirements.