Statistical Process Control for Software: Fill the Gap

The characteristic of software processes, unlike manufacturing ones, is that they have a very high human-centered component and are primarily based on cognitive activities. As so, each time a software process is executed, inputs and outputs may vary, as well as the process performances. This phenomena is better identified in literature with the terminology of “Process Diversity” (IEEE, 2000). Given the characteristics of a software process, its intrinsic diversity implies the difficulty to predict, monitor and improve it, unlike what happens in other contexts. In spite of the previous observations, Software Process Improvement (SPI) is a very important activity that cannot be neglected. To face these problems, the software engineering community stresses the use of measurement based approaches such as QIP/GQM (Basili et al., 1994) and time series analysis: the first approach is usually used to determine what improvement is needed; the time series analysis is adopted to monitor process performances. As so, it supports decision making in terms of when the process should be improved, and provides a manner to verify the effectiveness of the improvement itself. A technique for time series analysis, well-established in literature, which has given insightful results in the manufacturing contexts, although not yet in software process ones is known as Statistical Process Control (SPC) (Shewhart, 1980; Shewhart, 1986). The technique was originally developed by Shewhart in the 1920s and then used in many other contexts. The basic idea it relies on consists in the use of so called “control charts” together with their indicators, called run tests, to: establish operational limits for acceptable process variation; monitor and evaluate process performances evolution in time. In general, process performance variations are mainly due to two types of causes classified as follows:  Common cause variations: the result of normal interactions of people, machines, environment, techniques used and so on.  Assignable cause variations: arise from events that are not part of the process and make it unstable. In this sense, the statistically based approach, SPC, helps determine if a process is stable or not by discriminating between common cause variation and assignable cause variation. We can classify a process as “stable” or “under control” if only common causes occur. More precisely, in SPC data points representing measures of process performances are collected. These values are then compared to the values of central tendency, upper and lower limit of admissible performance variations. While SPC is a well established technique in manufacturing contexts, there are only few works in literature (Card, 1994; Florac et al., 2000; Weller, 2000(a); Weller, 2000(b); Florence, 2001; Sargut & Demirors, 2006; Weller, & Card. 2008; Raczynski & Curtis, 2008) that present successful outcomes of SPC adoption to software. In each case, not only are there few cases of successful applications but they don’t clearly illustrate the meaning of control charts and related indicators in the context of software process application. Given the above considerations, the aim of this work is to generalize and put together the experiences collected by the authors in previous studies on the use of Statistical Process Control in the software context (Baldassarre et al, 2004; Baldassarre et al, 2005; Caivano 2005; Boffoli, 2006; Baldassarre et al, 2008; Baldassarre et al, 2009) and present the resulting stepwise approach that: starting from stability tests, known in literature, selects the most suitable ones for software processes (tests set), reinterprets them from a software process perspective (tests interpretation) and suggest a recalculation strategy for tuning the SPC control limits. The paper is organized as follows: section 2 briefly presents SPC concepts and its peculiarities; section 3 discusses the main differences and lacks of SPC for software and presents the approach proposed by the authors; finally, in section 4 conclusions are drawn.