Effectiveness of software testing techniques in enterprise: a case study

 Research Design and Method Selection 
The research methodology and strategy attitudes.
A qualitative research strategy was chosen, 
since it enables the exploration of a phenomenon within its context using a variety of sources of 
evidence and allows multiple aspects to be revealed and understood (McGloin, 2008; Yin, 2012). 
According to Bryman & Bell (2011, pp. 26-28), "qualitative research investigates on the understanding 
and interpretation of individuals regarding their social world which leads to the epistemological 
position of interpretivism". We seek to observe organizational case study and to construct theories 
based on interpretations. Thus, an inductive approach is used for this purpose. The case study research 
is an appropriate strategy for us as we tent to “explore in depth a program, an event, an activity, a 
process” and “collect detailed information using a variety of data collection procedures” (Creswell, 
Plano Clark, Gutmann, & Hanson, 2003). According (Yin, 2003) categorization of case study, we 
choose to adopt exploratory type as our research is used to "explore those situations in which the 
intervention being evaluated has no clear, single set of outcomes". Since our research objective is to 
explore the most problematic areas and potential improvements in software testing process, we seek to 
gather preliminary information that will help define the problems that are faced with during software 
testing and then hypotheses for later studies could be suggested. In addition, outcomes are not clear 
and consist of various problematic areas. Each problematic issue in software testing process depends 
on different project, processes used in specific team, management decisions etc. Our qualitative case 
study focuses on detailed analysis of specific enterprise which faces with undiscovered problems that 
delay software testing process. In order to explore the topic in depth and get wider spectrum of 
potential software testing limitations, all of research and development (hereinafter - R&D) teams are 
involved in our research. All teams have a common quality assurance process, because the software 
components of each team are very close related with other teams and the well known incident handling 
process (e.g. bug reporting) should be used for all teams. On the other hand, the internal processes 

---

47
used in different teams can vary slightly as long as they do not affect other team processes. Thus, 
wider population of case study respondents will give more insights on exploring the limitations and 
improvements in software testing process, including the effective use of software testing techniques 
and processes. However, some researchers (Baxter & Jack, 2008; Yin, 2003) criticize that qualitative 
case study is too subjective and it is difficult to generalize the findings to other context and to maintain 
it in terms of credibility and validity. Indeed, this research results are relevant only for specified 
enterprise and its R&D teams. However, the results could fit partly for other organizations if their 
processes are similar to our enterprise's processes and based on: 1) Agile methodology; and 2) The 
International Software Testing Standard ISO/IEC/IEEE 29119 (“ISO/IEC/IEEE 29119 Software 
Testing Standard,” 2014). After evaluation of these factors it can be assumed that our research 
outcomes could give some relevant insights for other organizations. Furthermore, to overcome 
limitation of subjectivity as far as it is possible, we choose to use mixed method of qualitative and 
quantitative which is supported by Creswell, Plano Clark, Gutmann, & Hanson (2003). Bryman & Bell 
(2011) states that mathematical calculations can be applied to qualitative research as well. Thus, 
regarding this point, effectiveness of software testing techniques in terms of limitations and 
improvements are explored combining both qualitative case study method and quantitative document 
content analysis method.
Case study research is based on experts interview which was provided as semi-structural 
questionnaire. Few reasons for selecting this method are as follows: experts have more knowledge on 
specific field (software quality assurance and testing techniques) and they could provide a more 
comprehensive view on specified topic. Further, a semi-structural questionnaire is chosen to 
investigate only specific fields of problematic issues, as it gives detailed insights on desired topic. Our 
research focuses mostly in such specific areas: test levels, test techniques, test process management. 
We have chosen only these areas in order to avoid a vast of problematic issues, starting from 
employees characteristics and ending at process management in organization globally. Regarding the 
quantitative part of research, we use the analysis of statistical documents of enterprise. These 
documents provide trends on defects reported by different teams. After documents are analyzed and 
experts are surveyed, we can combine both methods. This approach is a combination based on results 
interpretations of experts survey and generalized insights of statistical documents. This is not even 
interpretations of experts results, but the statistical proof as well. Thus, compromising the subjective 
interpretations (including our interpretations and the thoughts of experts), the mixed qualitative and 
quantitative methodologies have been chosen.