Evaluating research quality: Validity and reliability

One of the classical approaches to evaluating the quality of research in positivist and post-positivist paradigms is assessing ‘validity’ and ‘reliability’ (Gibbert & Ruigrok, 2010). While alternative approaches, namely ‘trustworthiness’, have been proposed to assess research quality (Guba, 1981; Lincoln, 1995), validity and reliability were considered appropriate to confirm the quality of this research.

Validity

The validity of a research design is assessed on the basis of three criteria: construct validity, internal validity and external validity (Gibbert & Ruigrok, 2010).

Construct Validity

Construct validity is a test of ‘the extent at which the study investigates what it claims to investigate’ (Farquhar, 2012). The concept is more relevant to the data-collection phase, and triangulation has been suggested as one of the ways to ensure construct validity (Gibbert

& Ruigrok, 2010; Schell, 1992). Construct validity is ensured by the use of multiple data collection strategies and sources (Gibbert & Ruigrok, 2010). This study triangulated different data sources including interview data, incidental observation and documents such as policy documents, company reports, training reports, press reports etc.

Peer review has been suggested as one of the ways to ensure construct validity (Gibbert

& Ruigrok, 2010). The four sub-studies that make up this thesis underwent several rounds of rigorous peer review, which required revisions to be made to the manuscripts prior to the approval and publication of the sub-studies in reputable scientific journals. Additionally, the research findings were presented and discussed at various international academic conferences, allowing for feedback from experts and academics in this field of research.

The review process, responses and approval from members of the academic community and specifically experts in the field of China–Africa relations, public administration and other relevant disciplines equally contributed to validating this research.

Construct validity can also be ensured by clearly explaining the research process: from the development of research questions to the drawing of final research conclusions (Geertz, 2003; Gibbert & Ruigrok, 2010; Yin, 1994). To ensure construct validity in this work, the data collection and analysis procedures were described in the relevant sections of this chapter and are reiterated in the sub-studies. Further, the interview data utilised in this research was obtained from original interviews conducted by the researcher.

Internal Validity

Internal validity specifically concerns the data analysis process (Gibbert & Ruigrok, 2010; Schell, 1992). Simply, internal validity aims at ensuring that the research findings are derived from data (Farquhar, 2012; Silverman, 2005). Essentially, to ensure internal validity, a researcher should be able to convince themselves and their readers that the findings from their study are derived from a critical analysis of all their data and not just from a few suitable examples (Silverman, 2005).

Several strategies have been proposed to ensure internal validity, such as ensuring that the research framework utilised in a study is derived from literature (Yin, 1994). In this research, internal validity was confirmed by the use of a clear research framework, explicitly derived from literature, which guided the sub-studies. Further, pattern matching was done with previous research; that is, the relationships between data generated in the sub-studies (specifically Sub-sub-studies 3 and 4) were established and discussed vis-à-vis previous research. This step is in line with the argument that researchers can use pattern matching as a technique to ensure internal validity (Denzin & Lincoln, 1994; Eisenhardt, 1989), a process Eisenhardt (1989, p. 544) describes as ‘enfolding the literature’.

External Validity

External validity is highly crucial in the research design phase (Schell, 1992). On generalisability, which is also referred to as external validity, some scholars argue that statistical generalisations are not applicable to case study research because of its context-dependent nature (Arksey & Knight, 1999; Baskerville & Lee, 2003; Yin, 1994). However, some experts believe otherwise, arguing that multiple case studies on a topic carry more external validity than a single case study (Johnston, Leach, & Liu, 1999; Leonard-Barton, 1990). Interestingly, this view, too, has been rejected by some case study researchers (Flyvbjerg, 2006; Siggelkow, 2007; Yin, 2009). Yin (2013) posits that case studies are generalisable in an analytical manner rather than a statistical one. According to Yin (2013), statistical generalisations cannot be derived from multiple cases or a single case; however, analytical generalisations can be formed as they depend on using a study’s theoretical framework to establish a logic that can be applied to other situations. Simply put, analytical generalisations can be made from observations to theory (Yin, 2013).

This study takes that position as the goal of the study was to achieve analytical generalisation. This study was primarily concerned with investigating the opportunities and implications of CEE for HCD in Africa. The study finds that CEE does present opportunities for HCD in Africa and that Chinese enterprises are significantly well placed to contribute to HCD in their host countries in Africa. However, these contributions will vary across countries and sectors of the African economy because of the different contextual and sectoral conditions in these host countries. Thus, the results of this study transcend the boundaries of the case (Huawei in Nigeria) investigated. Further, the rationale for selecting the case organisation clearly articulated the representativeness of the case organisation the telecom industry and its appropriateness to the phenomena being studied and the research question. Additionally, details of the context of study were shared to justify the sampling choices made in this study (Gibbert & Ruigrok, 2010).

Reliability

According to Hammersley (1992, p. 67) reliability refers to ‘the degree of consistency with which instances are assigned to the same category by different observers or by the same observer in different occasions. Broadly, it’s seen as an assessment of whether the evidence is consistent and stable (Remenyi, et al. 1998). Some social science researchers are of the view that the need for reliability and validity only arises in quantitative research and not qualitative research (Silverman, 2010). They argue that if social reality is considered as always being in a state of flux, then ensuring research instruments measure accurately isn’t necessary. However, Kirk and Miller (1986) note that without paying attention to reliability in qualitative research, the results of field research will be ignored.

Reliability is most crucial in the data collection phase of research (Schell, 1992). To ensure reliability in this study, interviews were tape-recorded except in a few instances where permission to record the interview was denied. In such instances, detailed notes

were taken instead. All the tape recordings were carefully transcribed, and some extracts of interview responses have been included in the sub-studies. This is in line with Silverman’s suggestions for increasing reliability of interview data (Silverman, 2005). Additionally

‘careful documentation and clarification of research procedures’ has been identified as a strategy for ensuring reliability (Gibbert & Ruigrok, 2010, p. 715). This strategy was implemented in this study via detailed reporting of the processes and procedures of the case study as well as divulging the name of the case organisation.