Reliability, validity and ethics

The approach of Yin (2009) is applied into this research which assists when evaluating both reliability and validity of this study. He suggested that researcher should judge the research design quality through construct validity, internal validity, external validity and reliability (Yin 2009, 40-41) to provide more comprehensive evaluation compared to traditional view on validity and reliability. Reliability refers to repeatability of the results meaning whether data collection and analysis procedures would provide same findings if the study was conducted again (Hirsjärvi et al. 1997, 226; Saunders et al. 2009, 156), meaning that there is no discrepancy between studies. (Koskinen, Alasuutari & Peltonen, 2005, 255) Whereas validity refers to whether the findings measure what was indented (Hirsjärvi et al. 1997, 226;

Koskinen et a. 2005, 254; Saunders et al. 2009, 157). Researcher should be able to prove that the findings are reliable, meaning that they are not driven from irregular situations which

may lead to false observations or statements. (Koskinen et al. 2005, 254) Therefore quotations from interviews are applied to the study not only to increase the validity but also to enrich empirical findings. (Hirsjärvi at al. 1997, 228; Koskinen et al. 2005, 318) It should also be noticed that the results can be reliable even when the validity is low, yet the results cannot be valid if they are not reliable. (Koskinen et al. 2005, 255)

To increase construct validity several sources of evidence that have a logical connection are used in the data collection including the case company’s annual report, social media sites and the personnel from both managerial and frontline employee level (Koskinen et al.

2005, 257; Yin 2009, 41) to confirm that the research explores exactly what it is assumed to explore. (Hirsjärvi & Hurme 2000, 187) Researcher aims also to build a chain of evidence during the research to amplify construct validity, meaning that the phases of the study are not only both planned and documented carefully but also explained in sufficient detail.

(Hirsjärvi et al. 1997, 227; Yin 2009, 41) Moreover, a rough copy of the study is reviewed by the participants of the research before the study is finalized to ensure the validity of the content. (Yin 2009, 41) Internal validity instead refers to consistency and logicality of the research that occurs during the data analysis. To increase internal validity, both the explanations for the results and divergent explanations have been built during the analysis.

(Koskinen et al. 2005, 257; Yin 2009, 42) Moreover, the analysis followed themes that are logically derived from the interviews and are then matched to the theory pattern. (Koskinen et al. 2005, 255; Yin 2009, 42)

External validity refers to generalizability of the research among both various situations and individuals (Hirsjärvi & Hurme 2000, 188). It should be noted that the results of this study are generalizable in the context of the case company, yet the aim is not to generalize service companies’ approaches but to understand the case company’s operations and the nature of the possible issues related to social media service recovery. In other words, this study aims to observe how the case company implements and manages social media service recovery in a specific timeframe instead of aiming to generalize the results. However, the aspect of external validity is strengthened by collecting rich and comprehensive data from the case company’s operations. However due to a relatively small sample size (content analysis n=502, semi-structured interviews n=3) it is not possible to make major conclusions among the subject. (Hirsjärvi et al. 1997, 176; Koskinen et al. 2005, 265-266)

According to Koskinen et al. (2005, 167) generalizability is the main issue on single case studies, yet it is more important to design case study carefully even though the results may only be applied to this particular research. Moreover, the results of this study can be applied when for example comparing social media service recovery practices among delivery companies, different industries or even countries (e.g. how social media service recovery is applied by Post of Sweden compared to the case company). (Koskinen et al. 2005, 167-169)

As stated earlier, reliability refers to repeatability of the results which can be observed in three ways: 1) if the study is conducted again with the same results, 2) if two researchers have concluded the same results or 3) two researchers concluded similar results when applying different research methods. (Hirsjärvi & Hurme 2000, 186) It should be noticed that this study investigates a subject where the service process has not only humane characteristics but also changes as time goes by, thus later studies on the same subject may have slightly different results. In other words, organization changes when new practices are applied, employees may have different skills and customer expectations and perceptions may change. (O’Neill & Palmer 2004, 443)

When considering the limitations stated above, the aim is to provide sufficient information offering the possibility to evaluate both the research process and its results. In other words, this study incorporates a systematic view on how the research is done including the review on how both content analysis and semi-structured interviews are conducted enabling repeatability to some degree. Moreover, correlation between the data is explored and evaluated, yet occurred deviations are discussed in results.

In addition, to comprehensively evaluate reliability it should also be considered how both the researcher and organization may affect the results. (Koskinen et al. 2005, 258-259) When evaluating the organizational point of view, the results may be affected due confidential information (e.g. internal processes), respondent’s position and the time the case company has employed the interviewees. This is discussed further in Limitations (see chapter 6.6.) To ensure that the interviewees were capable of providing the best possible information, the aim of the research was discussed briefly during each interview. Moreover, the researcher is familiar with the case company’s products and services to some degree and as the researcher has no bad experiences with them it supports the objectivity.

Researcher must be appropriate with actions towards the research subjects and those who may indirectly be affected during the research to respect the research ethics. (Saunders et al. 2009, 181-182) From the ethical point of view this research does not contain any significant conflicts as the subject of this study is not particularly sensitive, the case company’s employees have agreed to take part in this study and the data from social media is publicly available. Moreover, instead of individuals this study aims to explore behavior and practices of certain groups from either customer or company point of view.

Yet, in order to provide ethically valid research, researcher has to adhere good scientific practices. To respect the human rights and free will by giving the opportunity to either participate or not to participate on the interviews is top priority. Moreover, it is critical that interviewees have the information on how they are involved in the research process and understand the given information. (Hirsjärvi et al. 1997, 24-25; Saunders et al. 2009, 185) Participation on this study is voluntary as interviewees’ willingness to take part on the study’s interviews is requested via email in advance while informing the terms of the subject and why the study is done.

According to Saunders et al. (2009, 185) the primary ethical issues along volunteering include privacy and confidentiality which should be ensured for the participants. Therefore even the data collected from publicly available sources is used without further announcement to the participants. Participants are also protected in terms of anonymity which is ensured by anonymizing the data before and during the analysis and reporting.

When quoting the material the names have for example been changed to “interviewee” and

“complainer”.

However, there are some other ethical issues to discuss when conducting qualitative research on internet. This study is done with passive analysis meaning that the researcher does not participate in conversation but gathers the data from a certain internet site, yet there are basic ethical precepts that scientific research requires. Even though discussion on the case company’s Facebook and Twitter sites is public the users may not expect to be a part of the research, but on the other hand most of the participants are seeking publicity when voicing disappointment out loud online. As the data is first collected and then translated to English and the participants names are either deleted or changed to “original complainer", “third party” or “customer service personnel”, original messages are not very easily trackable by search engines ensuring the privacy among participants. (Eysenbach &

Till 2001, 1103-1105)