3.3.1 Reliability
Qualitative study researcher has to constantly reflect on the solutions made and take a stand both for the coverage of the analysis and for the reliability of the work (Eskola and Suoranta 1998b, 209).
“The repeatability of the study is called reliability, and it is designed to resist the effects of chance, and to highlight the similar results from different measurement times. The reliability of the study is directly proportional on the reliability metrics of the present study”
(Metsämuuronen 2005, 64-‐5). Research reliability refers to the manner in which the research data results can be generalized.
I find it necessary to bring up a few things out of the collected data. The players who responded to this survey were on average quite active digital sports game players. Minority of the players played the games only once a week. Does this mean that the digital sports game players are on average active players? Or rather that the people selected for this research are active digital sports game players? It is likely that no generalizations should be based on this study sample and that further study on the matter is needed.
The basic idea of this research is to study the correspondents own experience of the effects of digital sports game playing, not the effects themselves. Because of this, the responses and the analysis cannot directly be compared to other studies in the same field. My research focus was on eight people, four men and four women. Since the survey was conducted with such a small number of subjects, it is clear that the results cannot be generalized and conclusions on digital sports game playing and about the players cannot be done.
If this research was conducted again and the object of the research were the same people, the results would most likely be different because of the time interval and the fact that people's behavior and preferences can change radically even within a short period of time. However, this does not mean that some conclusions about human motivation, reasons for action and experiences could not be made.
Inspite of that, some conclusions can be made. This material tells a story of the reasons this specific group of Finnish men and women play digital sports games. The most important thing this study reveals is the personal emotions and feelings the games and playing raises on the players. In addition, this study reveals reasons why people keep on playing the games and why some of them do not play.
The present study does not attempt to cover everything. I cannot be sure whether the respondents were honest. It may also be the case that respondents may "refine" the responses, that is to provide a picture better than the reality. On one hand, some respondents may underestimate themselves. Hirsjärvi et al. conclude that the researcher cannot be sure how seriously the respondent has answered the questions and if the answers are honest. On the other hand, the researcher cannot know whether the answer options were successful judging from the perspective of the defendant. There is always the possibility that the respondent did not understand the question in a way the researcher intended. Misunderstandings are difficult to control, Hirsjärvi concludes (2004, 191).
There are a number of limitations specific to this study that should be noted.
1) The study sample is very small, so generalizations should not be made.
2) Respondents of this research were people who mainly enjoyed playing digital sports games and who have previous experience in these games. Thus, this study does not provide broader understanding about the people who are not interested in playing digital sports games or who only play the games randomly.
3) Analysis of this study consists only of those games played by the defendants.
Multiple digital sports games were therefore excluded from this study.
4) The respondents' age range is very limited, comprising of players between 27–46 years of age. That is to say this study does not include children, young adults or seniors.
During this study, a number of assumptions were made by the researcher. First, it was assumed that respondents answer the questions truthfully. Second, it was also assumed that problems with the questionnaire form regarding the layout and opening the form might occur because of software issues. This was taken into account and the defendants were informed that they could answer the questions without utilizing all the automatic functions.
Taking all these things into account, I still consider the material reliable.
3.3.2 Validity
Validity defines how well the material and questions measure the theoretical framework.
The concept of validity is used to answer the question “does the measurement measure what it should measure?” The instruments and methods may not necessarily correspond to what the researcher thinks they would examine. For example, the defendant may understand the questionnaire questions in a completely different way compared to what the researcher means (Hirsjärvi et al. 2004, 216–217, Metsämuuronen 2005, 109).
According to Eskola and Suoranta, the internal validity, i.e. competence, refers to the harmony of the theoretical and conceptual specifications demonstrating the scientific validity of the researcher. External validity means the validity of the relationship between the interpretations and conclusions made of the material. External validity is more closely connected to the behavior of the researcher than the respondent (2008, 214).
The concept of validity can be considered to some extent as lacking since a couple of the respondents felt the question number 20 to be unclear. On the basis of their answers, the researcher believes that the respondents had understood the question as intended. However, it may be that something has been missing. Perhaps too many examples were given with the questions, which confused the respondents.