E VALUATIVE CRITERIA : ASSESSING TRUSTWORTHINESS

Schwartz-Shea & Yanow (2009, 56-82) summarize the evaluative criteria for interpretative ethnographies that is emerging as “standard” for this type of research, and aid in assessing the trustworthiness of a manuscript (rather than validity and reliability of positivist sciences). In this type of research, there is not only a double hermeneutic – researchers interpreting actors’

interpretations – but also a third “interpretive” moment: the act of reading and the prior knowledge that readers bring to their readings of text - Schwartz-Shea & Yanow call this the

“triple hermeneutic”. In order to enable this third interpretation as well as possible, certain elements need to be carefully tended to. I follow Schwartz-Shea & Yanow in these guidelines.

Below I have summarized first the evaluative criteria, and show how my research aims to fulfil each.

Thick description. This refers to the detailed descriptions of events, activities, interactions, people etc. in order to make clear the “lived experience” of the phenomena. Of importance is the notion of sufficient detail, as relevant to the research question - one should not aim for a complete description. Yanow (2010, 1398) states it a “puzzle” how many researchers e.g. in organizational studies present the research as “ethnographic” when it is evident that the material is interviews alone, with no observational data. Participant observation and “thick”

field diaries are however the key to thick description. My primary materials consist of three separate field diaries: one which is the overall diary of my time on-site, reflections and thoughts; second, which resulted from the shadowing one team, and third, which centred around my interactions and observations of the key courses of Product Development Project (PDP) and ME310 (see Chapter 8.3.1) and their participants. I complemented my field notes with 31 in-depth transcribed interviews - or rather “talks” (Yanow 2009, 77), as they were very open-ended and little structured (details of the interviews can be found in Appendix 1).

In addition, I used an open-ended survey that 55 students answered, in order to grasp how they define interdisciplinarity. From secondary sources, I collected material such as flyers, posters and other printed material found at ADF as well as used its website, Facebook and Flickr –sources for material and photos.

When telling the story of ADF, I have aimed for a balance between details and overviews.

The phenomena I observed were broad, and I have chosen a holistic view of the phenomena,

as in the “tour” in Part III. Thus the need to omit some details was inevitable. My tale is not one of minutiae, but more about evoking the lived-experience of my time sent on-site.

Researcher reflexivity. The aim is to show that the researcher understands him-or herself as the means through which the research is produced. The positionality of the researcher affects the nature of interactions, access, persons met and the kinds of data generated during the process. In order to support arguments, researchers need to take account of the particular characteristics researcher brings to the field and how those personal, intellectual, professional and other characteristics shape what it is that we may see (or not), whom we might speak or interact with (or not), the events we can experience (or not). (Schwartz-Shea & Yanow, 2009.)

Triangulation. This means as drawing on different kinds of sources or analytic tools in trying to understand a phenomenon. Thus the researchers own experience and meaning making is triangulated both on other persons and their experiences (observational material, interviews) as well as using other sources (organization’s own documents, research and reports). Multi-site ethnography is another form of triangulation, as is multiple approaches to looking at the data. Important is to bring to light the inconsistent and even conflicting findings.

My thesis uses a variety of material (see Chapters 1.3.2 and 7.5), and thus the main method of triangulation is the multiple methods for generating data. Additional methods are also used (textual analysis, discourse analysis) in order to feasibly analyse the more textual second-source material. I also referred to materials that relate to the role and position of ADF within Aalto University, in order to better grasp the practices within this broader context. A (very) tentative multi-site triangulation occurred when a multi-site ethnography was part of the research design (later abandoned, see Chapter 7.1). Overall I have attempted to sincerely report any arising conflicting findings.

Audit trail. An audit documents changes in the research process while doing the research, and describes the steps taken to conduct the research. The researcher is expected to identify and describe any changes to original research design as a response to changing situations, and provide transparency into how the research evolved over time and why.

The research process of this thesis has been long and changes have been inevitable. In hindsight I identify three main types of changes that have happened: first were the changes in the actual scope of the ethnography itself from one-site to multi-site to one-site again; second type has to do with the clarification of the actual phenomena itself under observation; and third involves the theoretical underpinnings that evolved during the research itself, pointing me towards issues that needed new material generation.

Negative case analysis. There are many ways to challenge researcher’s own meaning-making processes, e.g. peer evaluation, using extreme cases, following up surprises, checking out rival explanations. The immersion in generated data may easily create blind spots, and the researcher must constantly ask: “How do I know what I think I know?”; “Would I know if I am missing something?” in order to challenge the favoured explanations.

This has been the most challenging part of my research process. As most of it was done part-time and outside the academia, it has been indeed quite a lonely process. However, there has been an advantage to this as well: as a researcher I have been an outsider in the sense that I am not guided explicitly or implicitly by discipline-specific discourses and am not socialized into certain paradigms. In this sense I feel I have been able to be quite “earnest” in my interpretations, as I hold no strong value statements to certain explanations.

Member-checking. Going back to the people in the setting studied for an assessment if the researcher did a good job in capturing their understanding is essential to all fieldwork. It is the recognition of their experience at full value – even if not accepting their critique outright. It evaluates the researchers ability of having really “been there”, engaged and present. It is not about “getting it right”, as there is no objective reality to be captured – rather it looks for the potential differences in the interpretations between the researcher and research participants.

My core time on-site at ADF was between January – April 2011. I returned to ADF in November 2014 and met many of the same people, discussing my findings. The manuscript was read and commented by Factory Director Ekman in January 2015, who kindly pointed out some factual mistakes.