Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=usnr20
Society & Natural Resources
An International Journal
ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/usnr20
Rethinking Decision Support Under Conditions of Irreducible Uncertainty: Co-Designing a Serious Game to Navigate Baltic Sea Nutrient Enrichment
Neil Powell, Thao Do, Steven Bachelder, Sirkka Tattari, Jari Koskiaho, Turo Hjerppe, Sari Väisänen, Marek Giełczewski, Mikołaj Piniewski & Marta Księżniak
To cite this article: Neil Powell, Thao Do, Steven Bachelder, Sirkka Tattari, Jari Koskiaho, Turo Hjerppe, Sari Väisänen, Marek Giełczewski, Mikołaj Piniewski & Marta Księżniak (2021) Rethinking Decision Support Under Conditions of Irreducible Uncertainty: Co-Designing a Serious Game to Navigate Baltic Sea Nutrient Enrichment, Society & Natural Resources, 34:8, 1075-1092, DOI:
10.1080/08941920.2021.1934930
To link to this article: https://doi.org/10.1080/08941920.2021.1934930
© 2021 The Author(s). Published with
license by Taylor & Francis Group, LLC. View supplementary material Published online: 02 Jul 2021. Submit your article to this journal
Article views: 643 View related articles
View Crossmark data
Rethinking Decision Support Under Conditions of
Irreducible Uncertainty: Co-Designing a Serious Game to Navigate Baltic Sea Nutrient Enrichment
Neil Powella,b , Thao Doa , Steven Bachelderc, Sirkka Tattarid, Jari Koskiahod, Turo Hjerpped, Sari V€ais€anend, Marek Giełczewskie, Mikołaj Piniewskie, and Marta KsieRzniak_ f
aSustainability Learning and Research Centre (SWEDESD), Uppsala University, Uppsala, Sweden;
bSustainability Research Centre, University of the Sunshine Coast, Maroochydore DC, Australia;
cDepartment of Game Design, Uppsala University, Uppsala, Sweden;dFinnish Environment Institute (SYKE), Helsinki, Finland;eDepartment of Hydrology, Meteorology and Water Management, Warsaw University of Life Sciences, Warszawa, Poland;fDepartment of Remote Sensing and Environmental Assessment, Warsaw University of Life Science, Warszawa, Poland
ABSTRACT
Science-informed, reductionist policy has systematically failed to address wicked situations. Such situations are highly interconnected and unpredictable. As a consequence, the implementation of so- called desirable interventions can lead to the export of vulnerabilities within and across different societal domains, sectors, intersections and scales. Systemic practice is an emerging field, and highlights the need to enrich scientific inquiry and policy actions through action learning with an“extended peer community’’as a means to navigate wicked situations. In this paper, we report on the potential of game co-design as a systemic practice to improve the situation of Baltic Sea nutrient enrichment. Findings from water catchments in Finland, Sweden and Poland suggest that the co-design of serious games can both enhance the comprehension of wicked situations, and fos- ter self-organized concerted action without imposing a convergence of perspectives amongst diverse stakeholders.
ARTICLE HISTORY Received 13 October 2020 Accepted 13 May 2021 KEYWORDS
Baltic Sea; co-design of serious games; collaborative decision making;
multifunctionality; river basin governance; systemic practice; wicked situations
Introduction
Decades after the publication of Rittel and Webber (1973) seminal paper, there is grow- ing consensus that sustainability issues are wicked problems. Wicked problems stem from the irreducible uncertainties that arise from contested views on their root cause, and diverging preferences in terms of their solutions (Dewulf et al. 2005). The Baltic Sea is among the world’s most polluted seas, and its nutrient enrichment, that is pre- cipitated by the anthropogenic emissions of nitrogen (N) and phosphorus (P), is often
CONTACT Neil Powell neil.powell@swedesd.uu.se Sustainability Learning and Research Centre (SWEDESD), Uppsala University, Uppsala, Sweden.
Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/08941920.
2021.1934930
ß2021 The Author(s). Published with license by Taylor & Francis Group, LLC.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
2021, VOL. 34, NO. 8, 1075–1092
https://doi.org/10.1080/08941920.2021.1934930
used to exemplify wicked problems (Steffen et al. 2015). This wicked situation grows out of complex and dynamic systems that are characterized by non-equilibrium proper- ties, as exemplified by droughts and floods. Similarly, the systemic nature of nutrient enrichment leads to “complex interdependencies between the biophysical and socio-eco- nomic domains” and the amplification and export of problems between: upstream and downstream, terrestrial and marine contexts, societal intersections, and nation states (Powell et al. 2017). Moreover, other existing pressures such as climate change are pro- jected to amplify enrichment (Bartosova et al. 2019) and the flow-on effects that mag- nify other issues such as food security, renewable energy and biodiversity.
Faced with these types of indeterminate problems, neither normal scientific inquiry, nor the prevailing hierarchical and siloed governance structures are well equipped to deliver definitive solutions (Head, Ross, and Bellamy 2016; Paquet 1999). This can be instanced by the low level of compliance to the first generation of EU environmental directives deployed by the agricultural, wastewater and environmental sectors to tackle the sources of Baltic Sea nutrient enrichment (Coffey and Richartz2003).
In recognition that wicked situations are characterized by multiple and conflicting interests that transcend both sectors and stakeholders, the science and policy commu- nity have been increasingly seeking inspiration from participatory and systemic modes of knowledge production and governance. This can be exemplified by the introduction of the EU Water Framework Directive (WFD) that promotes both integrated and par- ticipatory river basin management as the means to achieve good water and ecosystem status. Despite the ongoing paradigm shift toward participatory and systemic modes of knowledge production, preexisting sectoral and hierarchical character of governance structures remain unreceptive to pluralistic knowledge. The tradeoff or consensus mak- ing processes used to “translate” this knowledge into implementable actions tend to be deeply political, and inherent power imbalances often allow those in dominant positions to unduly shape outcomes and reinforce path dependencies (Knight et al. 2019;
Westberg and Powell 2015).
To address the failure of existing governance structures to effectively draw on plural- istic knowing, Knight et al. (2019) argues that the knowing/doing (science/policy) dichotomy needs to be bridged. In recent years, serious games - games whose purpose transcends entertainment - have been used as a systemic practice to engage stakeholders in sustainability research (Flood et al.2018). Serious games can bridge the acts of know- ing and doing through testing and evaluating the impacts of different actions in a play- ful and safe environment that represents real-world contexts (Jean et al. 2018; Medema et al. 2016). However, the predefined nature of the game artifact fails to embrace the intrinsic uncertainties faced by stakeholders in their specific contexts. In recognition of this, we argue that there is a need to involve stakeholders in the design and prototyping of serious games, and to use this safe setting to pre-experience possible futures and devise appropriate responses.
In order to explore the potential benefits of a co-design approach to reconcile wicked sustainability issues, a serious game entitled SELECT ECOTECH was co-designed with stakeholders in three catchments in the Baltic Sea Region (BSR)—namely Fyrisån in Sweden, Słupia in Poland, and Vantaanjoki in Finland—as part of the BONUS RETURN project (www.bonusreturn.eu). The co-design process aimed to provide a safe
and inclusive learning space to support the selection and assessment of different constel- lations of eco-technologies (measures and actions) that could tackle both nutrient emis- sions and provide local co-benefits.
This paper presents the co-design process of SELECT ECOTECH and reflects on its potential as a systemic practice that can foster concerted action without imposing a con- vergence of perspectives. In so doing, the paper makes a methodological contribution to the existing body of knowledge on systemic practices to navigate wicked situations.
Furthermore, we hope to inspire and enrich the field of game design in sustainability research and practice.
Decision Making Processes within Water Governance
In the BSR, hydrological, water quality, and economic models are used as sensemaking tools to support the selection and assessment of the most appropriate measures to meet the goals defined by several EU environmental directives (European Parliament 2000).
The reductionist nature of these models exhibit limitations in representing the complex- ities underpinning nutrient enrichment in qualitative terms (Beven 2010). They are also unable to display value-laden assumptions and uncertainties (Van der Sluijs 2002), such as the interconnections between nutrient enrichment, health and extreme events on the populations living within basins (Bertone et al.2016). As models are generally calibrated to accommodate a limited number of dimensions, e.g. the emissions of nitrogen and phosphorus, they have difficulties in incorporating local practices which tend to be multifunctional and entail values which are often difficult to quantify. For instance, a cost-effectiveness analysis of nutrient mitigation measures conducted by Carolus et al.
(2020) puts forward that many of the measures suggested by local stakeholders to achieve multiple goals such as increased biodiversity, better connectivity for fish migra- tion, or improved flood mitigation were not included in the assessment due to the limi- tation of the hydrological model in simulating those measures. Critiques of models also point to their poor capacity to deal with uncertainty at many levels (Maier et al. 2016).
Aleatory uncertainty is inherent in the non-linear dynamics of change, which existing models struggle to represent as future system behavior shaped by multifaceted compo- nents that cannot be quantified or observed. Epistemic uncertainty stems from lack of knowledge and ambiguity caused by diverging and contested views about a given situ- ation, which challenges models’ ability to predict the impact of an intervention and determine a preferred course of action (Maier et al. 2016).
The critique of models is part of a wider debate on the legitimacy and credibility of science and the dominion of expert knowledge in supporting decision-making. It high- lights the need to transcend the boundary of science to include other knowledge gener- ation processes via the formation of an “extended peer community” (Saltelli, Vidoni, and Mascherini 2009). Such extended peer communities that go beyond scientists and policy actors can enrich scientific inquiry processes and enhance the rigor of scientific outputs and the legitimacy of policy decisions that are derived from them (Funtowicz and Ravetz 1993). This way, values and interests are evoked, and views on what consti- tutes a desirable action are diversified and potentially contested.
As a response to the proliferation of studies advocating for extended peer commun- ities in decision-making processes for improved water governance, the WFD stipulated that a broad cross-section of stakeholders should participate in the definition of the goals and actions underpinning River Basin Management Plans (RBMPs) in order to enable its systemic objective of improving ecosystem health. The approach involved both defining the system of interest (catchment/river basin) to be improved, developing a program of measures (PoMs), and an integrated monitoring system. Despite the WFD systemic aspirations, a comprehensive EU audit undertaken in 2016 concluded that the RBMPs, which form the basis for the Baltic Marine Environment Protection Commission’s (HELCOM’s) nutrient reduction targets, “lack ambition” as they focus only on ‘basic measures’ attributed to sector specific EU directives, and in particular, the Urban Wastewater Directive and the Nitrates Directive (European Court of Auditors 2016).
Despite a strong emphasis on participatory and systemic governance in sustainability pol- icy narratives, the reductionist implementation of the WFD still prevails and can be partly attributed to the preexisting institutional landscape of the implementation organizations.
Westberg and Powell (2015) uncovered that within these organizations, participatory approaches have a lower status than normal scientific methods. Those with “feminine attributes” (empathy and humility), normally women, were considered best placed within these organizations to manage the stakeholder consultation process; and those who were coded“masculine” (rational and objective), normally senior men, were generally considered best placed to facilitate the tradeoff processes. The dominating norms and structures of the local management agencies enabled the actors with “masculine” identities to retain their agency in terms of shaping policy implementation, such as the prioritization of the meas- ures in the PoMs. This example points at the distorting role of the power asymmetries manifest at the junction where knowledge is translated into governance actions.
Systemic Practice: Serious Games and Co-Design
Systemic practice explicitly focuses on managing and/or navigating wicked situations (Burns 2007; Ison 2008). A systems perspective recognizes that complex systems are highly interconnected, and enacting a desired intervention tends to transfer vulnerabilities from one group of stakeholders to another. Moreover, these situations are non-linear, whereby unexpected shocks and stressors can transform a desired inter- vention into a perverse one. As a response, systemic practice applies an iterative approach, in which the boundary circumscribing an inquiry into a problematic situation is an emerging property of an action learning process between those who have a stake in the intervention. In doing so the acts of knowledge production, knowledge transla- tion and implementation is collapsed into a seamless act of learning by doing.
Serious games represent real-world issues and capture complexity while offering stakeholders greater freedom to think outside the box to explore different pathways for sustainability transformation (Gugerell and Zuidema 2017; Jean et al. 2018). In light of this, they are increasingly being recognized within sustainability research and practice, as an approach to support the comprehension and improvement of wicked situations (Medema et al. 2019). They can also be considered as an creative action learning
approach that supports exploration and experimentation by harnessing diverse local knowledge and experience to determine appropriate actions (Rodela, Ligtenberg, and Bosma 2019; Salvini et al. 2016). In contrast to the high stakes that characterize real-world tradeoffs and consensus making processes, the inconsequential and playful serious game setting provides a safe space for stakeholders to reflect upon, challenge their existing beliefs, deliberate, negotiate, and resolve conflicts (Flood et al. 2018;
Rumore, Schenk, and Susskind 2016; Schulze et al. 2015). While there have been numerous studies focusing on the benefits of serious games (e.g. Flood et al. 2018), there is an absence of critique of the nature of the knowledge underpinning their devel- opment. In contrast to systemic practice, the boundary circumscribing the problematic situation and the solutions in most conventional serious games (e.g. those featured in Reckien and Eisenack (2013) review) is pre-coded into the game mechanics as prescrip- tive storylines and definitive learning outcomes. In order to couple pluralistic knowing with systemic doing, we argue that more focus should be given to enabling the partici- pation of stakeholders in the iterative process of game design, rather than on the play of the game artifact. Co-design goes beyond merely offering creative approaches to sci- ence-policy communication by opening up the opportunity to co-define problems and co-develop alternative solutions (Blomkamp 2018).
Co-design refers to the collective creativity of a group of stakeholders that is enacted across the whole design process (Sanders and Stappers 2008). This approach embraces stakeholders as “co-designers” who contribute with real-world knowledge, values, per- ceptions, and interests, which in turn will minimize the risk of failure due to blind spots, designer bias, or the misinterpretation of context-specific content (Ampatzidou and Gugerell 2019; Khaled and Vasalou 2014). More importantly, co-design challenges existing power structures manifested in the dominance of expert knowledge through the enactment of a learning environment that allows for a diversity in values and perspec- tives (Sanders and Stappers 2008).
Methods
Prior to inception of the research process it was agreed that a case study analysis across the BSR would be undertaken in order to gain contextual insights and engage decision- makers in governance learning. The three case study catchments provided real contexts that allowed for in-depth and multi-faceted co-enquiry of complex nutrient-related issues, which in turn supported the co-design of the serious game SELECT ECOTECH.
The catchments were selected owing to their similar size (1000–2000 km2) and the pres- ence of a variety of water dilemmas, e.g. nutrient pollution from agriculture and forests, urban runoff, recurrent floods and droughts, and climate change (Supplementary Appendix 1). A combination of hard and soft system methodologies were used to gain empirical understanding of the social-ecological properties of the catchments.
Hard System Methodologies
Hard system methodologies are underpinned by reductionist and expert modes of inquiry and approach water catchments as biophysical systems, with fixed or closed
boundaries, in which problems such as nutrient enrichment, can be addressed by engin- eering and biomonitoring. In order for these methods to elicit rigorous data, catchment properties need to exhibit linear and causal relationships (Pimm 1984). Whilst our framing of the case study catchments primarily does not lend itself to hard system methodologies, its deployment was motivated as a means to generate data in two instan- ces. The Soil and Water Assessment Tool (SWAT) was used to determine the baseline nutrient emissions under normal, low and high flow conditions (Koskiaho et al. 2020).
Additionally, SWAT was used to model the impact of eco-technologies if they have a known and predictable causal relationship with nutrient enrichment.
Soft Systems Methodologies
Soft Systems Methodologies (SSM) served as a meta-level framework to inform the co- design process. SSM are considered as pluralistic modes of systemic inquiry. From a SSM perspective, the properties of water catchments are considered to arise out of the nature of interactions between multiple and interdependent stakeholders (Ison 2010).
Here, the boundaries orchestrating the systemic inquiry are fluid, dynamic and negoti- able (Powell, Larsen, and van Bommel 2014). Under this tradition, problems are per- ceived as issues which cannot be solved, but rather can be managed or navigated by engaging stakeholders in issue formulation, co-inquiry, learning and concerted action.
The interactive nature of SSM demands substantial stakeholder dialogue, and at the out- set of this study the researchers who were experienced in applying SSM, were only con- versant in Swedish and English. Thus, at the beginning of the research process, local researchers from the Finnish and Polish catchments participated in a SSM workshop so they could support in facilitation of these methods (Supplementary Appendix 2).
Key informants and a snowballing approach were used to initiate the stakeholder iden- tification process. Key informants were the initial set of stakeholders, identified through the BONUS RETURN researchers’ preexisting networks, knowledge of the case study catchments, online desk research, and media screening. Additionally, many of the stake- holders we encountered served as informants for identifying new stakeholders, an approach referred to as snowballing (Colvin, Witt, and Lacey 2016). The co-design pro- cess was supported by a number of methods which are summarized inTable 1. The plat- forms for stakeholder interactions were hosted by a series of five workshops, six focus groups and eight interviews. Emergent insights from these sessions were recorded through photographic documentation and note-taking. After each session, the notes were reviewed, consolidated, and interpreted jointly by the research team. Rich pictures, board game prototyping and other interactive methods were used to mediate the co-inquiry process. Rich picture refers to a graphical technique which represents complex situations, problems, or concepts through the use of pictures, texts, symbols, and icons (Checkland and Scholes 1990). In SSM, rich pictures are considered to support the reframing of a wicked situation and serve as a basis for identifying additional stakeholders.
Board Game Prototyping
Board game prototyping can support a co-design process that facilitates learning, exploration, and experimentation (Ampatzidou and Gugerell 2019). Unlike digital game
prototypes, the rules underpinning a board game prototype are generally explicit and can be modified easily and rapidly, allowing for flexibility and swiftness that is desirable in iterative game-design processes. Another advantage of board game prototyping is its transparency, as it exposes players to various mechanics and data that create complex and dynamic situations (Castronova and Knowles 2015). The co-design sessions were designed as free-form activities with an initial rule set, rather than imposing a strictly standardized procedure on players (Deterding et al. 2013). In this regard, the act of playing, creating game content, and validating game mechanics were interwoven to allow for open and dynamic co-design sessions (Gugerell and Zuidema 2017).
In order to guide the co-design process, a statement was developed as part of the research proposal that describes the study’s system of interest (i.e. improved governance of Baltic Sea nutrient enrichment). In SSM, such a statement is referred to as a root defin- ition. The PQR formula is often used as a basis to develop the root definition (Ison2010);
P refers to (what) needs to be done; Q refers to (how) this transformation is undertaken;
and R refers to (why) this transformation is motivated. Our root definition consisted of:
P¼improved capacity to navigate the uncertainties associated with the trans- formation of the Baltic Sea environment
Q¼by means of a transdisciplinary approach (the co-design of the serious game) for identifying and piloting systemic eco-technologies
R¼in order to reduce nutrient emissions and generate co-benefits within other interconnected sectors.
In line with the root definition, a SSM conceptual framework entitled TWOCAGES, was actively drawn upon (Table 2).
Results –The Co-design Process of SELECT ECOTECH
This section describes the co-design process of SELECT ECOTECH, the purpose and outcome of each phase, and an explanation of how the stakeholders were involved as Table 1. The co-design process of SELECT ECOTECH and the role of the researchers and stakeholders.
Purpose Method
Role of the researchers
Role of the stakeholders/
co-designers Phase 1 Surfacing issues and
stakeholders
Workshops in the three case study contexts respectively
Document review and facilitating the rich picture exercise
Developing rich pictures and co-analyzing them with the researchers Phase 2 Co-design: selecting
constellations of measures and actions to generate multiple benefits and navigating epistemic uncertainty (conflicts of interest)
Two focus groups in the three case study contexts and a series of interviews in the Fyrisån case
Developing an initial game prototype Facilitating the
gameplay sessions
Playing with the prototype Populating the game
content during play Providing feedback on specific game features
Phase 3 Co-design: reconfiguring constellations of measures and actions to navigate aleatory uncertainty (socio- ecological shocks)
Two cross-case workshops
Facilitating the gameplay sessions Reviewing the co-
designers’feedback and fine-tuning the game prototype
Playing with the prototype Populating the game
content during play Providing feedback on specific game features
co-designers in each phase (Table 1). In addition, a description of the final game proto- type is provided as an outcome of the co-design process.
Phase 1: Surfacing Issues and Stakeholders
During the first stakeholder workshop held in each case study catchment, a group exercise was run in order to elicit rich pictures of the socio-ecological dynamics and issues within the catchments from the perspective of the workshop participants. The rich pictures appeared to mediate their systemic cognizance of the catchment. For instance, in the Słupia workshop, participants identified the relationship between poorly targeted fertilizer regimes, the lack of calcium usage, and the presence of intensive livestock production in the upper part of the catchment and downstream nutrient enrichment. However, although many of the participants recognized the interconnection between different forms of land- use and downstream nutrient enrichment, transforming the management of their catch- ment to reduce Baltic Sea nutrient enrichment was always aligned to their worldviews . For example, some of the worldviews that surfaced during the co-inquiry process included, the adaptation of catchment management to promote biodiversity, crop productivity, local eco- nomic development, the expansion of the aquaculture industry, and tourism. These insights served as a valuable input into understanding which co-benefits would be desirable to derive from a reconfigured regime of eco-technologies. At the conclusion of the workshops, we asked if any of the participants would be interested in participating in the co-design of SELECT ECOTECH. The response was overwhelming, and those who volunteered, were a diverse constellation of actors, owners and clients of a Baltic Sea marine environment under transformation. These stakeholders represented various sectors, e.g. wastewater, agri- culture, forestry, energy, aquaculture, environmental protection, and a multiplicity of func- tions, including local and regional authorities, academics, interest groups, advisory associations, and businesses (Supplementary Appendix 3).
Phase 2: Co-design: Selecting Constellations of Measures and Actions to Generate Multiple Benefits and Navigating Epistemic Uncertainty (conflicts of interest) During Phase 2 the co-inquiry process focused on surfacing desirable inputs and outputs of transformed catchment management by exploring the feasibility and impacts of different constellations of eco-technologies in agricultural, forestry, urban, and marine settings.
Thirty-six stakeholders from the three catchments participated in a series of focus groups.
Eight interviews were also conducted in the Fyrisån catchment to increase stakeholders’
Table 2. The TWOCAGES approach (adapted from Checkland and Poulter (2006)).
Description
T–Transformation What is the desired transformation? (A transformation includes both input and outputs) W–Worldview What are the values and ethical justifications for proposing the transformation?
O–Owner Who controls or owns the transformation? (Who can start or stop it?) C–Clients Who are the beneficiaries and victims of the transformation?
A–Actors Who carries out the activities required to achieve the transformation?
G–Guardians Who provides the insights into the unintended consequences of the transformation?
E–Environment What are the influences outside the control of the owners that may help or hinder the transformation?
S–System What is the conceptual model that leads to the transformation?
willingness to participate in the co-design process before the first round of focus groups took place. These interactions were designed to foster critical reflection on the controversies stakeholders faced in their respective contexts and surfaced numerous local innovations to address these dilemmas. One of the most salient controversies was the policy environment’s preexisting focus on targeting nutrient emissions from point sources, namely the waste- water sector rather than the diffuse source from the agriculture sector which contributes most of the loads to the Baltic Sea. This has led to an overt focus on costly one- dimensional technical solutions, rather than multifunctional approaches. Stakeholders dis- cussed the difficulties of capturing diffuse emissions from the multiple sources in their catchment, and rather than reproducing the status quo, they suggested recycling the nutrients downstream at their sink. Growing out of this discussion they proposed utilizing nutrients through the farming of microalgae, mussels and seaweed in the Baltic Sea for fuel and food. The recycling of nutrient-rich sewage sludge in agriculture was also taken up as a controversial issue during the workshops. Its application is a cost-efficient alternative to the widespread use of synthetic fertilizers, and moreover, the presence of organic matter in sludge leads to a reduction of nutrient leakage. However, farmers are reticent about its usage on account of low consumer acceptance for produce fertilized by sewage sludge.
Insights from the focus groups further supported the initial design of the board game prototype. This included the incorporation of different forms of land-use (agriculture, for- estry, urban and marine settings), the transport of their respective nutrient loads through the water flows to the Baltic Sea, and the presence of an initial set of eco-technologies to entrap, redirect or recycle nutrients within or outside the immediate catchment.
These insights were incorporated into the first iteration of the SELECT ECOTECH prototype, and in a second round of focus groups it was playtested as a means to mediate a co-inquiry into eco-technology preferences and development interventions, and the sys- temic consequences of this transformation. The prototype consisted of a simple game board representing different forms of land use in interconnected hexagonal tiles that circum- scribed the Baltic Sea, and a deck of cards for eco-technologies and development interven- tions. A player’s objective was to both reduce their emissions of nutrients into the Baltic Sea, and increase the development potential of their discrete systems of interest (land-use type). During the playtesting, stakeholders developed constellations of synergistic eco- technologies and development interventions aimed at providing multiple benefits within their systems of interest (Supplementary Appendix 4). Blank cards were also provided so they could introduce new eco-technologies and development interventions during the course of the play. The prototyping supported deliberations over the efficiency of discrete versus constellations of eco-technologies and development interventions in reducing emis- sions and providing co-benefits. Moreover, players became attentive to emergent conflicts at catchment level resulting from selecting development interventions that generated nutri- ent emissions that flowed into other players’systems of interest.
Phase 3: Co-design: Reconfiguring Constellations of Measures and Actions to Navigate Aleatory Uncertainty (Socio-Ecological Shocks)
The aim of Phase 3 was to incorporate an acknowledgement of the E - Environment (influences outside the control of the owners that may help or hinder the
transformation) within TWOCAGES in the co-design process, by introducing socio- ecological shocks into the second version of the game prototype. An interactive cross- case workshop was held in Uppsala, Sweden, with stakeholders who had a deep understanding of the impact of aleatory uncertainty associated with biophysical shocks in their respective catchments. During the playtesting they explored how different con- stellations of eco-technologies performed under the post-normal conditions precipitated by floods, drought, fire, and biosecurity related threats. For example, in the absence of aleatory uncertainty (normal conditions), during the prototyping in Phase 2, players dis- covered that nutrient retention ponds were a cost-effective measure in the upper reaches of water bodies, that both served to reduce emissions, and provide nutrient-rich sludge that could be used as a fertilizer or input for local biogas production. However, during the prototyping in Phase 3 players discovered that during an extreme flood (post-normal conditions), the retention ponds no longer served as a nutrient trap and were transformed into a point source for nutrient emissions. The insights from this ses- sion were consolidated to support the continuous improvement and validation of SELECT ECOTECH as a conceptual model to bridge understanding and action to navigate aleatory uncertainty.
The third version of the game prototype supported a second cross-case workshop in Helsinki, Finland, by mediating a co-inquiry into the barriers and possibilities posed by the existing socio-political environment to the implementation of eco-technologies and development interventions in the catchment. For instance, changes in the chemical and environmental legislation would affect what kind of products that could be applied on farmland. In Finland, the popularity of sludge application varies in different places.
Sludge is not permitted as an external input in organic farming due to lack of informa- tion about its contents such as antibiotics, and chemical substances. In Sweden, there has been discussion on a complete ban on spreading sewage sludge on farmland or a partial ban under stricter quality requirements as a result of the Swedish government inquiry conducted during 2018–2020. Such prohibition would arguably increase favor of incineration and require major investments in alternative technologies that allow for phosphorus recycling from sludge at scale. At the EU level, concern was raised by farm- ers about potential cuts in subsidies due to the shrinking Common Agricultural Policy (CAP) budget as a consequence of Brexit, while having even more ambitious goals in terms of environmental and climate action and flowing from this, stricter mandatory requirements. The new rules on fertilizers recently adopted by the EU, e.g. tightening the limits for cadmium content in phosphate fertilizing products would substantially affect farmers’choice and costs of using fertilizers. On the bright side, there was a posi- tive reception to the future revisions in CAP’s governance model, which promises to shift from rules and compliance toward results and performance. A list of social, market and policy changes emerging from the workshop discussion was included as system shocks in SELECT ECOTECH (Supplementary Appendix 5).
Description of SELECT ECOTECH
The co-design process resulted in a final board game prototype, which can be consid- ered as an artifact of the co-design process and the S - System (conceptual model that
leads to the transformation) within TWOCAGES. The board game is a turn-based strat- egy game between two players or two teams and requires a facilitator to maintain the flow of the gameplay, and guide the discussion throughout the play session. It is envisaged that the game could be used by a wide range of actors from governmental agencies, the private sector, interest groups and civil society organizations to deliberate over the selection and implementation of measures and actions that address nutrient management, and other interconnected issues. The game materials in SELECT ECOTECH consist of a main board, six hexagon tiles that represent three different land-use systems (two each for agriculture, forestry, and urban areas), a number of game tokens, including eco-technology (white), development (gray), resources (green), and emissions (red) tokens, and decks of cards for eco-technologies, developments and system shocks respectively (Figure 1).
The players’ objective in SELECT ECOTECH is threefold: (1) to reduce emissions from their respective land-use systems; (2) to increase the productivity of these sys- tems; and (3) to increase the adaptive capacity of their land-use systems. Players must use resources to purchase both eco-technologies and development interventions to achieve these objectives. When deploying eco-technologies and development Figure 1. Setup of the board game SELECT ECOTECH.
interventions, players navigate the uncertainty associated with the checks and balan- ces from a dynamic socio-ecological environment and the conflicts of interest with other players.
Each player or team is given a number of resource tokens at the start of the game. At the beginning of each round, players throw the dice to determine how many eco- technologies and/or development interventions they can purchase with their given resources. On the one hand, players reduce emissions by investing in eco-technologies but decrease their resources due to the costs of purchase. On the other hand, they increase emissions but gain more resources by placing development interventions on the board. Players gain bonus resources through compiling synergistic constellations of eco-technologies and developments. Aleatory uncertainty is introduced in the game through a deck of “shock” cards. At the end of each round, a random “shock” card from the given deck is drawn, which would affect both teams’ eco-technologies and developments. For example, extreme flooding could render certain eco-technologies such as constructed wetlands, buffer strips less effective in reducing emissions to the Baltic Sea or cripple the tourist activities. Similar to eco-technologies and developments, new shocks can be written on “blank cards” and introduced into the game. At the end of the game, players calculate the socio-ecological performance of their respective catch- ment with their remaining resources and emissions. The remaining emissions are to be removed using a resource-emission ratio agreed upon by the players (e.g. It takes five resources to remove one emission). Having removed all the emissions on the board, the player who ends up with the most resources wins the game.
The co-design process also resulted in a digital game prototype featuring a functional
“alpha” version that utilized data generated from the playing of the board game prototype. It is a digitized interface that provides additional tracking capacity and the ability to test and evaluate a multiplicity of constellations of eco-technologies and devel- opment interventions. The digital version is accessible at https://zygodact.itch.io/moni- tor-eco-tech, allowing for its extended use beyond the stakeholder group involved in the co-design process.
Discussion
Rethinking Decision Support under Wicked Situations
Insights from the co-design process revealed the shortcomings of modeling for decision sup- port in the face of wicked sustainability issues. This was particularly apparent in situations characterized by epistemic uncertainty. At the outset we intended to use the data generated by the SWAT model as input into the co-design process of SELECT ECOTECH. However, of the 35 eco-technologies identified by our stakeholders, only five eco-technologies, i.e. con- structed wetlands, retention ponds, buffer strips, catch crops, and crop rotation, exhibited a known causal relationship with nutrient emissions that allowed them to be modeled in SWAT. Furthermore, while our aim was to support the development of synergistic constella- tions of eco-technologies to tackle both nutrient emissions and a host of other co-benefits, SWAT could only model one eco-technology at a time, and therefore calculated the total load of a constellation by aggregating their performance. Thus, the model’s reductionist
operating system did not enable systemic insights into the synergies or conflicts between the eco-technologies comprising a constellation.
Shortcomings were also identified in the SWAT model’s decision support in situa- tions characterized by a high degree of aleatory uncertainty. The performance of differ- ent measures in reducing the emissions of nutrients from catchments is generally applied to calibrate models at so-called normal loading rates. However, several empirical studies suggest that nearly all nutrient export occurs at ratesnormal loading (Royer, David, and Gentry 2006), for which models are not calibrated. Aleatory uncertainty is likely to deepen with climate change, and projections suggest that the frequency of peak flows will increase by more than 25% in parts of the Baltic Sea basin (Bartosova and Capell 2018).
Unlike the rigid structures of models, the iterative and open structures of the co- design process made it possible to develop responses even when faced with uncertainties inherent in wicked situations. While models are a “black box” filled with assumptions and calculations that most stakeholders are alienated from, the co-design of the serious game, using prototyping materials, was much more transparent and accessible, which aided stakeholder dialogue and learning (see also Lankford and Craven (2020)). In response to incomplete data sets from modeling on account of epistemic and aleatory uncertainty, the performance of new constellations of eco-technologies were assessed using proxy values co-produced by different groups of stakeholders (see also Quinn et al. (2021) use of indirect proxies to measure community resilience). The use of co-pro- duced proxy values in the co-design incorporated different ways of knowing in deci- sion-making processes, rather than over-relying on scientific expertise. These values stemmed from the embodied knowledge of stakeholders and were deliberated over in the co-design of SELECT ECOTECH. Most stakeholders involved in the co-design wel- comed the opportunity to “think outside the box” (a “box” which is normally defined by scientists and policy actors) through an appreciation of multiple perspectives and values. Suskevics, Hahn, and Rodela (2019) suggest that reflection through self-emergent experiences facilitates action learning which in turn can inform future policies.
Feedback from our stakeholders indicated that implementing concerted actions in the co-design process supported their learning as the basis for future decision-making.
Co-design of Serious Games as an Innovative Method to Foster and Enact Actions That Attend to Multiple Benefits
The non-scripted approach enacted in the co-design of SELECT ECOTECH triggered meaningful discussions on nutrient-related issues and management actions in local con- texts. Stakeholders were invited not only to reflect and bring their own real-world experience into constructing the serious game but also to “pre-experience alternative futures” (Bengston 2019) and devise appropriate responses. Within the game setting, players were faced with challenging situations referred to as “system shocks”, such as flooding, drought, policy reform, changes in market conditions, or public awareness.
The players responded by devising constellations of eco-technologies that had the cap- acity to adapt to shocks while best serving their multiple interests. The safe space, evoked by playfulness of the co-design process, allowed stakeholders to recognize others’
perspectives, underlying goals and values, openly share, and discuss why barriers to the implementation of certain eco-technologies persisted or were amplified in their own contexts. It fostered the opportunity to learn about how these situations could be trans- formed through practices that have the adaptive capacity to navigate wicked situations.
This further highlights that our study was not directed at predicting the future state of a system like traditional models do, but rather developing a continuously evolving framework (embodied in the serious game prototypes) through co-design that supports stakeholder co-inquiry into how to manage the system in relation to future uncertainties (see also Souchere et al. (2010)).
Many participatory approaches are rooted in the notion of building consensus and fostering collective action (Burns 2007). Given the wicked nature of nutrient enrich- ment, i.e. the presence of competing interests and unequal power relations, it may be impossible to reach a solution that can satisfy all the different parties. Thus, many par- ticipatory processes often end up reinforcing solutions that favor the perspective of those in power. Rather than relying on reified scientific narratives, the co-design of SELECT ECOTECH allowed for the coexistence of diverse perspectives and actions in response to multifaceted nutrient-related issues. In this process, all the different meas- ures proposed by stakeholders were included in the serious game prototype for deliber- ation during the gameplay without the condition that they were to display measurable and quantitative characteristics, as found in reductionist models. Hence, tradeoffs or consensus was not necessary before measures could be enacted in the game setting. We also observed that none of the stakeholders appeared to promote their agency or pos- ition, but were rather preoccupied with the concerted act of co-designing. Moreover, by way of the iterative process of generating diverse actions and learning from the systemic consequences of these actions, the status quo in nutrient management was challenged.
This allowed for new possibilities to improve systems understanding and identify multi- functional solutions to complex issues. During the co-design process, we observed that the game content and mechanics increasingly took shape as a result of stakeholder par- ticipation. More importantly, during the playtesting, stakeholders were empowered to evaluate and modify the explicit assumptions and values underpinning the game proto- type. In so doing, our study resonates with Jessen, Mirkovic, and Ruland (2018) that co-design can serve as a vehicle to enable meaningful participation of stakeholders.
Although co-design was seen as a purposeful means to engage stakeholders, there was a notable limitation in our approach. Due to its experimental and novel characteristics, we were strongly reliant on stakeholders’ enthusiasm and voluntary participation as co- designers. This implies that some other important perspectives may have been excluded from the co-design process, and those who were part of the co-design process might have been more aligned in terms of their interests, values and positions.
Conclusion
Our study suggests that the co-design of serious games can serve as a systemic practice to navigate “wicked” contexts. Here, the prescriptive and linear way of transferring knowledge, that characterizes traditional science-informed reductionist policy, gives way to an emergent and continuous process of crossing knowledge boundaries through the
act of co-designing. The co-design approach can be understood as an alternative to the inevitable tradeoffs inherent in collective action in the face of “wicked” sustainability issues, as it fosters self-organization as a premise for concerted action without imposing a convergence of perspectives among stakeholders. In doing so, consequential actions can be tested and evaluated in the safe game environment where diverse forms of knowledge and ways of knowing and acting are less distorted by power asymmetries.
In recognition of the shortcomings of normal scientific approaches in reconciling wicked situations, the co-design of serious games surfaced co-produced proxy data embodied in multiple ways of knowing. Inherent in this approach was a shift in the mode of knowledge production and the distribution of agency from a group of experts to an extended peer group. The playfulness of the game co-design process fostered the equitable participation of diverse stakeholders in the implementation of governance actions. Our findings suggest that the constellation of measures that emerged through their enactment in the co-design process are potentially more legitimate and adaptive under wicked conditions, as they embody a multi-dimensional narrative which does not need to be collapsed by way of a tradeoff process to allow for their implementation. As such, the co-design approach outlined in this paper suggests that it could be applicable in many different natural resource management contexts.
Acknowledgments
This work is part of the BONUS RETURN project (www.bonusreturn.eu).
Disclosure statement
No potential conflict of interest was reported by the author(s).
Funding
BONUS RETURN has received funding from BONUS (Art 185), funded jointly by the EU and Formas, a Swedish Research Council for Sustainable Development; Sweden’s Innovation Agency, Vinnova; The Academy of Finland; and the National Centre for Research and Development in Poland. Furthermore, our work with the serious game received substantial contributions from 36 stakeholders in the three case study settings in Sweden, Finland, and Poland. This work was also supported by European Commission; Svenska Forskningsrådet Formas.
ORCID
Neil Powell http://orcid.org/0000-0002-8665-2370 Thao Do http://orcid.org/0000-0002-5353-8918
References
Ampatzidou, K. Gugerell. 2019. Participatory game prototyping–balancing domain content and playability in a serious game design for the energy transition. CoDesign15 (4):345–60. doi:10.
1080/15710882.2018.1504084.
Bartosova, R. Capell. 2018. Baseline and climate change scenarios for the whole Baltic Sea region. BONUS MIRACLE project. Deliverable 2.5.http://www.bonus-miracle.eu.
Bartosova, A., R. Capell, J. E. Olesen, M. Jabloun, J. C. Refsgaard, C. Donnelly, K. Hyyti€ainen, S.
Pihlainen, M. Zandersen, and B. Arheimer. 2019. Future socioeconomic conditions may have a larger impact than climate change on nutrient loads to the Baltic Sea.Ambio 48 (11):1325–36.
doi:10.1007/s13280-019-01243-5.
Bengston, D. N. 2019. Futures research methods and applications in natural resources. Society &
Natural Resources32 (10):1099–113. doi:10.1080/08941920.2018.1547852.
Bertone, E., O. Sahin, R. Richards, and A. Roiko. 2016. Extreme events, water quality and health:
A participatory Bayesian risk assessment tool for managers of reservoirs. Journal of Cleaner Production135:657–67. doi:10.1016/j.jclepro.2016.06.158.
Beven, 2010.Environmental modelling: An uncertain future?Boca Raton, Florida: CRC Press.
Blomkamp, E. 2018. The promise of co-design for public policy. Australian Journal of Public Administration77 (4):729–43. doi:10.1111/1467-8500.12310.
Burns, D. 2007. Systemic action research: A strategy for whole system change. Bristol, UK: Policy Press.
Carolus, J. F., A. Bartosova, S. B. Olsen, S. Jomaa, A. Veinbergs, A. Zilans, S. M. Pedersen, G.
Schwarz, M. Rode, and K. Tonderski. 2020. Nutrient mitigation under the impact of climate and land-use changes: A hydro-economic approach to participatory catchment management.
Journal of Environmental Management271:110976. doi:10.1016/j.jenvman.2020.110976.
Castronova, E., and I. Knowles. 2015. Modding board games into serious games: The case of cli- mate policy.International Journal of Serious Games2 (3):41–62.
Checkland, P., and J. Poulter. 2006. Learning for action: A short definitive account of soft systems methodology and its use for practitioner, teachers, and students. Chichester: Wiley
Checkland, P., and J. Scholes. 1990.Soft systems methodology in action. Chichester: Wiley.
Coffey, C., and S. Richartz. 2003. The EU habitats directive: Generating strong responses. Berlin:
Institute for European Environmental Policy.
Colvin, R. M., G. B. Witt, and J. Lacey. 2016. Approaches to identifying stakeholders in environ- mental management: Insights from practitioners to go beyond the “usual suspects. Land Use Policy52:266–76. doi:10.1016/j.landusepol.2015.12.032.
Deterding, S., S. L. Bj€ork, L. E. Nacke, D. Dixon, and, and E. Lawley. 2013. Designing gamifica- tion: Creating gameful and playful experiences. In CHI’13 Extended abstracts on human factors in computing systems; April, 2013, 3263–3266. New York: Association for Computing Machinery.
Dewulf, A., M. Craps, R. Bouwen, T. Taillieu, and C. Pahl-Wostl. 2005. Integrated management of natural resources: Dealing with ambiguous issues, multiple actors and diverging frames.
Water Science and Technology: Technology52 (6):115–24. doi:10.2166/wst.2005.0159.
European Court of Auditors. 2016.Combating eutrophication in the Baltic Sea: Further and more effective action needed. Special Report No 03, 2016.Luxembourg: Publications Office.
European Parliament. 2000. Directive 2000/60/EC of the European Parliament and of the Council of 23 October 2000 establishing a framework for community action in the field of water policy.
Official Journal of the European Communities22 (12):2000.
Flood, S., N. A. Cradock-Henry, P. Blackett, and P. Edwards. 2018. Adaptive and interactive cli- mate futures: Systematic review of “serious games” for engagement and decision-making.
Environmental Research Letters13 (6):063005. doi:10.1088/1748-9326/aac1c6.
Funtowicz, S. O., and J. R. Ravetz. 1993. The emergence of post-normal science. In Science, Politics and Morality, ed. R. Von Schomberg, 85–123. Dordrecht: Springer.
Gugerell, K., and C. Zuidema. 2017. Gaming for the energy transition. Experimenting and learn- ing in co-designing a serious game prototype. Journal of Cleaner Production 169:105–16. doi:
10.1016/j.jclepro.2017.04.142.
Head, B. W., H. Ross, and J. Bellamy. 2016. Managing wicked natural resource problems: The collaborative challenge at regional scales in Australia. Landscape and Urban Planning 154:
81–92. doi:10.1016/j.landurbplan.2016.03.019.
Ison, R. 2010. Systems practice: How to act in a climate change world. London, UK: Springer Science & Business Media.
Ison, R. L. 2008. Systems thinking and practice for action research. In The Sage handbook of action research participative inquiry and practice (2nd Edition), ed. P. W. Reason and H.
Bradbury, 139–58. London, UK: Sage Publications.
Jean, S., W. Medema, J. Adamowski, C. Chew, P. Delaney, and A. Wals. 2018. Serious games as a catalyst for boundary crossing, collaboration and knowledge co-creation in a watershed gov- ernance context. Journal of Environmental Management 223:1010–22. doi:10.1016/j.jenvman.
2018.05.021.
Jessen, S., J. Mirkovic, and C. M. Ruland. 2018. Creating gameful design in mHealth: A participa- tory co-design approach.JMIR mHealth and uHealth6 (12):e11579. doi:10.2196/11579.
Khaled, R., and A. Vasalou. 2014. Bridging serious games and participatory design. International Journal of Child-Computer Interaction2 (2):93–100. doi:10.1016/j.ijcci.2014.03.001.
A. T., Knight, C. N. Cook, K. H. Redford, D. Biggs, C. Romero, A. Ortega-Argueta, C. D.
Norman, B. Parsons, M. Reynolds, G. Eoyang, et al. 2019. Improving conservation practice with principles and tools from systems thinking and evaluation. Sustainability Science 14 (6):
1531–48. doi:10.1007/s11625-019-00676-x.
Koskiaho, J., T. Okruszko, M. Piniewski, P. Marcinkowski, S. Tattari, S. Johannesdottir, E.
K€arrman, and M. K€am€ari. 2020. Carbon and nutrient recycling ecotechnologies in three Baltic Sea river basins–the effectiveness in nutrient load reduction. Ecohydrology & Hydrobiology 20 (3):313–22. doi:10.1016/j.ecohyd.2020.06.001.
Lankford, B. A., and J. Craven. 2020. Rapid games designing; constructing a dynamic metaphor to explore complex systems and abstract concepts. Sustainability 12 (17):7200. doi:10.3390/
su12177200.
Maier, H. R., J. H. Guillaume, H. van Delden, G. A. Riddell, M. Haasnoot, and J. H. Kwakkel.
2016. An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together? Environmental Modelling & Software 81:154–64. doi:10.1016/j.envsoft.2016.
03.014.
Medema, W., A. Furber, J. Adamowski, Q. Zhou, and I. Mayer. 2016. Exploring the potential impact of serious games on social learning and stakeholder collaborations for transboundary watershed management of the St. Lawrence River Basin. Water 8 (5):175. doi:10.3390/
w8050175.
Medema, W., I. Mayer, J. Adamowski, A. E. J. Wals, and C. Chew. 2019. The potential of serious games to solve water problems: Editorial to the special issue on game-based approaches to sus- tainable water governance.Water11 (12):2562. doi:10.3390/w11122562.
Paquet, G. 1999.Governance through social learning. Ottawa, Canada: Les Presses de l’Universite d’Ottawa [University of Ottawa Press].
Pimm, S. L. 1984. The complexity and stability of ecosystems. Nature (5949)307:321–6. doi:10.
1038/307321a0.
Powell, N., K. R. Larsen, A. de Bruin, S. Powell, and C. Elrick-Barr. 2017. Water security in times of climate change and intractability: Reconciling conflict by transforming security concerns into equity concerns.Water9 (12):934. doi:10.3390/w9120934.
Powell, N., R. K. Larsen, and S. van Bommel. 2014. Meeting the “Anthropocene” in the context of intractability and complexity: Infusing resilience narratives with intersubjectivity. Resilience 2 (3):135–50. doi:10.1080/21693293.2014.948324.
Quinn, T., W. N. Adger, C. Butler, and K. Walker-Springett. 2021. Community resilience and well-being: an exploration of relationality and belonging after disasters.Annals of the American Association of Geographers(2)111:577–14. doi:10.1080/24694452.2020.1782167.
Reckien, D., and K. Eisenack. 2013. Climate change gaming on board and screen: A review.
Simulation & Gaming44 (2–3):253–71. doi:10.1177/1046878113480867.
Rittel, H. W., and M. M. Webber. 1973. Dilemmas in a general theory of planning. Policy Sciences4 (2):155–69. doi:10.1007/BF01405730.
Rodela, R., A. Ligtenberg, and R. Bosma. 2019. Conceptualizing serious games as a learning-based intervention in the context of natural resources and environmental governance. Water11 (2):
245. doi:10.3390/w11020245.
Royer, T. V., M. B. David, and L. E. Gentry. 2006. Timing of riverine export of nitrate and phos- phorus from agricultural watersheds in Illinois: Implications for reducing nutrient loading to the Mississippi River. Environmental Science & Technology 40 (13):4126–31. doi:10.1021/
es052573n.
Rumore, D., T. Schenk, and L. Susskind. 2016. Role-play simulations for climate change adapta- tion education and engagement. Nature Climate Change 6 (8):745–50. doi:10.1038/
nclimate3084.
Saltelli, A., D. Vidoni, and, and M. Mascherini. 2009. Recommended practices in global sensitiv- ity analysis. In Uncertainties in Environmental Modelling and Consequences for Policy Making, ed. P. Baveye, J. Mysiak, and M. Laba, 183–202. Dordrecht: Springer.
Salvini, G., A. van Paassen, A. Ligtenberg, G. C. Carrero, and A. K. Bregt. 2016. A role-playing game as a tool to facilitate social learning and collective action towards climate smart agricul- ture: Lessons learned from Apuı, Brazil. Environmental Science & Policy 63:113–21. doi:10.
1016/j.envsci.2016.05.016.
Sanders, E. B.-N., and P. J. Stappers. 2008. Co-creation and the new landscapes of design.
Codesign4 (1):5–18. doi:10.1080/15710880701875068.
Schulze, J., R. Martin, A. Finger, C. Henzen, M. Lindner, K. Pietzsch, A. Werntze, U. Zander, and R. Seppelt. 2015. Design, implementation and test of a serious online game for exploring complex relationships of sustainable land management and human well-being. Environmental Modelling & Software65:58–66. doi:10.1016/j.envsoft.2014.11.029.
Souchere, V., L. Millair, J. Echeverria, F. Bousquet, C. L. Page, and M. Etienne. 2010. Co-con- structing with stakeholders a role-playing game to initiate collective management of erosive runoff risks at the watershed scale. Environmental Modelling & Software25 (11):1359–70. doi:
10.1016/j.envsoft.2009.03.002.
W., Steffen, K. Richardson, J. Rockstr€om, S. E. Cornell, I. Fetzer, E. M. Bennett, R. Biggs, S. R.
Carpenter, W. de Vries, C. A. de Wit, et al. 2015. Planetary boundaries: guiding human devel- opment on a changing planet.Science 347 (6223):1259855. doi:10.1126/science.1259855.
Suskevics, M., T. Hahn, and R. Rodela. 2019. Process and contextual factors supporting action- oriented learning: A thematic synthesis of empirical literature in natural resource management.
Society & Natural Resources32 (7):731–50. doi:10.1080/08941920.2019.1569287.
Van der Sluijs, J. P. 2002. A way out of the credibility crisis of models used in integrated envir- onmental assessment.Futures34 (2):133–46. doi:10.1016/S0016-3287(01)00051-9.
Westberg, L., and S. Powell. 2015. Participate for women’s sake? A gender analysis of a Swedish collaborative environmental management project. Society & Natural Resources28 (11):1233–48.
doi:10.1080/08941920.2015.1014594.
