The urge to improve governance has been, and remains, unrelenting (Khademian 2012). Underlying many of these efforts is the belief that governance can be im-proved by generating and making use of relevant information to guide decision-making. Alternatively praised as evidence-based policymaking or critiqued as the ritualistic use of “data” to give the appearance of modern management (Dahler-Larsen 2012), this perceived value of feedback requires faith that the evidence can support value judgments of what constitutes “improved” performance and
“better” outcomes. To consider in what ways this faith is justified, this section introduces some of the value judgments made regarding public initiatives, consid-ers their contexts, and argues for being strategic in how we judge value (Julnes 2010).
38 Acta Wasaensia
Goals of Valuing in the Public Interest
To reach judgments about the value of public initiatives, for example that linking teacher pay to student achievement test scores leads to better education, we en-gage in valuation. What is valuation and how is it to help improve our society?
Valuation in Society. The term valuation is defined as being both the process of valuing and the estimated value that follows from this process: “1. the act or pro-cess of valuing; specifically : appraisal of property; 2. the estimated or deter-mined market value of a thing; 3. judgment or appreciation of worth or character”
(Merriam-Webster, online). Whereas the definition emphasizes the market value of property or other holdings, it can apply to any judgment of worth, including judgments about which approaches to governance are better and which outcomes are indicative of a better society. As such, we engage in valuation as a process in preparing evaluation reports, with the final judgment of value (e.g., “the net bene-fit of building the road is $4 million”) being a valuation as an estimate.
Process and Outcome Goals for Valuation. The logic of engaging in valuation is that evaluative evidence can help improve decision-making processes (Henry &
Julnes, 1998). Indeed, all efforts to assess the value of public policies and pro-grams are predicated on the idea that awareness of estimated values will lead to improved governance processes and, as a result, to “better” decision-making.
This is the process goal for valuation, but we rarely assess whether such attempts at evidence-based decision making really lead to a better society, and, further, we do not have standard procedures for how this process goal for valuation is to be assessed.
One approach to valuing different approaches to governance is to assess the ex-tent to which the approaches lead to valued outcomes for society (Moore, 1995).
In thinking about what constitutes a valued outcome, consider the above example of linking teacher pay to student achievement test scores, with the value of such a policy judged by whether it leads to higher test scores. These higher test scores are valuable to society only if the higher test scores lead to higher graduation rates and later, perhaps, with greater career preparation, higher incomes, and even a more productive citizenry. The analogous argument is that evaluating and reward-ing public managers for achievreward-ing proximal outcomes (e.g., better informed wel-fare recipients) is of little value if the services provided do not lead to more distal outcomes, such as addressing citizen needs (e.g., information sessions are effi-cient and effective in improving recipient understanding of welfare policies but do not help recipients become prepared for leaving public assistance). Similarly, finding that implementation of a performance management system leads to
evi-dence-based decision-making is valuable only if the organizational outcomes lead to the more distal outcomes of citizens being better served.
This limitation of many proximal outcomes is why many evaluators include out-comes like “happiness” (Mohr 1995) or “social betterment” (Mark, Henry, & Jul-nes 2000; Weiss 1998) as the ultimate intended outcomes (on the right side of simple outcome lines) of public initiatives. However, including ultimate outcomes such as advancing social betterment simply begs the question of what constitutes
“better,” or what we would call improvement. How can we best make sense of what is better; how do we judge the value to the public of available options?
Contexts of Valuing
Answering the question of how we should try to make sense of the value of public policies and programs requires some sense of context, with the expectation that different methods are appropriate for different contexts (Julnes & Rog 2007). This section presents two quite different contexts as examples and then considers some of the major differences in these contexts.
Two Contrasting Examples of Public Valuing. Consider a local government agency responsible for managing traffic in the city. As the city has grown, traffic congestion has increased to the point of being an issue in local elections. This agency, a city department of transportation, is, therefore, committed to “improv-ing” the traffic flow in the city, with a particular focus on the areas with greatest congestion that are being reported on by the local news outlets. Relevant valua-tion quesvalua-tions, then, are “what are the ‘best’ uses of limited resources?” and, when the selected projects are completed, “are the resulting traffic improvements
‘worth’ the resources required and any other negative consequences entailed?”
For the purpose of this chapter, let us assume that a needs assessment was com-pleted and one project was adding a left-turn signal at a busy intersection that has been congested at rush hour due to all of the people needing to wait to make a left turn. Was the public interest served by this initiative?
Consider also a very different valuing context, a federal agency responsible for welfare policy (in the U.S., this is the Department of Health and Human Services, or HHS) has implemented a 60-month lifetime limit on the number of months of welfare cash assistance (in the U.S., Temporary Assistance for Needy Families, or TANF) any individual may receive. Five years after implementation, the agency (specifically, in HHS, and Office of the Assistant Secretary for Program Evalua-tion, or ASPE) needs to evaluate this policy change. Was the public interest served by this initiative?
40 Acta Wasaensia
Implications of Context for Strategic Valuation. Although these two examples may seem extreme in their differences, they highlight the many differences be-tween a valuing context in which the decision is largely a technical matter (which interventions have the greatest impact on traffic congestion), and a context which is saturated with cultural and political considerations. As a result, there is greater clarity regarding the values to be considered in evaluating the changes in traffic policies, and greater consensus on the appropriateness of focusing on efficiency as the goal to be maximized. If the left-turn signal can be represented as reducing rush hour wait-times at the intersection by an average of five minutes, and at a reasonable cost, it will be judged by most as serving the public interest.
In contrast, judging whether welfare reform served the public interest will result in more debates about the goals of welfare reform and about whether technical analyses of benefits and costs can suffice in giving voice to the perspectives of welfare participants. As such, people are more likely to trust the conclusions of experts in the left-turn example; people affected by welfare reform are more like-ly to have a mix of contrasting values that they use in judging the overall value of a public initiative.
This recognition of the implications of the differences in the two examples high-lights the need to align methods of valuing with their contexts (Julnes, 2012a). A technical analysis of a single outcome (e.g., reduction in traffic congestion) con-ducted by a small group of experts (analysts in the transportation department) can be adequate for judging the value of a local decision about traffic regulations but not for judging the value of welfare. Aligning activities with contexts is the focus of strategy in organizational theory and so it is also here in arguing for strategic approaches to valuation. Developing frameworks to support strategic use of for-mal valuation methods requires understanding the options available. For this, the following section develops a standard model of the steps involved in essentially all approaches to assisting our efforts to value alternatives and then considers var-iations in how those steps are typically accomplished.