Copyright © 2015 by the author(s). Published here under license by The Resilience Alliance.
Go to the pdf version of this article
The following is the established format for referencing this article:
Milne, K. M. G. 2015. Can sense-making tools inform adaptation policy? A practitioner’s perspective. Ecology and Society20(1): 66.
Can sense-making tools inform adaptation policy? A practitioner’s perspective
Kyla M. G. Milne 1
1Nova Scotia Environment
ABSTRACTAs governments struggle to find solutions to complex problems like climate change, policy makers look for tools that can capture complexity and elicit insight. I explored the application of one such tool, known as “SenseMaker,” in helping Canadian policy makers understand the factors that enable or hinder climate change adaptation in Canada. I have reflected on the usefulness of SenseMaker and of a multiperspective, multimethod approach to investigating perceptions and experiences of adaptation. The challenges and advantages of applying this analysis in government were explored, and data findings assessed for their impact on policy. Findings showed that although the approach has promise, further work and testing are needed before sense-making approaches support adaptation policy.
Key words: climate adaptation; complexity; SenseMaker; sense making; social psychology; wicked problems
INTRODUCTIONI explore from a government practitioner’s perspective whether sense-making studies can inform adaptation policy. As a Canadian climate adaptation policy analyst and the only nonacademic to participate in the research reported in this special issue, I and my policy unit are uniquely positioned to assess the relevance of the research to climate adaptation policy practice and to speculate on how other government policy analysts in the field might respond. However, what might seem at first sight to be a straightforward question promising an equally straightforward answer is not that simple and is limited to one analyst’s experience and perspective alone. The particular application of “SenseMaker” and of its findings as set out in the essays in this volume will no doubt provoke legitimate debate over its promise and weakness as a policy aid. I have anticipated and explored this matter, but any judgments made are not themselves conclusive on the ultimate utility of the tool.
The title of this piece also implicitly asks how receptive the government policy community might be to this kind of approach. How might other government climate analysts and advisers in Canada and abroad use SenseMaker to develop smart adaptation plans, policies, and programs? Would they value its worth? The answer requires some reckoning with the prevailing paradigms in the policy profession and how open and flexible they might be. At the moment, policy makers, a catchall term I use to describe all government analysts and advisers involved in climate adaptation policy, are themselves divided about how best to support adaptation to environmental change. Most are heavily preoccupied with traditional risk management and engineering approaches to climate preparedness, relying heavily on empirical data and rational policy thinking for strategy development (Dessai et al. 2009, Eakin et al. 2009). A small but growing minority are looking instead to psychology and sociology research for insights on how to enable the kinds of behavioral changes that will better protect communities from climate impacts and build their capacity to adapt. Policy makers who take a more nuanced social science approach to adaptation may be less married to hard scientific and expert-led research and practice and more open to new disciplines, methods, and tools. Clearly, the fate of SenseMaker will depend in part on how these two policy paradigms or archetypes within the policy community play out.
The Climate Change Directorate, my unit, within the provincial government of Nova Scotia in Canada is itself at a crossroads. As the government body responsible for coordinating and supporting adaptive action across the civil service and across local governments and businesses more broadly, the directorate must deliver research that not only informs decision makers but also inspires them to act. A lot of time is devoted to supporting climate impact and risk assessment planning in various sectors and departments, but these assessments have not on their own galvanized wide social change or led to new adaptive behaviors; policy change requires more than just evidence-based research to make that shift. Climate impact assessment research can also sometimes box decision makers into simplistic and maladaptive strategies if the scope of the study is overly narrow or ignores important cultural, psychological, and socioeconomic factors (Eakin et al. 2009, Kennedy et al. 2010). The directorate recognizes these limitations and the need to think outside the traditional policy and research box. There is a curiosity for what wider social science approaches in complexity thinking, in social psychology, might offer decision makers and policy makers that would help them “make sense” of issues that risk management studies cannot, such as how to engage their stakeholders in discussions about acceptable levels of risk or how to create institutions that work collaboratively across department silos on climate issues that affect them all.
This is why in 2011 the directorate participated in the research reported in this special issue and joined the Australian research team. We were interested in exploring the usefulness of complexity and social psychology tools in informing adaptation policy. We were interested in what they might reveal about the many different people with whom we worked and how different groups in Canada make sense of and act in the face of climate change. We were also interested in evaluating the effectiveness of this new online SenseMaker “survey” tool for its ability to collect and contrast opposing views and experiences of climate adaptation and to engage policy analysts and decision makers alike in the kind of collective inquiry that complexity approaches advocate.
I reflect on our experience and thinking on the value of this research in informing climate adaptation policy and practice. First, the policy challenge is outlined as it appears to policy makers, exploring why climate adaptation is such a “wicked” problem and what might be gained by applying complexity tools, such as SenseMaker. I then reflect on our role in the research, as well as its limitations and findings. I conclude with an evaluation of the usefulness of this work in informing policy and with recommendations for improvements for greater policy impact.
The policy challengeAdaptation to climate change is highly complex, involving multiple players across temporal and spatial scales, facing different stressors, at different times, with unique vulnerabilities and capacities to adapt (Berkes et al. 2003, Adger et al. 2009, Smithers and Smit 2009). What enables some groups to thrive and others to decline is often not the result of a single factor but of multiple interacting factors (Folke et al. 2002, Diamond 2005). It is difficult to capture these dynamics a priori, let alone affect them through policy (Adger et al. 2007).
There are also many tools in the policy toolbox, such as regulations, market-based incentives, and behavior-change campaigns that can be used to steer behavior (Halpern et al. 2004). Identical policy instruments may have positive and negative applications and unintended as well as intended outcomes in different policy contexts (Fiorino 2006). Policy makers cannot, therefore, rely on off-the-shelf solutions or naively assume that by adopting so-called best practices they will evoke the positive responses experienced elsewhere. Achieving desired policy outcomes is particularly complex because of this context sensitivity. The factors that lead to success cannot always be predicted.
Adaptation policy is daunting too because of the diversity of views about adaptation held by different groups. These preferences often influence what policies get supported and later adopted. However, frequently what people think and say may not always align with how they act or behave, particularly if factors change or policies shift (Kahneman 2011). Policy makers cannot assume, for instance, that public support for proposed initiatives will continue once they are implemented or that stated preferences would dictate actual behavior on the ground. This exacerbates political uncertainty about how people will respond to different policy regimes and renders decision making difficult.
In this sense, climate adaptation fits many of the characteristic traits of what Rittel (1972) called “wicked problems.” The causes and the solutions to the problem are difficult if not impossible to define. There is no one actor responsible for the problem or the solution. Actors hold different views about what should be done, and unpredictable social behaviors determine to a great extent policy success (Rittel 1972, Conklin 2006, Australian Public Service Commission 2007, Head and Alford 2013). These and other characteristics of wicked problems help analysts to distinguish them from more benign challenges where the problem definition and solution is clear and where there are already best practices in place to guide decision making.
Standard rational policy planning is not designed to solve wicked problems (Rittel 1972). It is designed to solve “simple” or “complicated” problems where the causes and effects are known, linear, and empirically grounded (Kurtz and Snowden 2003). Managing wicked problems requires a different strategy, and this poses a great challenge to policy makers who tend to default into traditional policy thinking, even when it comes to complex issues (Conklin 2006, Australian Public Service Commission 2007, Head and Alford 2013). As Einstein once said, “We can’t solve problems by using the same kind of thinking we used when we created them” (Mielach 2012). However, changing how policy makers approach wicked problems is tough because it requires reconciling two very opposing modes of thinking (http://www.homerdixon.com/2010/05/05/complexity-science-and-public-policy).
The archetypal “traditional policy maker” is trained to distill from complex problems logical rational solutions, using evidence to support political decision making (Rittel 1972, Howlett and Ramesh 2009). These analysts rely on experts and independent advisers for objectivity and tend to place great faith in the scientific method (Rittel 1972, Kurtz and Snowden 2003, Howlett and Ramesh 2009, Head and Alford 2013). They are geared toward providing tangible, efficient, and one-time policy solutions.
The “complexity analyst” archetype, by contrast, captures and tackles problems in their inherent complexity. These analysts rely on the multiple perspectives of nonexperts for insights on how to navigate, not necessarily solve, wicked problems (Rittel 1972, Conklin 2006). People on the ground are thought to be more knowledgeable because they interact in the problem environment every day, as opposed to experts who may be removed from the environment and only able to understand it through abstraction. Through creative conversation and engagement with peers, complexity policy makers believe that people come to identify and negotiate ways of interacting that eventually lead to solutions that are more effective than ones that are imposed.
We use this dichotomy between traditional and complexity policy makers to help illustrate the tension between these two opposing approaches to policy making and how it influences policy-makers’ impressions of SenseMaker and the value of this kind of work. In reality, policy makers are neither one of these extremes and may exhibit any range of perspectives and behaviors along the continuum. Like people, they are not constrained to a single, consistent position. Their views can change as the policy context, goals, and problems shift. That said, policy makers, like anyone else, are still vulnerable to habitual modes of thinking, bias, and preference that can color the way they approach policy issues.
Both traditional and complexity approaches have merit provided they suit the problem at hand. If the policy problem is tangible and simple, then a traditional approach is the most efficient and effective (Rittel 1972). It is when traditional approaches get wrongly applied to complex problems that troubles arise (Snowden and Boone 2007). The criticisms of risk assessments in climate adaptation planning are a case in point. Risk assessments involve identifying impacts, ranking risks, and finding cost-effective measures to reduce those risks (Willows and Connell 2003, Carter et al. 2007, Burton et al. 2009, Smith and Petley 2009, Kennedy et al. 2010). Risk analysts believe that there are “known knowns,” or at least “known unknowns,” in the way that climate change will impact society and that these factors can be drawn out with the help of expert modeling or analysis (Snowden and Boone 2007, Smith and Petley 2009). Adaptation is treated as a “simple” or “complicated” domain issue, not a complex or wicked one (Snowden and Boone 2007). In reality, climate adaptation is often more complex. As argued previously, it requires addressing cause-and-effect relationships that are sometimes only retrospectively knowable, which makes identifying the problem and the solution difficult if not impossible. Applying linear rational policy approaches may not work in this case because there is no clear policy problem or solution to work toward. If we force a problem diagnosis where it is not clear, simply to adhere to the steps of our approach, we risk oversimplifying and misdiagnosing the problem (Rittel 1972, Eakin et al. 2009, Kennedy et al. 2010).
Risk management approaches also tend to ignore the psychology of different players, the connection between their cognitive frames, and their subsequent actions. There is little emphasis on complexity, psychology, and culture, despite the important role they play in enabling or hindering adaptation (Adger 2003, Adger et al. 2004; K. Brown, unpublished manuscript, http://www.youtube.com/watch?v=FdnJ52VkQLQ). If we reduce climate adaptation to the mere study of climate hazards, we may miss the important cultural and social change elements that adaptation requires (Eakin et al. 2009, Kennedy et al. 2010).
As complexity scholars argue, “Wise executives tailor their approach to fit the complexity of the circumstances they face” (Snowden and Boone 2007:69). The same logic can and should be applied to policy making. Clearly, the task goes beyond how governments deal with the unpredictable nature of climate hazards and the complex environment in which adaptation policy making takes place. Policy makers must also reconcile traditional and complexity approaches to policy making and be able to discriminate between different types of problems and the kind of analytical tools and thinking they require (Rittel 1972, Kurtz and Snowden 2003).
Why sense-making toolsSome complexity-oriented policy makers and scholars find standard analytical tools or at least the epistemology behind those tools, i.e., multiple-choice surveys and reductionist models, ineffective, at least on their own, at adequately representing social complexity, behavior, and preference (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). As one Nova Scotian government analyst stated:
Traditional tools tend to categorize and break things down into simple constructs and apply logic to what are sometimes illogical problems. Policymakers need tools that complement but also challenge existing models that have been relied on for years. The tools must honor the nature of the problems that governments are asked to solve. (A. Poirier, Halifax, Nova Scotia, 15 September 2011, personal communication)SenseMaker is a type of survey tool that aims to capture and analyze complexity in a way that reductionist surveys supposedly cannot (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). It can demonstrate how different groups perceive and relate to one another and to their environment through the capture of “narrative fragments,” allegedly a richer way of eliciting experiences, perceptions, and knowledge than would otherwise be possible from generic question-and-answer surveys (Lynam and Fletcher 2015, M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). The narrator chooses a story relating to a particular topic and signifies to the analyst the importance or meaning behind it. This allegedly reduces misinterpretations, allows respondents to communicate more freely, and widens the scope of what can be observed to reveal the complexity (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication).
In this way, SenseMaker is more than just a data collection tool: it is a new complexity-oriented approach to policy. It may support policy makers to become familiar with, and to be informed by, the patterns of their own environment as expressed by the people in it. It may force policy makers to suspend judgment on what is the problem or the solution until after the experiences, perceptions, motivations, and behaviors of people are better understood (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication). It asks policy makers to move beyond seeing people as simply supporters or dissenters of preconceived policy goals and instead as legitimate experts in their own right, with information to tell about the system and experiences within it. Policy makers must withhold judgment to see all perspectives as equally valid and as opportunities for learning, recognizing that no one perspective holds the “truth” (Snowden and Boone 2007). This approach promises to reveal social patterns of interaction and beliefs that policy makers would not otherwise see, including “outlier” patterns that may impede policy goals. It may also provide clues about what policies work for different people in different contexts, serving as a critical supplement to standard risk assessments in informing, in this case, adaptation policy.
Another sense-making tool used in the research in this special issue was word association analyses. Although not normally part of a SenseMaker survey design, its inclusion in this case allowed the research team to solicit additional useful information. The team was interested in what words respondents were most likely to associate with climate change with the aim of helping those in policy practice in Canada and Australia to get to know and understand people’s mental framing of climate change (Moloney et al. 2014). Such methods may reveal contradictions between people’s representations of climate change and their experiences of climate change. This could give policy makers the ability to identify and modify those patterns of thinking and behavior by actively working to influence the mental representations of those they wish to effect. For example, in better understanding their audiences, policy makers could, the research team argued, shape consultations and messaging more strategically for different audiences.
A challenge in this, and similar, research projects is whether these tools can live up to the traditional policy-makers’ standards of scientific rigor without compromising the complexity and broader patterns these tools try to illuminate. Traditional policy makers, even those sympathetic to complexity, still prefer or require “certainty” or, at the very least, scientifically defensible and logical grounds for a policy direction particularly where major resource investments are at stake. Policy makers are therefore more likely to rely on conventional tools because, unlike complexity methods, these tools emphasize reducing uncertainty and finding statistically supported relationships that will meet evidence-based decision-making demands. Word association analysis may be better able to meet these demands than would SenseMaker, as the findings are based on statistical significance. Although SenseMaker software is capable of doing some traditional statistical analysis, it is more so designed to detect and make sense of emerging “patterns” in the data set that may or may not be statistically relevant. The research in this special issue helped us to anticipate whether tools like SenseMaker could meet those demands and sway both traditional and complexity types to see the value in this kind of approach.
THE STUDYAs described by Lynam and Fletcher (2015), social psychology and climate adaptation scholars at the Commonwealth Scientific and Industrial Research Organization (CSIRO) in Australia initiated the research in this special issue. They partnered with the Australian-based company, Emerging Options, which had staff trained in complexity thinking and had the license to use the SenseMaker software. Together they designed and launched two separate online surveys in 2010 and 2011 targeting climate change and adaptation scholars at a conference in Australia in 2010 and an Australian state government department involved in environmental sustainability. Our Climate Change Directorate learned about the study in early 2011 and joined the research team just before the third and final survey launch. This allowed us to solicit Canadian responses to the third survey and to broaden the initial geographic scope of the pilot for greater cross-cultural study and comparison Lynam and Fletcher (2015), for a full methodological description).
Because we were late to join the study and merely piggybacking on the original process, we did not have the opportunity to devise the initial probing question in the SenseMaker survey. Not having a rigorous set of policy questions or problems in mind at the outset likely had an impact on how relevant the data will be to current policy issues. The survey question of what enables and constrains climate change, for example, was broad enough to be relevant but also rather vague. Early immersion in the design and planning would also have assisted the directorate in achieving a better assessment of the survey results and a more rigorous process to draw out practical policy implications and consequences. In the course of applying the survey, several other methodological and epistemological problems arose. The following is a synthesis of the most prominent challenges and surprises. I and the Climate Change Directorate critiqued the research and the SenseMaker tool in light of those challenges.
The challenge with data collectionData collection proved more difficult with SenseMaker than for traditional analytical surveys, thus hindering researchers’ ability to get sufficient data. The Australian survey collector for the general public panel, for instance, noted higher dropout rates with SenseMaker than with other surveys that had been launched. This was in part attributed to the unusualness of the survey instrument. It could also be that SenseMaker requires more engagement on the part of respondents than a typical “question-answer” survey. Reflecting on Canadian responses to the survey, some respondents felt that the opening question and following prompts were confusing, and in frustration with the tool’s design, they dropped out. Whatever the reasons, these weaknesses posed a barrier, particularly for more traditional policy makers who understandably wanted sufficient data to give confidence to any conclusions drawn.
Assumptions about narrativesA repeated assertion made by SenseMaker proponents is, “Humans convey complex knowledge through narratives [and] that people are able to write or tell narratives easily” (K. S. Ghee, unpublished manuscript). The term “narrative” is often thought of in a strict or narrow sense requiring “at least three elements: an original state of affairs, an action or an event, and the consequent state of affairs” (Czarniawska-Joerges 1998:2). It is precisely SenseMaker’s ability to elicit and analyze such narrative fragments that differentiates it from traditional opinion surveys (M. Cheveldave, Halifax, Nova Scotia, 15 November 2011, personal communication).
For this reason, it was assumed that survey responses would be in narrative form. However, as was the case in a separate SenseMaker study in Singapore, “it was observed that most respondents more naturally shared opinions rather than narratives” (K. S. Ghee, unpublished manuscript). “Experiences provided mainly talk about climate change—what it is, [or] is not and, to a lesser extent, [a] focus on how to adapt” (Fletcher, unpublished manuscript). Respondents did not follow a plot or narrative, involving characters, place, or time markers describing their own adaptation experiences.
This could be because of the style of the opening survey question, which solicited a perspective rather than a narrative about climate change. It was designed this way because the research team worried that respondents might not have had direct experiences managing the impacts of climate change:
Imagine you are in a lift (elevator) with 2 people who are discussing how people and institutions are reacting to climate change. One person mentions that several obstacles constrain the extent to which people are able to prepare for impacts and/or adapt. The other person says that he knows of a few examples in which people and institutions are already responding. They turn to you and ask for your perspective on what makes preparation/reaction possible or difficult. How would you respond?Respondents’ confusion with the survey may have in part been caused by the fact that the follow-up prompting questions did not seem to match the opening one. For instance, one of the generic prompts asks: “Is your response about you or someone you know?” Or, “Who are the main characters in your story?” This would be an irrelevant or confusing question for the respondents to answer if their initial response was an opinion about how to adapt rather than an experience of adapting. It could also be a problem with online surveys generally where respondents may not take the time to read the instructions that emphasize the importance of relaying experiences and stories around climate adaptation. It could also be that narratives are just more difficult to conjure up than SenseMaker proponents had anticipated. Those who think strict narrative form is critical to the success of SenseMaker may find the lack of narrative form a problem, even a contradiction, in the tool’s application. It may undermine the “richness of the data collected since opinions unlike narratives do not encapsulate real experiences which provide important contextual data” (K. S. Ghee, unpublished manuscript). Others, however, are less concerned about the structural form of the responses and more concerned with the message. Although the directorate was initially surprised at the content of the responses, given the repeated emphasis on narrative, we concluded that all the data could prove insightful regardless of the form.
SenseMaker seems equipped to study these patterns of opinion and has been useful for such analysis, although it is debatable whether it is superior to or even different from traditional surveys in this regard (K. S. Ghee, unpublished manuscript). This is why applying different analytical tools beyond those of SenseMaker, such as narrative or text analysis using QDAMinor, for instance, can be useful, because researchers and policy makers alike can begin to compare and assess how well different software tools interpret and draw insights from the data.
Involving policy makers in the analysisOne of the critical questions to be faced in this kind of undertaking is how best to involve policy makers in the SenseMaker process and investigation of the data. SenseMaker proponents assert that policy analysts in the field will detect patterns that other general analysts will not see and should therefore be involved in the analysis process (Fletcher, unpublished manuscript). Such involvement moves many policy makers out of the realm of practice and into the world of academia. Most nonresearch-based civil servants including many policy analysts tend to outsource this analysis, partly because of time constraints but also to get independent expert advice. They look at data and make decisions but do not often crunch the data or analyze the patterns themselves. A major culture shift is required especially for traditional, nonresearch-based policy makers to work with SenseMaker consultants and academics to draw out conclusions from the data together.
Our main finding from being involved in the data analysis phase was that there was little difference in the ability to detect patterns in the data between policy makers versus individuals with limited knowledge of adaptation. However, we found that policy makers were better able to hypothesize why those patterns might be forming because of prior knowledge of the policy environment.
Making use of the findingsThe word association analysis was moderately helpful in giving policy makers insight into how different respondents contextualized climate change. The most frequently cited words in Canada, i.e., “weather,” “sea-level rise,” “melting,” “flooding,” “greenhouse gases,” and “oil,” seem to reflect the way that media and governments currently contextualize climate change in Canada and suggest that popular media is a key contributor to people’s mental framing of the issue (Lorenzoni et al. 2006, Lorenzoni and Hulme 2009; T. Lynam, 2011, personal communication). People also seem to resonate more with impacts they see in their own back yard, hence the reason why most respondents, being northern coastal inhabitants, chose “sea-level rise,” “melting,” and “flooding” as impacts. This prompted the Climate Change Directorate to think about the potential of using familiar images that resonate with people to steer communication strategies, but the data were still perceived to be too vague to inform a detailed strategy or propel a reluctant traditional policy maker to invest in such a strategy.
Traditional analysts are likely to have trouble finding clear-cut explanations for the mental representations that people hold. Moreover, traditional policy makers will likely want to know whether the prevailing mental anchoring will be good news or bad news for specific climate policies. What does the evidence suggest about people’s behavior in the face of climate change, and how are governments going to work with these groups? The depth of the data analysis is less useful, permitting researchers and policy makers only to speculate.
A traditional policy maker, for instance, might take a positive interpretation of the way Canadians associate climate change. The policy maker may think that Canadians are correctly associating climate change with “weather,” climate is after all a long-term weather trend; with causes, i.e., “greenhouse gases” and “oil”; and with the consequences of the problem, i.e., “sea-level rise,” “melting,” and “flooding.” The policy maker may think that because Canadians have this factual anchoring, they may be more rational and reasonable in their response to proposed adaptation policies.
A complexity policy maker, on the other hand, may find this mental anchoring too narrow and too simple in its sole focus on images that resonate, marginalizing and forgetting other important climate impacts that are not so prominent. The question then becomes whether communication strategies should build principally on the known target associations or purposely target other neglected issues to broaden people’s knowledge and support for other adaptation efforts. It is not immediately clear which policy response is best.
Such is the real world of sense making, which presents complexity but no clear direction on where policy makers should necessarily go. Traditional policy makers will likely retreat from this uncertainty, whereas complexity theorists may be more ready to try small and safe experiments to test policies for wicked problems in complicated times. However, even getting small experimental projects up and going is difficult when resources and capacity are strapped and the evidence to warrant experiments is slim.
Patterns in the narratives can also show how receptive or fearful respondents are to certain policy tools. Canadian respondents, for instance, saw regulations as more strongly helping adaptation than did Australians, who saw technology as more strongly associated with enabling adaptation than were regulations (T. Lynam, personal communication). Respondents from both countries saw money as the greatest hindrance to adaptation.
These can be useful observations for policy makers to understand their audience and the extent to which policy can address public feelings and concerns. However, the data likely do not provide sufficient evidence for a traditional policy maker to warrant further exploration unless there is a specific reason to do so. The data would not be a critical consideration in the day-to-day policy decisions because there are so many other subsidiary technical questions to be answered for the data to be useful, such as what kind of technology and why it is preferred, what kind of regulations, and so forth. The data allow you to probe a little further, but the interrogation appears endless.
DISCUSSIONThe directorate’s expectations going into the study were that the application of the SenseMaker tool would enable us to assess public perceptions as well as the applicability of the SenseMaker tool. The reality was that the project was able to give us a flavor for the potential of SenseMaker and to reflect on our experiences in using it, but it did not enable us to draw definite conclusions about the quality of the tool and its impact on policy.
The findings from the study are a little underwhelming, in terms of their ability to have much policy impact. They do provide good initial direction to local climate adaptation policy but lack specifics. However, they have prompted the directorate to think about how multiple groups interpret and respond to climate change and to different adaptation tools. Even the more traditional policy makers in the group have started creative conversations with openness to strategies around these concepts, which was of course one of the intentions of the study. We just need now to provide a deeper analytical rigor to win more confidence from these and other traditional policy makers.
This said, whatever the limits of the tool or the study approach, the research that the directorate undertook has broken ground on several levels. It was one of only a few Canadian studies looking at public perceptions and experiences of climate adaptation, as opposed to climate mitigation, and the only one to use SenseMaker software for the analysis. It has forged new relationships between policy makers and researchers, which created new opportunities to learn how different disciplines would approach and go about analyzing and handling different climate change problems. There are doubtless further policy clues to be drawn from the SenseMaker data with further analysis.
RecommendationsResearchers need to acknowledge that a majority of policy makers in climate adaptation are not yet thinking about more nuanced social science research. If this is to change, aspects of these sense-making approaches must be more defensible from a rigorous “evidence-based” point of view and more directive in what policy makers must do. Problems with research design, data collection, and dropout rates must be tackled if research of this kind is to produce the volume and quality of data to ensure reliability and validity.
Researchers also need to be clear about what they mean by “narrative” and be careful not to expect that all responses to a SenseMaker survey will elicit strict narrative content. They must be prepared to use other analytical tools beyond SenseMaker to assess this content, especially given the diversity in the way responses are structured, because some tools may be better than others at assessing different aspects of the response. Finally, the initial probing question must be carefully designed to prompt exactly the kind of responses policy makers are interested in, so that there is some control over the kind of responses policy makers get.
Our experience suggested that policy makers might not need to be involved in all aspects of pattern detection or analysis. Researchers can still probe the data based on policy-maker reactions to it, so policy engagement is not compromised but is also not diverted to endless investigations. Nonresearch-based policy makers should be consulted on instrument questions and design at the outset and then left to be involved afterward in interactive analysis of data patterns with investigators. The goal should be to identify what factors are most engaging from the policy practitioners’ perspectives and how best to draw out the implications and consequences for both the policy process and public policy.
ConclusionsMany policy makers are looking for certainty, or at least to reduce uncertainty, despite the fact that social psychology and complexity approaches suggest that we cannot, at least currently, provide that certainty. Under such circumstances, we might follow Conklin’s (2006:10) advice: “You don’t so much ‘solve’ a wicked problem as you help stakeholders negotiate shared understanding and shared meaning about the problem and its possible solutions. The objective of the work is coherent action not final solution.”
This is wise advice for these kinds of problems, but the dilemma is that politicians want, and traditional policy analysts are trained to provide, a clear problem definition and decisive solution that they can work toward. SenseMaker and word association analysis gives analysts a rich data set in which to probe and sense for answers, but they do not always give policy makers definitive answers or certainty on the direction to be followed. Instead, the evidence raises yet more hypotheses and investigations. This can be an excellent starting point for creative policy discussions, but it is frustrating for policy makers or politicians, who want action, are short on time, and cannot get definitive expert guidance.
As an analytical tool, we consider SenseMaker strong in its ability to gather large suits of data at the collection stage, yet weak in the wide room left for bias and misinterpretation when trying to make sense of the findings. There is also a traditional policy tendency to want to make judgments about the data before fully absorbing the patterns themselves and their meanings. In this way, it is easy to see how a top-down traditional policy focus might end up reframing the narrative and hijacking its policy consequences.
In our view, the greatest value of SenseMaker is the epistemology and complexity theory behind it. It reminds policy makers that there is a role for government in helping to shape public discourse on climate change, to augment or dampen patterns of thinking around adaptation, and to play more of a hand in directing cultural adaptation for climate change. It is these gaps in current practice and policy that sense making can begin to fill. Overall, SenseMaker permits a more nuanced understanding of public knowledge and opinion, particularly that of specific target groups, and it prepares policy makers with a lens through which to engage in intensive analysis and consultations and frame climate policy.
RESPONSES TO THIS ARTICLEResponses to this article are invited. If accepted for publication, your response will be hyperlinked to the article. To submit a response, follow this link. To read responses already accepted, follow this link.
A very special thanks goes to Dr. Timothy Lynam, coeditor of this special issue for his unwavering support and guidance on this article and throughout the project. Without his leadership, none of this work would have been made possible. I am also grateful to the editor and reviewers at the CSIRO for their valuable insights and constructive feedback on earlier versions of this document and to Ray MacNeil and colleagues at the Climate Change Directorate for their ongoing support of this work and their critical perspective on complexity issues. This article reflects the views of the author and does not necessarily represent the views of the Province of Nova Scotia.
LITERATURE CITEDAdger, N. W. 2003. Social capital, collective action, and adaptation to climate change. Economic Geography 79:387-404. http://dx.doi.org/10.1111/j.1944-8287.2003.tb00220.x
Adger, N. W., S. Agrawala, M. M. Q. Mirza, C. Conde, K. O’Brien, J. Pulhin, R. Pulwarty, B. Smit, and K. Takahashi. 2007. Assessment of adaptation practices, options, constraints and capacity. Pages 717-743 in M. L. Parry, O. F. Canziani, J. P. Palutikof, P. J. van der Linden, and C. E. Hanson, editors. Climate change 2007: impacts, adaptation and vulnerability: contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK.
Adger, N. W., N. Brooks, G. Bentham, M. Agnew, and S. Eriksen. 2004. New indicators of vulnerability and adaptive capacity. Technical Report 7, Tyndall Centre for Climate Change Research, East Anglia, UK.
Adger, N. W., I. Lorenzoni, and K. L. O’Brien. 2009. Adaptation now. Pages 1-22 in N. W. Adger, I. Lorenzoni, and K. L. O’Brien, editors. Adapting to climate change: thresholds, values, governance. Cambridge University Press, Cambridge, UK.
Australian Public Service Commission. 2007. Tackling wicked problems: a public policy perspective. Australian Government, Canberra, Australian Capital Territory, Australia.
Berkes, F., J. Colding, and C. Folke. 2003. Navigating social–ecological systems: building resilience for complexity and change. Cambridge University Press, Cambridge, UK.
Burton, I., S. Huq, B. Lim, O. Pilifosova, and E. L. F. Schipper. 2009. From impacts assessment to adaptation priorities: the shaping of adaptation policy. Pages 377-393 in E. L. F. Schipper and I. Burton, editors. The Earthscan reader on adaptation to climate change. Routledge, London, UK.
Carter, T. R., R. N. Jones, X. Lu, S. Bhadwal, C. Conde, L. O. Mearns, B. C. O’Neill, M. D. A. Rounsevell, and M. B. Zurek. 2007. New assessment methods and the characterisation of future conditions. Pages 133-171 in M. L. Parry, O. F. Canziani, J. P. Palutikof, P. J. van der Linden, and C. E. Hanson, editors. Climate change 2007: impacts, adaptation and vulnerability: contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK.
Conklin, J. 2006. Wicked problems and social complexity. Pages 3-40 in J. Conklin. Dialogue mapping: building shared understanding of wicked problems. John Wiley and Sons, Chichester, UK.
Czarniawska-Joerges, B. 1998. A narrative approach to organization studies. Sage, London, UK.
Dessai, S., M. Hulme, R. Lempert, and R. Pielke. 2009. Climate prediction: a limit to adaptation? Pages 64-78 in N. W. Adger, I. Lorenzoni, and K. L. O’Brien, editors. Adapting to climate change: thresholds, values, governance. Cambridge University Press, Cambridge, UK.
Diamond, J. 2005. Collapse: how societies choose to fail or succeed. Viking Press, New York, New York, USA.
Eakin, H., E. L. Tompkins, D. R. Nelson, and J. M. Anderies. 2009. Hidden costs and disparate uncertainties: tradeoffs in approaches to climate policy. Pages 212-226 in N. W. Adger, I. Lorenzoni, and K. L. O’Brien, editors. Adapting to climate change: thresholds, values, governance. Cambridge University Press, Cambridge, UK.
Fiorino, D. J. 2006. The new environmental regulation. MIT Press, Cambridge, Massachusetts, USA.
Folke C., J. Colding, and F. Berkes. 2002. Synthesis: building resilience for adaptive capacity in social-ecological systems. Pages 352-387 in F. Berkes, J. Colding, and C. Folke, editors. Navigating social-ecological systems: building resilience for complexity and change. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/CBO9780511541957.020
Halpern, D., C. Bates, G. Mulgan, and S. Aldridge. 2004. Personal responsibility and changing behaviour: the state of knowledge and its implications for public policy. Cabinet Office, Prime Minister’s Strategy Unit, London, UK.
Head, B. W., and J. Alford. 2013. Wicked problems: the implications for public management. Administration & Society, in press.http://dx.doi.org/10.1177/0095399713481601
Howlett, M., and M. Ramesh. 2009. Approaches to public policy. Pages 18-47 in M. Howlett and M. Ramesh. Studying public policy: policy cycles and policy subsystems. Oxford University Press, Oxford, UK.
Kahneman, D. 2011. Thinking fast and slow. Doubleday Canada, Toronto, Ontario, Canada.
Kennedy, D., L. Stocker, and G. Burke. 2010. Australian local government action on climate change adaptation: some critical reflections to assist decision-making. Local Environment: The International Journal of Justice and Sustainability 15:805-816. http://dx.doi.org/10.1080/13549839.2010.514602
Kurtz, C. F., and D. J. Snowden. 2003. The new dynamics of strategy: sense-making in a complex and complicated world. IBM Systems Journal 42(3):462-483.
Lorenzoni, I., and M. Hulme. 2009. Believing is seeing: laypeople’s views of future socio-economic and climate change in England and in Italy. Public Understanding of Science 18:383-400. http://dx.doi.org/10.1177/0963662508089540
Lorenzoni, I., A. Leiserowitz, M. D. F. Doria, W. Poortinga, and N. F. Pidgeon. 2006. Cross-national comparisons of image associations with “global warming” and “climate change” among laypeople in the United States of America and Great Britain. Journal of Risk Research 9:265-281. http://dx.doi.org/10.1080/13669870600613658
Lynam, T., and C. Fletcher. 2015. Sensemaking: a complexity perspective. Ecology and Society 20(1): 65. http://dx.doi.org/10.5751/ES-07410-200165
Mielach, D. 2012. We can’t solve problems by using the same kind of thinking we used when we created them. Business Insider, 19 April. [online] URL: http://articles.businessinsider.com/2012-04-19/strategy/31366385_1_business-lessons-success-business
Moloney, G., Z. Leviston, T. Lynam, J. Price, S. Stone-Jovicich, and D. Blair. 2014. Using social representations theory to make sense of climate change: what scientists and nonscientists in Australia think. Ecology and Society 19(3): 19. http://dx.doi.org/10.5751/ES-06592-190319
Rittel, H. 1972. On the planning crisis: systems analysis of the ‘first and second generations.’ Bedriftskonomen 8:390-396.
Smith, K., and D. N. Petley. 2009. Environmental hazards: assessing risk and reducing disaster. Fifth edition. Routledge, New York, New York, USA.
Smithers, J., and B. Smit. 2009. Human adaptation to climate variability and change. Pages 15-33 in E. L. F. Schipper and I. Burton, editors. The Earthscan reader on adaptation to climate change. Routledge, London, UK.
Snowden, D. J., and M. E. Boone. 2007. A leader’s framework for decision making: wise executives tailor their approach to fit the complexity of the circumstances they face. Harvard Business Review 85:68-76.
Willows, R. I., and R. K. Connell, editors. 2003. Climate adaptation: risk, uncertainty and decision making. United Kingdom Climate Impacts Programme (UKCIP) Technical Report, UKCIP, Oxford, UK.
Kyla M. G. Milne
1903 Barrington Street
PO Box 442
Based on the three-step analysis described above, we now describe how the activities of telling and retelling of stories function as a practice of sensemaking in which police students articulate their first occupational experiences. Findings are presented in the following three sections, which outline storytelling practices, selection of stories and how knowledge retention was achieved through storytelling. Themes within the empirical material are presented in overview as well as in detail through exemplifying field note extracts.
The students frequently expressed that the follow-up sessions were important and that they appreciated the educational events. The storytelling within these sessions was often of a collective nature, prompted by questions from the facilitator and by peers. The sessions were open-ended, characterized by improvisation and were more disorganized than initially expected. Stories about occupational realities were often not in finished form when they were first voiced. Rather, the activity of articulating one’s understandings and experiences of field work was of an interactive nature, without clear beginnings, ends, or plot twists. Narratives were often structured by an external criterion such as time (describing day-to-day experiences), theme (e.g., describing a number of dispatch calls of a particular sort) or by association with a previous story or statement. The primary strategy for storytelling during the sessions was characterized by ongoing composition by the storytellers, which was regularly supported and facilitated by peers and by the instructor who often triggered stories through probing questions. These questions were important in the light of a common and somewhat paradoxical occurrence, which was that students often began their stories by assuring peers and the instructor that they did not have anything to tell, a statement that often was followed by quite severe, serious or sometimes incredible accounts of occupational police practice. One example from the field notes that highlights the importance of triggering questions for the facilitation of storytelling was when a female student (D) was summarizing her field experiences when prompted to do so by the instructor (I.N):
D seems to have some trouble knowing where to begin. She says that she spent some time on patrol, that she followed investigators around for a few days and did a little bit of everything such as reporting and interrogations. I.N asks some questions to assist D: “Did you mainly work dayshifts, nights…?” This question seems to resonate with D, and she starts talking about how she had done night shifts during the weekend. I.N asks if something “stood out” during these shifts. In response, D starts recounting an upsetting episode that she says she thought a lot about afterwards. It was about a disturbance in a nightclub where a guest had been severely beaten up by bouncers who covered for each other in their stories. D says that there were two witnesses at the scene who had seen the doormen beating and kicking the man and throwing him from a loading bay. The patrol took reports and called for an ambulance. D says that she was glad that they had witnesses, as this should make it a clear-cut case. However, upon discussing it with her more experienced colleagues, she learned that it would probably not lead to convictions due to the bouncers’ backing up each other’s stories.
Upon recounting this, D elicits recognition and frustration in the room. Several others fill in and describe similar experiences of injustice, such as bouncers who had been pointed out by patrol officers as infamous for being involved in beatings. The discussion starts to revolve around the paradoxical fact that in the story, the bouncers were the ones calling upon police as victims of violence, and in that capacity, they were plaintiffs. However, D’s story exemplifies how they also were the ones causing distress.
This episode was recounted by a student who initially had a hard time sorting out her impressions and selecting what to address in the session, especially she had a hard time sorting out what happened on which day. However, after some coaxing from others in the room (especially the instructor I.N), she recounted a serious and stressful account that brought recognition and sparked dialogue amongst her peers. The episode represents several defining aspects of how retrospective storytelling arranged the meaning of occupational experiences. First, the story exemplifies a constructed narrative that moves from a generic description of work placement and night shift work to a vivid example of injustice and power abuse. Important to note is that this example of narrative sensemaking was a collective achievement: The story was triggered by probing questions and dialogue, and the recognition of the story’s additional meaning was a shared experience of feelings of being upset. In the example, the instructor played a role as a dialogic counterpart to the student and as the one who initially gave the word to her and, as such, sparked the conversation, however, the instructor also held back after D started talking, giving her space to elaborate on her story.
Selection of Stories
Based on the second step of our analytical approach, we analysed our empirical material in search of themes describing which subjects, social relations and domains of policing recurred in police students’ stories. In this regard, three aspects stood out. When selecting subjects and social relations from experience, students tended to discuss colleagues (good guys) and criminals (bad guys). Furthermore, when contextualizing stories, a common tendency was to focus upon extreme situations, and when discussing policing, stories that reinforced notions of policing as actionable occurred frequently. These aspects are detailed below.
Regarding subjects and social relations, normative narratives about colleagues within the police community were the most common theme. These stories tended to depict heroic and desirable behaviours of colleagues or, conversely, less-desirable behaviours. The stories were, for example, about police who were good at “reading” situations, shifting tempo, addressing citizens in respectful ways, having good attitudes in general or being good at specific policing tasks (such as incident management or K9 tracking). Conversely, stories about unwanted behaviour from colleagues regularly fell into two categories: stories about worn-out, lazy, tired police or about overly macho and unnecessarily harsh police officers who had made bad impressions on students. Besides colleagues, students also referred in their stories to other groups that police come in contact with. An obvious theme of storytelling regarded criminals. These stories were regularly about students’ newly gained insights into criminal life, such as codes of silence, brutality between criminals or gang-related practices. A number of stories also revolved around students’ feeling sorry for criminals. Examples of this theme were often about exploitation amongst criminals, such as stories of individuals who had been robbed and/or beaten/stabbed/shot or about tragic life destinies, such as young offenders who had fallen victim to drugs. Finally, a number of stories about criminals also featured incongruities and contradictions between expectations and realities. A commonly occurring example of such incongruities featured criminals—“bad guys”—who were nice and/or sociable:
Recruit A tells a story about a thief he had apprehended. The thief had been caught red-handed whilst stealing from a store, and the recruit detained him. However, in their interactions, the offender was really nice, accommodating and humorous. A says that it was a pleasant experience to be around this guy and almost felt that it was a shame to ‘bust’ such a sympathetic person. This was something that was at odds with how he had pictured this type of work. A says that the episode made him think about the hardships of being fully objective in all situations as a police officer. He thought about how it might be challenging to respond to all equally even though one may feel very differently towards those individuals one apprehends, either in positive or negative aspects.
These types of incongruities were a common theme not only in relation to criminals. A reverse example of inconsistent behaviour was when “good guys” such as colleagues behaved in a blameworthy way. Whilst police and criminals expectedly were at the focal point in stories, there were also stories about mentally ill people and stories about members of the general public. Mentally ill individuals often appeared in stories either as doing harm to themselves (for instance, threatening suicide) or in cases of order disturbance. Members of the general public were often discussed in terms of bystanders, victims of crime or as visitors to bars and nightclubs.
Regarding stories and extreme situations, work experiences that demonstrated something out of the usual were frequently subjects for narration. For instance, stories exemplified a number of incidents of death that students encountered in their service, either through natural causes, such as being called out when corpses were found, or suicide dispatch calls, such as being called out to incidents of so-called train-jumpers. Other examples of extreme storylines regarded blood, gore and misery, such as people self-harming (i.e., cutting oneself, swallowing razorblades) or being exposed to violence by others (aggravated assaults, stabbings, torture-like situations). The tendency of stories to gravitate towards the extreme is a finding that is in accord with earlier observations of police storytelling as being focused on war stories (see Van Maanen 1973). As Moskos (2009) concluded, selection from memory out of police practice may favour the extreme rather than the mundane, as extreme courses of events simply might be more memorable. In the observed sessions, these themes in storytelling tended to arouse interest and spark further discussions about similar experiences of other students. Extremeness was furthermore, by the instructors regularly acknowledged as potential but not commonplace feature of policing.
Students also told stories about policing as actionable, for instance, of ‘blue-light’ emergency driving, the excitement of foot chases, house searches, drug seizures, arrests and conducting interrogations. Work tasks such as these have been described as related to a popular image of policing as an action-oriented, fast-paced occupation (cf. Reiner 2010), and such popular images were reinforced through peer storytelling. Whilst more-mundane realities of policing also occurred as themes, these were less common. When these themes were raised, the storytellers often expressed frustration over boredom and tedious work shifts as well as the normative message that these types of work tasks are not what policing ideally should be about. Examples of this category were stories about citizen complaints, report writing, shoplifters or patrolling of quiet neighbourhoods by car whilst waiting (and wanting) for something to happen.
Knowledge Retention through Storytelling
In addition to analysing the practices of storytelling and the manifest levels of stories, an important facet of narrative sensemaking through storytelling is how stories are collectively understood and what messages stories convey, as these factors are indicative of how stories function to establish meaning. The analysis of what messages, standpoints and underlying meanings the stories conveyed in practice indicated three overarching themes: belonging, contributing and emotion management.
Regarding belonging, a number of stories established how students were part of the police community rather than being external to it. One facet of occupational positioning was expressed merely through the capacity to tell stories. These types of stories were interpreted as a means to transfer the message that students had access to the ‘frontlines’ as well as the ‘backstages’ of policing and through such access also had the capability to ‘tell it like it is’ to peers and instructors. Through stories, students repeatedly casted themselves as subjects who were part of an occupational community, rather than as students undergoing education and experiencing work practice from an external viewpoint. This type of positioning of oneself as a professional practitioner was, for instance, showcased in storytelling when students expressed themselves normatively around occupation-specific issues such as frustration over reorganizations, understaffing or coordination between the police and other agencies. Similarly, students expressed frustration over priorities that the police set as an organization, as some work tasks such as dealing with shoplifters or petty crimes were thought to claim an unreasonable amount of time and effort from the police. The ability to express these types of value-laden opinions about how the police should organize and what the police should do with their time hinges on credibility that stems from occupational belonging. Other aspects of belonging in stories were about immersion in policing, where students expressed how they became part of a community or a fellowship and they felt camaraderie. Such a story was told by a male student (M) who had spent his work placement period in one of Sweden’s most crime-ridden districts, where gang problems were a major issue for the police.
M says he had been working with district police who had embraced “hot spots hot times” policing, which means they concentrated their efforts on places and times where things were ‘going down’. M says that it was a really hard climate, and they experienced a lot of ‘hairy situations’ policing the district. One example of such a situation was when M had been off duty at a local grocery store in proximity to the district and neighbourhoods he had been policing. At the store, some gang-affiliated individuals identified him as a policeman and started shouting at him, upon which he quickly withdrew and left the place. M says that the situation was unpleasant because he was not on duty. However, when he told this to his colleagues at the district, they gave him exemplary support and arranged so that he got daily rides home from the district for the remaining time of the field training so he did not have to use public transport in the area and risk being recognized while off duty.
This story, on a manifest level, exemplifies police work practice and the realities of working in a tough neighbourhood and in a frightening situation. However, the key message of the story was rather the aspect of being recognized as a policeman by the ones policed (gang members) and, more significantly, by colleagues and the law enforcement community, who acted in support and defence of a novice officer.
Similar to stories that expressed how students were part of the police, a number of stories also indirectly exemplified how students were able to contribute to successful outcomes of the situations they faced in work practice. These types of stories could showcase how working methods proposed or suggested by students had successful outcomes. Other examples included when students could contribute to policing with “new” fresh knowledge, outlooks or perspectives that helped more experienced police practitioners in their work duties. These types of stories often exemplified the duplicitous nature of storytelling, as on a manifest level, they could be about serious crime, whilst on a latent level, they could be about the empowering feeling of being a novice police officer who was able to contribute to results in police practice.
A final tendency of stories on a latent level regarded emotion management, such as students dealing with fears, fright or, conversely, humorous episodes. A number of stories of action-filled or extreme policing, for instance, contained issues of being afraid and being worried as an important ingredient that amplified the seriousness of the situations. Likewise, in stories about how they had to intervene in situations involving bloodshed, students often expressed feelings of exposure as well as risk and fright of being infected by contagious blood (specifically hepatitis C or HIV viruses). They also told stories of injustice that often involved feelings of frustration and resignation. In contrast to the negative feelings, a number of stories also involved humorous elements, such as comic situations, in which students had to restrain from laughing in order not to embarrass colleagues or the involved individuals they were intervening against. Whilst these emotional reactions are opposed, a common denominator is the importance of managing one’s emotions when being faced with work tasks.
Thus, storytelling practices supported students in regards to the question of how one should ideally feel about encountered phenomena and how one should handle, overcome or contain certain feelings. As a way to manage, manoeuvre around and process fear, fright, frustration or joy and personal feelings such as those of belonging, narrative sensemaking through storytelling seemed to entail a practice of emotion management. Through stories that elucidated and actualized emotions, students could validate their reactions in the face of peers and the instructor. Not at least, the instructor had an important role to aid students in their framing of stories. In this way, students’ interpretation were often mirrored by how the instructor related to the content of stories or the implicit or explicit messages of stories. For instance, if the instructor showcased emotional distancing, this could be a cue that such distancing entails a facet of the professional ethos of the police.