What perceptions do scientists have about their potential role in connecting science with policy?

In recent decades, widespread demand for science to effectively inform real-world policy decisions (NRC, 2009; CSHR, 2002; UNDP, 2001) has highlighted concerns about the real-world applicability of scientific research (Lemos et al., 2012; McNie, 2007; Sarewitz and Pielke, 2007). While many scientists conceive that their research has inherent value for policy actions, this potential is often not realized (McNie et al., 2016; Kirchhoff et al., 2013). Furthermore, demands to move science into policy can force participating scientists into political debates (Pielke, 2007) and might do more harm than good if scientists are not adequately prepared (Porter and Dessai, 2016).

Currently, scientific training inadequately addresses the engagement of science with policy (Selin et al., 2017). Creating training that more effectively assists scientists with bringing their work into policy debates requires a better understanding of the current state of scientists’ perceptions about the roles they might play in these processes. In this paper, we provide insight into the existing conceptions that geoscientists at all stages of their careers hold about the migration of science into real-world policy decisions. Doing so allows us to not only gain a better understanding of geoscientists’ perceptions about their potential roles in this process, but to explore the implications of these perceptions for training the next generation of geoscientists to work more effectively and comfortably in a world increasingly looking to scientists—from students to retired professionals—for expert input on policy decisions.

The view that science operated as a self-regulating market dominated discussions of science and policy in the U.S. in the mid-twentieth century (Sarewitz, 1996; Sarewitz and Pielke, 2007). This perspective presumed that science best contributes to society if scientists attend to addressing debates within science itself rather than on pursuing specific applied concerns (Polanyi, 1962). Much of the research on science and policy since that time has focused on critiquing the isolation of scientists from policy. Gibbons et al. (1994) referred to the pursuit of scientific knowledge divorced from practical application as “Mode 1” science and argued that science in isolation would be progressively replaced with the newer, emergent “Mode 2” science in which problem-focused multidisciplinary teams tackle existing social challenges. Similar to Mode 2 science, proponents of “post-normal science” (Funtowicz and Ravetz, 1993) have advocated for the development of extended peer communities featuring scientists and citizens (Ravetz, 2006). Such collaborations can identify and address gaps inevitably forming between scientific models of the impact of policy interventions, and the actual impact of these interventions in the complex world of practice (Ravetz, 2006; Mayumi and Giampietro, 2006; Kalafatis et al., 2015b). Ideally these collaborations will lead to collective adaptive learning that results in richer, more accurate scientific understanding as well as evidence-informed social policies (Ravetz, 2006).

More recently, scholars have expanded on these insights about engagement between science and policy while de-emphasizing Mode 2’s notion of a fundamental shift in the pursuit of science as well as post-normal sciences’ emphasis on social change. Stokes (1997) examined the extent to which new breakthroughs throughout the history of science were often inspired by attention to practical challenges. Those interested in innovation and economic development have similarly described the history of a “Triple Helix” of mutual support between academic institutions, industries, and governments underlying technological innovation and development (Etzkowitz and Leydesdorff, 2000). Sarewitz and Pielke (2007) explored the relationship between the supply of information consisting of scientists’ research efforts and the demand for scientific information throughout society. Similar to the relationship between supply and demand in economics, scientists’ research agendas and peoples’ interest in applying science influence one another, although this influence will be out of sync to varying degrees over time (Sarewitz and Pielke, 2007). Research being conducted might not have immediate relevance for practical use, its relevance might go unnoticed, or people might desire research that scientists cannot currently pursue. People also might misinterpret or misuse potentially useful science or be blocked from using scientific findings by other concerns (e.g., politics, financial concerns) (Sarewitz and Pielke, 2007).

Transdisciplinary research features sustained efforts to forge alignment between scientists and stakeholders through collaboratively developing ways of better understanding and addressing a complex social problem (Thompson Klein, 2004; Jahn et al., 2012). However, most transdisciplinary research scholars currently view these efforts as a particular kind of problem-driven research approach that complements ongoing traditional, discipline-based scientific work (Klein, 2004; Jahn et al., 2012). Others have focused on “boundary organizations” acting as intermediaries for engagements between science and policy in ways that explicitly help maintain the distinctive identity and integrity of both science and politics (Guston, 2001; McNie, 2007; Kirchhoff et al., 2013). In the process of facilitating engagements between scientists, policymakers, and stakeholders, boundary organizations accrue experience-based insights into their facilitation role that result in adaptive learning (Briley et al., 2015; Lemos et al., 2014).

This increasingly complicated picture reveals that the engagement of science and policy is a dynamic process that takes many different forms depending on who is involved and what their underlying interests are (McNie et al., 2016). As information flows back and forth between the scientific research community and society, it comes in contact with networks of organizations, actors, and objects (Kalafatis et al., 2015b). Social norms, politics, and culture influence the information people in these networks are exposed to and the ways in which people interpret, alter, and spread the information they receive (van Kerkhoff and Pilbeam, 2017). Scientists can decide to participate in the development of knowledge systems, the series of relationships that coalesce within these networks to connect information with action (Buizer et al., 2010; Cash et al., 2003). Efforts to connect science and policy ultimately require participants to play multiple roles since processes linking science and policy take on different forms from project to project (McNie et al., 2016). However, whatever form these knowledge systems take, successfully informing decision making requires attention to the ways in which the political communities involved construct, review, validate, and deliberate about information (van Kerkhoff and Pilbeam 2017; Miller, 2008). Scientists’ cultivating greater “policy literacy” (Selin et al., 2017)—the extent to which they have a broad and functioning understanding of the public policy process that allows them to engage effectively with the political world—can help scientists navigate expectations that science contribute to policymaking efforts. Scientists can then make explicit decisions about the roles they do or do not want to play in policy making as well as how to engage most effectively.

In order to assess scientists’ (from students to retired professionals) conceptions of how science migrates into policy decisions, a computer-supported drawing analysis was conducted. Researchers have successfully used visual representations to study internal models of processes in a number of studies (e.g., Libarkin et al., 2015; Ross et al., 2013; Smith and Bermea, 2012). Addressing the research goals of the current study required understanding common patterns in how respondents represented these processes. Therefore, to enhance the sophistication of this analysis of patterns beyond the limits of using visual inspection alone, we used SPSS statistical software to conduct an automated grouping analysis (Libarkin et al., 2015).

In the fall of 2016, volunteers attending a professional meeting in the USA were surveyed. Seventy-three pencil and paper surveys related to this study were collected. Seventy-one respondents provided demographic information. The average age of these respondents was 27. Twenty-six identified their gender as male, 42 as female, and 3 identified as genderqueer or other. Regarding their highest earth system science degree attained, 18 had completed a graduate degree (master’s or Ph.D.), 25 had completed an undergraduate degree, and 28 had not yet completed one (27 of these 28 were currently pursuing a degree). These undergraduates were included in this study because earth and environmental science undergraduate students are likely to engage in research (Russell et al., 2007) and the fact these students were attending a major professional conference makes them particularly likely to be actively involved in research. Students attending this event are also likely relatively upper-level undergraduates and will shortly be able to obtain employment in the environmental science field as a scientist (NACE, 2017). In addition, scientists often begin their engagement with policy debates very early in their careers, including as undergraduates, and other studies have incorporated conference attendees in investigations of the science-policy boundary (e.g., Singh et al., 2014).

Surveys included a large empty box preceded by the question prompt: “Scientific research is expected to produce benefits for society beyond advancing scientific knowledge. Draw a picture describing how you think scientific research migrates through society and leads to public policy decision making. Please label any major people, organizations, or processes represented in your drawing.” To generate a comparative perspective on representations, four drawings were also collected from faculty members recognized as experienced “experts” in engagements between science and policy. These faculty members were primarily housed in an interdisciplinary public policy program, a community sustainability program, a fisheries and wildlife program with a focus on the legal aspects of resource management, and a sociology department. This allowed collection of a wide range of expert perceptions.

Due to the role of scientists’ experience and personal interactions in the transfer of science into policy (Lemos and Morehouse, 2005; Lemos et al., 2012; Frank et al., 2012), the survey also included a question addressing the extent to which these individuals engaged with six different kinds of organizations: colleges/universities, non-academic research institutions, private industry, non-profit advocacy organizations, political offices, and governmental organizations. It read: “In the context of your professional career: Please indicate how often you have interacted with people affiliated with the following types of organizations.” For each type of organization, respondents marked a response on a five-point Likert item (never, rarely, occasionally, often, very often). For the analyses below, these responses corresponded with ordinal values from 1 (never) to 5 (very often).

Of the 73 collected drawings, 61 completed drawings were analyzed. Eight respondents elected not to respond to the drawing prompt despite completing other parts of the survey. One wrote a question mark in the space and another wrote “Honestly I’m not exactly sure how this process works.” Two other responses were removed from the analysis because they contained depictions that were uninterpretable and thus uncodable.

The method of Libarkin et al. (2015) was used to identify salient elements of drawings and then evaluate for patterns across drawings via factor analysis, resulting in a set of 16 codes (Table 1). One author coded drawings via thematic content analysis (Patton, 2002) and discussed emergent themes with the second author until an agreed upon rubric was developed. The second author then coded a sample of 10 random drawings to establish inter-rater reliability. The single measures intraclass correlation for this co-coding was 0.90 (min. = 0.87 and max. = 0.92). Since positive correlation is reflected by intraclass correlation close to 1.0, this correlation is interpreted as methodological reliability.

While all thematic codes were grounded in observations from the drawings, codes were also consistent with discussions in the literature that have highlighted the role of events (Frank et al., 2012), organizations (Lemos et al., 2014), and funding (Kalafatis et al., 2015b) in connecting the supply and demand for science. There are reasonably clear distinctions between the people, institutions, and processes associated with this supply and demand for science (Sarewitz and Pielke, 2007). To acknowledge this literature, the thematic codes were organized into supply-side and demand-side groups. Supply-side codes addressed the research process, the individuals performing it, and research results. The demand-side codes represented the application of science and the people and events involved in application. A third grouping of codes, points of exchange, was also added to encompass codes that exhibit connections between supply and demand-side elements, providing opportunities for exchanges to take place. Figure 1 illustrates the application of these codes with a sample drawing featuring eight elements.

With a sample size of 61 and at least five cases per code, this initial data set met the minimum conditions for principal components analysis (PCA), especially considering that this data was being used to analyze patterns in drawings rather than compose a scale. There is mathematical support for the use of PCA on binary data like that used here (Gower 1966, p. 332). Criteria indicating viability of PCA were also met. Most of the initial codes had at least a 0.3 correlation with one or more of the other codes, indicating that a PCA structure could be expected to emerge. Several iterations were conducted in which the code with the lowest communality was removed until every code had a communality of at least 0.3, resulting in the removal of two codes: Institutions and Issues. A rotation was performed to gain as simple, clear, and interpretable a picture as possible of distinctions between different components (Brown, 2009). Following Tabachnick and Fidell’s (2007) approach for selecting between an oblique or orthogonal rotation, an oblique rotation (promax) was performed first. None of the correlations between the resulting components exceeded 0.32, so an orthogonal rotation (varimax) was performed for the final analysis. Eigenvalues, scree plots, and loadings were interpreted to determine the number of components retained (Tabachnick and Fidell, 2007). While eigenvalues ≥1 implied that seven components may be possible, the scree plots indicated the presence of five components. Analysis of codes indicated that five components aligned well with the models of supply-side development of science, the demand-side of its application, and points of exchange determining connections between the two. Identifying science, policy application, and connection elements in each category shed light on the distinctive underlying descriptions about the process of connecting science with policy application that each category described. Finally, a model score was calculated for each emergent model as the sum of model elements present within each individual drawing. The higher a drawing’s model score for a specific model, the more that drawing included elements that aligned with those featured in that model. Readers wishing to assess how individual responses related to the different models in more depth may access an Excel sheet of Supplemental Data¹ for the article that provides the coding scores for each response.

Cronbach’s Alpha was calculated for each of the five models resulting from the PCA analysis to assess internal consistency. A series of Spearman’s correlations were also performed to investigate any relationships that might exist between the five model scores calculated for each drawing (one for each model identified via PCA) and the respondent’s level of engagement with colleges/universities, non-academic research institutions, private industry, non-profit advocacy organizations, political offices, and governmental organizations. Spearman’s correlation was selected because each drawing score consisted of ordinal data.

Mirroring Singh et al. (2014), an analysis of the impact of career stage was also conducted. Therefore, a test was also conducted to assess relationships that might exist between the highest degree that a respondent had obtained in earth system science and the model scores. A Kruskal-Wallis non-parametric test was used because the factor scores were ordinal data and degree attained had three categories: none, undergraduate, and graduate (master’s or Ph.D.). Based on these results, a Dunn’s Test was performed as a post hoc assessment of differences between the categories of degree attainment.

The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.486. While this indicates a larger sample would be useful, we considered the sample sufficient for a pilot study of a new experimental approach. Bartlett’s test of sphericity was significant (χ²(91)) = 99.913, p = 0.031).

Table 2 provides a summary of the results of the principal components analysis. In the final analysis with 14 codes, every code had a communality of at least 0.3 and 11 of the 14 had a communality of at least 0.5. The five-component solution explained 57.75% of the variance in the data. The first component explained 14.93% of this variance, the second 13.77%, the third 11.19%, the fourth 9.41%, and the fifth 8.46%.

Cronbach’s alpha was calculated for each of the models to assess their internal consistency. Factor A had three items and a Cronbach’s of 0.488. Factor B had four items and a Cronbach’s of 0.491. Factor C had four items and a Cronbach’s of 0.439. Factor D had three items and a Cronbach’s of 0.401. Factor E had three items and a Cronbach’s of 0.381. As per Libarkin et al. (2015), these alpha values indicate a reasonable level of consistency for salient features within drawings.

The grouping of salient codes as identified via the PCA suggests five different models underlying scientists’ reasoning about their perceived role in the science-policy diffusion process. Most respondents produced drawings containing elements of multiple models, although a few scientists produced single-model drawings that exemplified the underlying model. Figure 2 provides example single-model drawings to illustrate the characteristics of each model.

In drawings with higher Model A scores (Fig. 2A), scientists produce peer-reviewed research and then engage with non-scientists about their findings. Their efforts to understand the world and explain the results of their research to others is intended to inform voters about important issues, with the hope that this can lead to more informed policies. These scientists perceive that they might participate in knowledge systems connecting information with action as experts whose specialized knowledge enhances other participants’ understanding. In the example of the drawing with a high Model A score provided in Figure 2A, scientists function as people who have informed judgment that others can emulate. Researchers publish papers and then present findings to the general public and build bridges with policymakers. These more informed policymakers can then provide support for scientific research and help inform the public about the scientific foundations for policy. Policy drafts are developed and then voting ultimately determines the policy that results. Based on this perception that the scientist’s role is to be a source of knowledge that is subsequently exchanged and transferred amongst those working on public policy, this is described as the Beacon Model in this paper.

In drawings with higher Model B scores (Fig. 2B), scientists perceive that they contribute to the development of non-scholarly reports and presentations that influence policymakers and the general public. These scientists perceive that they might participate in knowledge systems connecting information with action as colleagues working alongside peers in the policymaking community. In the example of a drawing with a high Model B score provided in Figure 2B, scientists pursue research that is embedded within policy-related activities in several different ways: presentations, policy documents (“geopolicy”), and application in public parks. These actions eventually influence the government, lawmakers, and public. Based on this perception that the scientist’s role is to work alongside those in the policy community, this is described as the Collaborator Model in this paper.

In drawings with higher Model C scores (Fig. 2C), scientists perceive that scientists influence policy outcomes by acting as teachers in the classroom and in the media, rather than directly participating in knowledge systems connecting information with action. They perceive that they enhance peoples’ exposure to information through the media and through their teaching, altering the broader social norms, politics, and culture that shape how people interpret and apply information. In the example of a drawing with a high Model C score provided in Figure 2C, scientists enhance the general capacity of society to make decisions through providing public education and engaging on social media. Based on this perception that the scientists’ role is to act as teachers, rather than directly engaging with those working on policy, this is described as the Educator Model in this paper.

In drawings with high Model D scores (Fig. 2D), scientists perceive that they produce research they believe has value for society and attempt to inform policymakers and the public about their discoveries, but the policymakers and public ignore the scientists’ research because the findings challenge an existing system (e.g., beliefs, interests). They perceive that they cannot find a supportive role to play in knowledge systems connecting information with action even though they are attempting to do so. In the example of a drawing with a high Model D score provided in Figure 2D, an individual scientist is rejected when they attempt to inform other people about their research. A policymaker thinking about money and personal gain ignores the scientists’ efforts and exclaims: “We don’t care!” to a crowd of people. Eight people listen to the policymaker’s message while only one listens to the scientist’s message. Based on the perception that scientists will be ignored when they try to engage, this is described as the Outcast Model in this paper.

In drawings with higher Model E scores (Fig. 2E), scientists perform research on particular issues and publish peer-reviewed papers, but whether their work is used by policymakers is determined by its relevance for use in a particular law or policy. Scientists perceive that they produce information that may or may not enter into a knowledge system connecting information with action. Therefore, they perceive that they play a passive, rather than active, role in policy. In the example of a drawing with a high Model E score provided in Figure 2E, the impact of research is determined by others’ perceptions about its implications, rather than scientists’ actions. Some of this research will be published but will address issues that are only of interest to academics and so won’t have an impact on policy. Some of this research will address issues those outside of academia are interested in, but corporations will block these findings from influencing policy because the corporations perceive that these findings will cost them money. Some of this research concerns dangers that the general public faces and so will be used by policymakers in public policy. Based on the perception that scientists’ role is limited to pursuing research that may or may not have any impact, this is described as the Investigator Model in this paper.

The prevalence of each type of model within participant drawings was documented through calculation of average model scores. This approach was chosen since most drawings contained elements of two or more models. Drawings had an average Beacon Model score of 0.721 ± 0.951 (out of 3). Drawings had an average Collaborator Model and Educator Model score of 0.902 ± 1.091 (out of 4) and 1.377 ± 1.171 (out of 4), respectively. Drawings had an average Outcast Model score of 0.574 ± 0.826 (out of 3) and an average Investigator Model score of 0.656 ± 0.793 (out of 3).

A series of Spearman’s correlations were performed to investigate relationships between the drawings’ model scores and the respondent’s interactions with others. Higher Collaboration Model scores were correlated with higher levels of interaction with private industry (ρ(59) = 0.294, p < 0.05), non-profit advocacy organizations (ρ(59) = 0.290, p < 0.05), and governmental organizations (ρ(59) = 0.257, p < 0.05). Higher Educator Model scores were associated with higher levels of interaction with private industry (ρ(59) = 0.290, p < 0.05). Higher Outcast Model scores were associated with higher levels of interaction with colleges or universities (ρ (59) = 0.362, p < 0.01). Higher Investigator Model scores were associated with higher levels of interaction with non-academic research institutions ρ(59) = 0.294, p < 0.05) and private industry (ρ(59) = 0.298, p < 0.05).

The impact of education level on drawings’ model scores was also investigated. Kruskal-Wallis tests provided evidence for statistically significant variation for only Outcast Model scores and degree attained (χ² (4, N = 61) = 7.138, p < 0.05). Dunn’s Test was performed as a post hoc assessment of differences between the categories of degree attainment and Outcast Model scores. Based on this test, those whose highest degree in earth system science was a graduate degree had a statistically significant (p < 0.05) higher Outcast Model score than those with no degree in this field (e.g., working toward an undergraduate degree).

As noted, four drawings from a range of experts were collected to pinpoint salient features of expert models of the science-policy diffusion process. Analysis of these drawings provides evidence for nuances and range that might be anticipated in individuals with a high level of science-policy understanding.

The first expert model (Fig. 3A) depicts an individual scientist engaging with a policymaker. The scientist describes a theory while the policymaker describes a tangible need they have for information about a decision. This discussion leads to the development of applied research that results in a non-scholarly document. This document sits on a shelf in a policymaker’s office for years. Eventually, the policymaker gets attention in the media by describing how the issue they were previously concerned about has gotten worse. This attention results in a follow-up study that is used to develop a law to address the issue. In the ensuing political debate, policymakers arguing in support of the new policy use the scientific research—encapsulated in the non-scholarly document—as a justification. A graph at the end of their drawing describes the “badness of the problem” on the y-axis and time on the x-axis. An initial study and a follow-up study are pursued as time passes and the social problem gets worse. Ideally, after a law is passed, the policy intervention begins reducing the severity of the problem. This drawing is similar to the Collaborator Model as interactions between a scientist and policymaker lead to the development of reports that eventually connect information with action.

In the second expert model (Fig. 3B), science/knowledge informs preferences about society’s material and economic interests and influences society’s prevailing culture through new findings and technology. In turn, material/economic interests and culture influence choices about the development of science. Science/knowledge also provides information to policymakers for the development of policy and policy shapes the development of science through the allocation of funding for research. Material/economic interests and culture also influence the development of policy through shaping political influence. In this model, the policy itself is not the end of the process because the actual impacts of the policy are mediated by the policy’s implementation. The policy’s impacts ultimately loop back to the top of the model as they affect material/economic interests, catalyzing future scientific research that, ideally, will assess and inform policy improvements. This expert drawing offers an expansion of the Educator Model. The Educator Model described scientists changing the social norms, politics, and culture that influence the interpretation and application of information throughout society. This expert drawing shows science and knowledge influencing (and being influenced by) culture, religion, the economy, and politics.

The third expert model (Fig. 3C) begins with the United States Congress overseeing and funding federal agencies whose missions relate to understanding and managing the natural world such as the United States Environmental Protection Agency. These agencies both fund scientific research in universities and employ many research scientists who conduct their own scientific research related to their agencies’ interests in the natural world. These findings are spread throughout society, through various means, and lead to evidence-based targets or objectives to address the problem through stimulating social pressure and political will. At this point, the federal agencies previously represented in the model identify policy alternatives, including considerations about costs and benefits related to policy interventions and invite comments from the public. The resulting policies emerging from this process are then implemented. In some cases, these policy actions are subject to legal issues that ultimately require the intervention of the judicial system. This drawing presents an alternative view on the Beacon Model. The Beacon Model described knowledge systems connecting information with action where scientists’ expert judgment informs policy. This expert drawing depicts science informing policy as an intermediate step within a larger system where the United States Congress acts as the beacon influencing the development of science through its influence on federal agencies.

In the fourth expert model (Fig. 3D) the scientific community (mostly in universities) interacts with policy decisions primarily through a series of intermediaries. The private sector collaborates with scientists when it sees doing so is in its best interest but can challenge and block the findings of science when science challenges its interests. Non-governmental organizations (NGOs) and social movement organizations can also collaborate with scientists and offer funding while scientists can disseminate their findings to these organizations as well. Scientists can also directly or indirectly advise government officials (i.e., elected officials, their staff, and agency officials) based on requests from these government officials about the state of science related to issues that they address. However, elected officials are also influenced by the perspective of advocacy organizations and NGOs, traditional and social media outlets, and the private sector. Scientists can publicize through these media outlets as well. Government officials’ responses to these inputs result in policy decisions. These policy decisions ultimately rebound back into the rest of the model through influencing funding as well as priority setting considerations in the scientific community, NGOs, and the private sector. This model resonates with the Investigator Model as scientific research being performed affects policy decisions based upon the discretion of several intermediaries: NGOs, the media, elected officials, and the private sector.

Consistent with the discussion in the previous section about scientists describing different potential roles in the complex and amorphous process of exchanges between science and policy, the four models drawn by experts did not consolidate around one single, comprehensive picture of the science-policy diffusion process. However, expert drawings did provide a broader perspective that highlighted important aspects of science-policy diffusion that the models drawn by scientists (e.g., non-experts in science-policy nexus) did not include. The first drawing (Fig. 3A) emphasized the extent to which the passage of time and the development of a social problem affect whether or not science is used in a policy decision. The third expert drawing (Fig. 3C) provides a reminder that much of this development of knowledge for policy application is already built into the government’s infrastructure through major funding agencies. The second expert drawing (Fig. 3B) also points out that the policy process does not necessarily end with a law being passed and is instead iterative over time. Both this drawing as well as the fourth expert drawing (Fig. 3D) illustrates that ultimately policy decisions can come back to inform scientific efforts through determining funding, having impacts that change funding priorities, and the application of science in policy revealing new research questions that need to be answered.

In our analysis, we sought to identify common trends in the perceptions that scientists at all stages of their careers have about their potential role in the migration of science into policy. Our primary finding is that geoscientists’ representations of this process reflect five different models of the role of scientists in the migration of science into policy: beacon, collaborator, educator, outcast, and investigator. Four of these models align with conceptions held by experts working directly at the science-policy boundary, although none of the participants held models as nuanced as those conveyed by experts. We found that the Educator Model was the most prevalent model in the study, indicating that geoscientists primarily view that they influence public decision making by acting as teachers in the classroom and media. However, geoscientists also viewed themselves as beacons, collaborators, and investigators in the policy process. A subset of geoscientists expressed that they acted as outcasts, with little influence on policy overall.

Participant drawings with high Beacon Model scores tended to depict that scientific findings influenced public policy by way of engagements that shape the voting decisions of the general public. The Beacon Model most clearly reflects the notion that scientific understanding will guide policy efforts through clarifying what policies should be pursued (Pielke, 2006). In line with a deficit model of science communication, the Beacon Model implies a one-way flow of knowledge where scientific findings are sufficient for resolving political questions (Gross, 1994). To avoid improperly steering political decisions, scientists should play a beacon role only in situations where policy issues can be resolved empirically, or where scientists can be “honest brokers” who clarify or expand the scope of available policy options (Pielke, 2007). In contrast, the Collaborator Model takes a contextual approach to science communication where scientists form partnerships alongside policymakers to cooperatively develop new knowledge (Gross, 1994). This model emphasizes co-production of knowledge (Lemos and Morehouse, 2005) as interactions between scientists and non-scientists lead to new insights. Consistent with this conception of the potential role of scientists in the science-policy diffusion process, the results of the Spearman’s correlations indicated that those who interacted more with private industry, non-profit advocacy organizations, and governmental organizations made drawings with higher Collaborator Model scores.

The Investigator Model depicted scientists pursuing research whose relevance for application is assessed by those in the government and industry. This is suggestive of the role scientists play in Triple Helix relationships (Etzkowitz and Leydesdorff, 2000) where scientific research is pursued in contexts where academic, industrial, and governmental interests are continuously intersecting with one another. Scores for the Investigator Model were higher for those who interacted more with private industry and non-academic research institutions based on the Spearman’s correlations.

The Educator Model was distinctive from the preceding three models in that it did not address specific efforts to connect science with policy. Instead, it focused on enhancing the general capacity of society to ultimately make informed decisions through education. It reflects a civic epistemological (Jasanoff, 2005) approach to understanding the migration of science into policy and focuses on the social norms and culture that shape engagements between science and society.

While some scientists recognized the potential for an engaged role in policy, other scientists held a view that reflected frustration about their inability to engage effectively with policy makers. The Outcast Model depicted efforts by scientists to engage with the policy process as futile, with scientists being ignored by policy makers. Based on the Spearman’s correlations, those whose drawings had higher Outcast Model scores reported higher levels of interactions with colleges or universities. The Kruskal-Wallis and Dunn’s Test analyses indicated that drawings made by those with a graduate degree in earth system sciences also had higher Outcast Model scores than drawings made by those with no degree in earth system sciences. This sense that their work would be rejected or corrupted aligns with concerns that engagements between science and policy hold the potential to cause more harm than good (Porter and Dessai, 2016) and lead to science and politics inappropriately influencing one other (Guston, 2001; Pielke, 2007). It is possible that these individuals’ sense of alienation from the policy process has resulted from placing their focus on working within academia and not having experiences engaging with more policy-related efforts related to their work. On the other hand, their reported focus on working within academia might have resulted from having had bad experiences in previous efforts to engage with policy. A more nuanced understanding of the relationship between scientists’ experiences and their perceptions about alienation from the movement of science into policy or their perceived role in effective engagement in this process will require future studies.

The results of our analysis provided evidence that scientists recognize a set of distinctive roles that they might experience in the process of science migrating into policy. This finding supports Klenk and Meehan’s (2017) suggestion that interactions between scientists and non-scientists should not be viewed as binary engagements between two predictable, clearly defined sides, but as “encounters” between individuals with varying personal viewpoints. Similarly, research on the development of knowledge systems that connect information with action emphasizes that different projects will require different approaches (McNie et al., 2016), and that those participating in these systems must find ways to adapt to changing conditions over time (Lemos et al., 2014; Briley et al., 2015).

Our findings, however, also revealed a potential sense of futility and alienation from the policy process. Twelve out of the 61 respondents (20%) described science being rejected or corrupted in the process of its migration into policy. Greater policy literacy may help scientists avoid these negative experiences by explicating the four active roles (beacon, collaborator, educator, or investigator) scientists can take. Scientists can find ways in which they can realistically pursue roles related to policy without feeling that their efforts are being ignored or corrupted. Awareness of these different potential roles is a first step. However, cultivating policy literacy is an important next step that must be considered. We first draw on insights gleaned from expert models to build recommendations for policy training.

The first expert drawing (Fig. 3A) emphasized the extent to which the passage of time factors into the application of science in policy. Kingdon (1984) and Zahariadis’s (2014) influential work on agenda setting in policy studies emphasizes the extent to which the application of science in policy is the product of careful analysis offering a tangible solution to a ripening social problem in a political environment that finds this solution appealing. Kingdon (1990, p. 57) describes that “academics, researchers, and consultants” (p. 57) influence critical policy changes both directly when findings are immediately relevant and cited, and through slowly altering how these issues are discussed more broadly over time (Kalafatis et al., 2015a). In discussions in the policy literature at least, the applicability of scientific research for addressing social problems is not lost when a particular politician or law ignores it. Instead, knowledge will continue to be developed and refined so that it can be applied when the right moment arrives. Furthermore, even if information isn’t immediately connected with action through a knowledge system, it can play an influential role in altering the social norms, politics, and culture underlying debate around an issue in ways that will ultimately result in policy change.

Federal agencies are one critical location for preserving and refining scientific findings relevant for public policy. The expert drawings provided a reminder of the extent to which the extensive bureaucracy surrounding research offers important opportunities to integrate research and policy. However, the expert drawings also emphasize that another consequence of the role of time in the diffusion of science into policy is that the process doesn’t end when a law is passed. Policy impacts offer further opportunities for understanding social problems and why efforts to address them are effective or not is an area where scientific research can have a major role to play. As referenced in this drawing, the role of science in continuously evaluating and refining policy after it has been enacted underlies the influential ideals of adaptive management (Holling, 1978) and adaptive governance (Dietz et al., 2003).

The expert drawings also highlighted more clearly ways in which policy decisions inform scientific efforts. This is an aspect prominent in the literature on engagements between science and policy (Gibbons et al., 1994; Jahn et al., 2012, Kalafatis et al., 2015b), but only one of the drawings by the geoscience professionals surveyed drew a line connecting the outcomes of policy decisions with future research decisions in the same way that these two expert drawings did. This is perhaps unsurprising, given that preserving the perceived soundness of research means that scientists must continuously maintain boundaries between their scientific efforts and the political process (Gieryn, 1999). Several types of organizations that scientists might work with (i.e., private industry, NGOs, and media organizations) act as intermediaries in the diffusion of science into policy and back again. Such intermediaries can act as boundary organizations if they are facilitating the transfer of scientific knowledge into policy while allowing researchers to retain their standards of independence (Guston, 2001). There being multiple intermediary organizations that can link scientists to the policy process further emphasizes that expanding geoscience students’ policy literacy enhances their capacity to dictate the role they want to play in this process as professionals. Scientists with a better understanding of how to identify and navigate potential partnerships with intermediaries will also be able to achieve these goals related to engagement more efficiently, helping address burden being placed on scientists already involved in these efforts (Briley et al., 2015).

The preceding analysis and discussion have demonstrated that scientists at all stages of their careers perceive a variety of different roles that they might play in the diffusion of science into policy, and that these perceptions are consistent, albeit less complex, with a variety of models described by experts on these processes and the existing literature. The discrepancy between expert models and those held by scientists indicate that additional training would aid scientists in building more accurate and nuanced views of what it looks like for science to inform policy. At the same time, feelings of rejection can result if scientists’ efforts to play a role in the policy process are inhibited. Training that enhances the policy literacy of scientists through increasing their ability to engage effectively with the political world would cultivate their ability to both define their personal role in the process and fulfill that role more effectively in the context in which they work. The finding that those with a graduate degree in the earth system sciences had higher Outcast Model scores than those who did not yet have an undergraduate degree in the field suggests that upper-level undergraduate and graduate-level training in policy literacy might be particularly crucial for avoiding feelings of rejection developing over the course of a scientist’s career.

Experiential learning is a well-established educational strategy for building such expertise that might be particularly well-suited approach for addressing these issues. Experiential learning aims to cultivate knowledge through processes that put individual conceptions in tension with concrete experiences while encouraging experimentation and personal reflection (Kolb, 1984; Boyer, 1996; Kolb and Kolb, 2005; Butin, 2006; Checkoway, 2015; Hoffman, 2016). Of particular importance is that experiential learning explicitly encourages the learner to use feelings of doubt and discomfort they experience as opportunities to update their understanding of how the world works (Malinen, 2000). Experiential learning related to policy literacy could therefore help scientists learn to use alienation experiences as opportunities to understand engagements between science and public policy better and more clearly define their desired role.

Training of scientists to understand the role of science in policy decisions can occur through curricular (within courses) or co-curricular (outside of courses) means. Within the classroom, training can occur through brief exposures or deep explorations into problem-based learning around case studies where science might relate to policy discussions. Reflection activities—whether in the form of personal or public journals, debates amongst students, or presentations (Bringle and Hatcher, 1999)—can help participating students learn how to critically examine their own values and assumptions so that they can determine what kind of role they might want to play in policy discussions. Inviting policymakers or scientists with experience engaging with policymakers as guest speakers in classes or through departmental talks is a simple mechanism for bringing policy to scientists. Dialogue about how policymakers balance the concerns that they have in their everyday decision-making can help students understand how science does (and does not) actually factor into policy. These engagements with different perspectives can encourage students to reconcile their previous assumptions with a more complex picture of how things work (Keen and Hall, 2009). However, successfully integrating these engagements into the curriculum requires including opportunities for students to reflect on disruptions between previous training and their experiences so that they can make sense how they can adjust their understanding of their role in society as geoscientists (Bringle and Hatcher, 1999).

The impact of such reflective activities in experiential learning is more likely to persist if it is extended through multi-term programs that reinforce reflection across different contexts (Mitchell et al., 2015). Therefore, internship experiences that encourage science students to work in policy relevant contexts could have more impact if they integrate reflection activities that complement efforts integrated into coursework. Removing institutional barriers to community engagement that persist in academia—a lack of support for long-term engagement, commitment to service, and recognition of service-oriented research and teaching in faculty promotion decisions (Ostrander, 2004; Freeman et al., 2009; Seifer et al., 2012)—would help encourage the development of such student civic engagement activities. Despite the presence of such barriers, many scientists have actively pursued and demonstrated the value of pairing undergraduate student training in the geosciences with engagement (Savanick and Fortner, 2016), such as through the National Association of Geoscience Teachers’ Serving Our Communities Initiative.

Formal education can provide an important introduction to policy for scientists typically isolated into disciplinary silos. Still, learning should not need to end once a terminal degree is acquired. Several policy fellowships specifically geared toward scientists offer opportunities for scientists of all ages to learn about the policymaking process and the role science does and can have in the development of policy. For example, the Congressional Science Policy Fellowships offered by many U.S. professional societies offer hands-on experience building connections between science and policy makers. The American Geophysical Union’s (AGU) Thriving Earth Exchange connects community leaders with volunteer scientists to support local priorities while AGU’s Sharing Science program offers training related to enhancing the ability of scientists to communicate with policymakers, journalists, and the public. The American Association for the Advancement of Science’s Policy Fellowship Program has over 1000 alumni who have served congressional offices and in government agencies (Teich, 2002). About one-third of these fellows continue working in Washington, D.C., USA, following the completion of their fellowship while another one-third return to academia (Teich, 2002). The presence of such a large alumni network provides particularly rich opportunities for future research to understand how experiences of fellows have shaped their impressions of their scientific training. Insights from their reflections on how their scientific training did (or did not) prepare them for their fellowships as well as how their intensive fellowship experiences shaped their own understandings of being a scientist could provide further insights for enhancing science curriculum.

The National Research Council has asked that scientists enhance their ability to engage with policymakers through “learning by doing” (NRC, 2010, p. 9). However, persistent concerns about harms resulting from these engagements (Porter and Dessai, 2016) as well as the potential for rejection expressed in the drawings examined indicate that more training might be needed than simply learning on the job. Developing training that facilitates reflection within experiential education curriculum might help prepare geoscientists to both define and fulfill their personal desired roles in the diffusion of science into policy more successfully. The ways in which scientists engage in the policy debate can have social justice implications, and it is interesting to note that engagement in policy for social justice reasons was not explicit in any of the participant models. The underlying rationale for engaging in policy debate will inform how that engagement occurs. Future research should consider the motivations scientists have for conducting policy work and engaging with communities.

The findings of this study emphasize the multiple different ways that scientists might choose to engage in connecting science with policy. Training scientists to understand how to pursue policy work more effectively might not only make scientists’ experience of these engagements more fulfilling but could also lead to better public policy outcomes. In messy democratic processes, beacons, collaborators, educators, and investigators can all play meaningful roles in informing policy decisions. Training should therefore incorporate all avenues for engaging in policy. Single-minded training that limits the ways in which scientists may engage with policy efforts might inadvertently shut off important avenues for supportive participation, leading scientists to feel like outcasts and deepening divides between science and society by distorting how they interact with one another. Scientists being able to participate in informing policy more often and more effectively can result in more informed and well-rounded policy decisions. Ultimately, training should strive to provide scientists with the knowledge and skills they need to become effective stakeholders in decision making.

Data collection was made possible through personnel and in-kind support of the Geocognition Research Lab. We thank all participants in our research for their efforts in this work and offer thanks for the helpful insights provided by the reviewers and editors. We also thank the Dickinson College Research and Development Committee for its funding support.