In The Weeds – Himalayan Blackberry
About 140 years ago, the prolific horticulturalist Luther Burbank of Santa Rosa, California received...
SIERRA STREAMS
Watershed Monitoring, Research, & Restoration
It has been a while since we last discussed our 7 principles of sound science. As a reminder, we’re putting out this series in an effort to shine more light on what science is and isn’t in an era of increasing misuse of science or misinformation about science.
In these interesting times of politicization of science, we’ve decided we can’t sit idly by, taking a quiet backseat to public discourse as we often do at SSI, but instead speak up against what we see as the erosion of public understanding and trust in science. While recent analyses point to continued public trust, that trust doesn’t “trickle up”.
The following points have already been released in prior blogs (for points 1-3) or will be released over a series of blog posts in the coming months:
But today is a really significant one for me; Science doesn’t make decisions or tell you how to live your life, it only provides information.
Fair warning that this essay may touch on some topics that people feel very strongly about, but let me be clear from the start: the intent of this is not to challenge your personal worldview; it is to educate on origins of misinformation and discuss the very real problem of deciding when to enforce policy based on science or not. If you are open to discussion of what science is, where biases influence results in all directions, and separating science from policy, then read on.
So let’s go back to the very first post in this series, and the idea that science is knowing when to change your mind. The example I used was about eggs being healthy or not:
Eggs are good for you.
But then they’re bad for you.
But then they’re good for you.
Who do I believe?! If scientists can’t even agree on if an egg is healthy for me, why should I trust scientists?!
In that post I discussed two separate points: 1) the original research arguably didn’t even make the claim that eggs were good or bad in general, but instead reported on some finding about one particular protein or fat or acid that is “good” or “bad” for us and this was misinterpreted/generalized in popular media, and 2) speaking definitively by calling it “good” or “bad” is already inherently unscientific and not what should be gleaned from the research, which is simply providing a component or mechanism we can use to start to define what “good” and “bad” may mean.
Here, we take it a step further; what you do with that information is entirely up to you. Perhaps you eat more eggs when you hear they’re good and less when you hear they’re bad. Perhaps you don’t change your behavior at all. Perhaps you don’t eat eggs but you look up the chemical constituent they’ve identified as good and start taking that as a supplement, or eliminating other sources of the constituent identified as bad. Science didn’t make that decision for you, nor should it.
Science is a process by which information is gathered, analyzed, and added to the coffers of other prior knowledge to continue to understand the world around us. This process involves data, statistics, models, error, and uncertainty. Something “not being science” doesn’t make it “wrong”, nor does it mean you can’t happily believe in or live your life by something that doesn’t have a scientific mechanism.
Much like our prior blog posts, I’d argue from the get go that the primary thing we continue to struggle to communicate as scientists to the community is one of the underlying and most foundational concepts in science: uncertainty. What does it mean to “know” something? This question can itself be split into realms of things like religion, spiritualism, metaphysics, and simple observation, but for now we’ll stick with a simple scientific definition; we can observe it, measure it, and collect data on it to see if its a recurring thing or a random occurrence. Through statistical analysis of patterns we can start to fully understand a phenomenon in nature, perhaps its mechanism/what may cause it, or ask questions about drivers of a natural phenomenon and how they may change due to certain changes in the system around it. Once enough information has been gathered related to those questions (i.e. a “preponderance of evidence”), sharing of results through publications and other dissemination outlets lets people receive and digest (or challenge) that information. A key point here is that the process of dissemination is still focused on the primary result(s) and the experiment or observation itself; were the right questions asked, were the data gathered and analyzed in a fair way, and why or why not? The purpose of this stage of dissemination is not to then guide usage of the information, beyond a few bits of conjecture or suggestion that may show up in a discussion section of a paper to spark continued interest or new questions (more on the publication process in an upcoming post), but is instead to collectively scrutinize a result of a study that gives us just that much more insight into that particular topic. In other words, the science is what is presented, reviewed, and debated, not what you then are supposed to do with that information.
Making decisions based on science can be difficult. As an individual, we have our own experiences, observations, and worldviews that may compete with a new scientific idea. We have observational evidence or personal examples of something counter to the science, or that make us question the science. But we may be an outlier the represents the exception more than the rule. Or we may actually be within a range of uncertainty presented in the study, where our own experiences align with the results, but a predetermined opinion or worldview make it difficult to accept the finding. Both are okay! Our experiences and worldviews are valid, but we also need to be able to self-reflect and determine when we may be blocking ourselves from receiving new information because of a personal bias. Now think of applying this to policy-making, and imagine that all of those making the decision and all of those affected by it have their own personal experiences and opinions; applying policy based on science in this case could get difficult! That is the human element, and its a difficult one.
In effect, making a decision to enforce a policy of some sort (say, regional forest management decisions, or regulating water discharge) based on science is embracing uncertainty, trusting data and expertise, and trying to place the collective “good” above any one self. When someone says they are making a decision “based on the best available science”, I like to imagine they are effectively saying two things (even if they don’t mean to): 1) we think this is the right thing to do based on interpretation of current science and 2) we are acknowledging that this is simply the current understanding of this problem, but that may change, as its likely changed before.
Two interesting examples to me of the difficulties in science-based policy-making are Climate Science and Vaccine Science.
I personally find many of the discussions that come up around climate science frustrating and disappointing, not because debate shouldn’t occur, but because of what people debate and why, which have turned simple direct observation and measurement into politics. The basics of climate science are fairly simple irrefutable physical laws and relationships; CO2, one of many “greenhouse gases”, is called a greenhouse gas because of its ability to, when in the atmosphere, allow incoming shortwave radiation from the sun to enter the atmosphere, with only a portion of it reflected back to space, trapping a large portion of that energy along with outgoing longwave radiation from the earth itself. Essentially: more greenhouse gases = more insulation/a thicker blanket, with the earth snuggled underneath and the blanket being like a two-way mirror. This basic principle has long been established (through both simple theoretical physics and many, many repeated observation of this phenomenon), to the point that the basic principle is no longer directly tested in all studies of climate; it is essentially a “given” that we don’t need to re-ask at the start of any study on climate(…see our post of science being “built on the shoulders of giants” for more on how this works!). Now, in the climate science world, much of the new science comes from testing highly detailed, nuanced responses and interactions in complex models of how many physical, chemical, and biological processes interact in what are called “global circulation models”. There are global efforts to compare all of these models, test them versus observation, and continue to iterate on them until they become an approximation of reality.
This last part is important and something I am often a bit of a broken record for: models are “representations of real systems”, and “all models are wrong, some are useful”. No model, particularly one as complex as current climate models, could ever accurately predict or forecast every event or case that may arise at small/local scales; instead, models are used to understand how components of a system interact, and ask what might occur, giving targets to both test for in the real world and continue to improve the model. All models also include representations of uncertainty; how “confident” are we in the range of predictions? Climate models typically have a high degree of uncertainty the farther you get out from current or observed climate, but model building and testing allows us to continue to cut down on that uncertainty as move into the future, but that uncertainty will never be zero, as that is not possible in a predictive model.
So how then does a government agency, municipality, or society make decisions about what to do with the results of a model? As discussed above, that is an extremely complicated question whose answer may be a reflection of each individual’s views on uncertainty, government responsibility, and decision making. One of the interesting complications here is that climate science includes the outsized influence of humans (and our decisions) on global atmospheric conditions. Efforts like the UN IPCC were formed with a goal of closing that gap; taking research from the lab to the shared global decision making table. This resulted in incorporation of human “scenarios” into current models, representing what impacts shifts in global carbon emissions would have on climate. Still, here, the “science” part of the component is modelling itself, which can incorporate (and even be part of the research to more clearly define) human behaviors that directly and indirectly feed climate change through measurable impacts on real physical processes. But now you have human economic systems and well-being tied into models where decisions that would affect those models also would impact those systems. Confused yet? Add in political agendas, party dynamics, economic codependence, and good old fashioned ego, and you get attempts to make science-informed decisions that eventually boil down into generalizations politicization. Further still, you get expectations of specific results; if an agency decides to take action to implement policy based on climate models, (which are time and time again correctly predicting climatic trends) but a predicted event or shift doesn’t occur immediately, the agency looks like they made a bad decision, even though the model likely wasn’t predicting exact locations, times, etc. But no action and eventual event impacts can have real and disastrous consequences…but are likely more politically defensible because they didn’t have to make the more difficult decision. Only in the final alternative—action is taken and a specific event occurs—do both the decision maker and the scientist “win”, yet this is the rarest case in the real world.
In my ideal world, experts in each field would lead the decision making process on issues related to those fields, but there is an immense ethical conversation behind that; are “experts” considering tradeoffs and impacts on people’s lives rather than simply the response to a model? What is a truly “equitable” decision about something as broad as climate change, when drivers include some of the very foundations of modern global economics (fossil fuels)? This is also an example of one of my favorite (though arguably somewhat frustrating) quotes about science and scientists. Scientists are often great at finding or defining problems, but solutions come from engineers and policy makers. Science on its own can define mechanism and potential outcomes from changing something in a system; whether we do that is a question of human will and consensus decision making. In the case of climate decisions, a policy based in the science acknowledges both the demonstrated trends in warming as well as the notion that new information is coming in daily, and we must continue to be flexible to new information.
Let’s be clear: statistically speaking, vaccines (in the general sense…covering some of the primary historical ones such as MMR, polio, etc., and vaccines as a concept or approach to disease mitigation) have extensive evidence of saving lives and increasing community immunity to communicable diseases. When I say statistically speaking, this does not mean there are not side effects, or that people should not have a right to decide if they vaccinate their families. Side effects are statistical outliers—blips on the radar of total “average” impact—but they are very real for you and your family. Politicization of vaccines is a fairly recent phenomenon at the scale it has reached, and has stemmed from a misunderstanding of side effect likelihood, statistics in general, over-generalization of a term (“vaccine”), and frankly from another example of pseudoscience (some would argue much of the modern questioning of vaccine science stems from a now-retracted (meaning the author admitted the science was flawed) paper on links between vaccines and autism). This is only one example, and vaccines vary in method of effect, general chemistry, etc., but the underlying doubt expanded. The simple underlying doubt makes complete sense, as nobody is wrong to question what goes into your body and make the most informed decision for you and your family that you can. But what is important here is to disconnect the personal worldview and experience from the biology, the mechanism, and the data on general efficacy and disease proliferation. This is the separation of science from personal choice.
When thinking of the similarities and differences between vaccine science and climate science, again think of where the research comes in versus the decision making. In vaccine “science”, decades of basic research around individual diseases typically leads to a new model/understanding of the mechanism by which a disease causes death or illness. The “science” here would be looking at how removing or adding a particular condition influences the proliferation or cessation of disease, only then can that science be turned into action through development of a drug that creates those conditions. What is done with that drug and whether or not versions of that drug are then required via policy is then similar to the climate science problem: policy would be definitive but based on uncertainty, and will undoubtedly not work perfectly for every stakeholder, while still being “based on the best available science”.
(Long XKCD comic about vaccines follows…essay continues below!)
This whole essay does have a major caveat, however, that we all need to acknowledge: scientists are people! People are fallible. Science, as an unbiased process, and when done properly, is arguably not. With regards to vaccines—yes, obviously pharmaceutical companies have a financial incentive to play up success and downplay side effects. With regards to things like manufactured chemicals—yes, obviously chemical companies have a financial incentive to play up success and downplay negative impacts. But, once again, if the scientific process is fully followed, these issues should be “caught” in the act (albeit sometimes very slowly), and only through replication and continued building of evidence can true good information “rise to the top”.
Let’s also be clear: I am not a vaccine expert, but you know who I defer to? Those who have been conducting this research for…more than a century. When we have questions, we should put those questions in the context of the state of the field and the expertise that currently exists. We should acknowledge that when we have a question after thinking about a particular phenomenon or field, that those experts have likely (if doing good science) asked those same questions many times over! Why this particular carrier fluid? Is it toxic? How do we know or not know? Have we tried this other one instead? What are the statistical differences in efficacy versus side effects by dose? Was that study done on a group representative/broad enough to make general inferences about the general population? These are all very real questions, relevant to us as “end users”, but ones that are asked constantly in the field by people with the background and ability to dive deep into those questions, share their results with colleagues, and have informed debate among experts. Note, though, that maybe they haven’t, and then you should reach out and help them identify a new question, but after reviewing the literature or asking around in the realm of the experts!
Doing science, supporting science, and making decisions based on science are all different things. What you do with that information, is also up to you.
Science is a process, and one that is doable by anyone, but is also based on the shoulders of giants. So next time a question comes up about the nuance of a particular decision regarding using scientific information, take a skim of some of the literature in that field, see what questions have been asked, and take a deep dive to stay informed, but also embrace humility and curiosity. And with respect to the current political climate around science, just remember one thing: saying something louder doesn’t make you “right”; backing it up with evidence, uncertainty, and humility certainly can get you most of the way there.
