This is part 3 of a series on external validity problems with climate-science-communication studies.The problem, in sum, is that far too many researchers are modeling dynamics different from the ones that occur in the real world, and far too many communicators are being induced to rely on these bad models.
In my first post, I described the confusion that occurs when pollsters assert that responses to survey item that don’t reliably or validly measure anything show there’s “overwhelming bipartisan support” for something having to do with climate change.
In the second, I described the mistake of treating a laboratory “messaging” experiment as better evidence than 10 yrs of real-world evidence on what happens when communicators expend huge amounts of resources on a “scientific consensus” messaging campaign.
This post extends the last by showing how much different real-world scientific-consensus “messaging” campaigns are from anything that is being tested in lab experments.
All of these are exercpts from a paper I’ll post soon — one that has original empirical data relating to what measures what in the study of climate-change science communication.
5. “Messaging” scientific consensus
b. What is the “message” of “97%”? “External invalidity” is not an incorrect explanation of why “scientific consensus” lab experiments produce results divorced from the observable impact of real-world scientific-consensus “messaging” campaigns. But it is incomplete.
We can learn more by treating the lab experiments and the real-world campaigns as studies of how people react to entirely different types of messages. If we do, there is no conflict in their results. They both show individuals rationally extracting from “messages” the information that is being communicated.
Consider what the “97% scientific consensus” message looks like outside the lab. There people are likely to “receive” it in the form it takes in videos produced by the advocacy group Organizing for Action. Entitled “X is a climate change denier,” the videos consist of a common template with a variable montage of images and quotes from “X,” one of two dozen Republican members of Congress (“Speaker Boehner,” “Senator Marco Rubio,” “Senator Ted Cruz”). Communicators are expected to select “X” based on the location in which they plan to disseminate the video.
The video begins with an angry, perspiring, shirt-sleeved President Obama delivering a speech: “Ninety-seven percent of scientists,” he intones, shaking his fist. After he completes his sentence, a narrator continues, “There’s not a lot of debate left in this debate: NASA and 97% of the nation’s scientists agree . . .,” a message reinforced by a cartoon image of a laboratory beaker and the printed message “97% OF SCIENTISTS AGREE.”
After additional cartoon footage (e.g., a snowman climbing into a refrigerator) and a bar graph (“Events with Damages Totaling $1 billion or More,” the tallest column of which is labeled “Tornadoes . . .”) , the video reveals that X is a “CLIMATE CHANGE DENIER.” X is then labeled “RADICAL & DANGEROUS” because he or she disputes what “NASA” and the “NATIONAL ACADEMY OF SCIENCES” and “ 97% of SCIENTISTS” (bloc letters against a background of cartoon beakers) all “AGREE” is true.
What’s the lesson? Unless the viewer is a genuine idiot, the one thing she already knows is what “belief” or “disbelief in” global warming means. The position someone adopts on that question conveys who he is—whose side he’s on, in a hate-filled, anxiety-stoked competition for status between opposing cultural groups.
If the viewer of “X is a climate denier” had not yet been informed that the message “97% of scientists agree” is one of the stock phrases used to signal one cultural group’s contempt for the other, she has now been put on notice. It is really pretty intuitive : who wouldn’t be insulted by someone screaming in her face that she and everyone she identifies with “rejects science”?
The viewer can now incorporate the “97% consensus” trope into her own “arguments” if she finds it useful or enjoyable to demonstrate convincingly that she belongs to the tribe that “believes in” global warming. Or if she is part of the other one, she can now more readily discern who isn’t by their use of this tagline to heap ridicule on the people she respects.
The video’s relentless use of cartoons and out-of-proportion, all-cap messages invests it with a “do you get it yet, moron?!” motif. That theme reaches its climax near the end of the video when a multiple choice “Pop Quiz!” is superimposed on the (cartoon) background of a piece of student-notebook paper. “CLIMATE CHANGE IS,” the item reads, “A) REAL,” “B) MANMADE,” “C) DANGEROUS,” or as indicated instantly by a red check mark, “D) ALL OF THE ABOVE.”
The viewer of “X is a climate denier” is almost certainly an expert—not in any particular form of science but in recognizing what is known by science. As parent, health-care consumer, workplace decisionmaker, and usually as citizen, too, she adroitly discerns and uses to her advantage all manner of scientific insight, the validity and significance of which she can comprehend fully without the need to understand it in the way a scientist would. If one administers a “what do scientists believe?” test after making visible to her the signs and cues that ordinary members of the public use to recognize what science knows, she will get an “A.”
Similarly, if one performs an experiment that models that sort of reasoning, the hypothesis that this recognition faculty is pervasive and reliably steers the members of culturally diverse groups into convergence on the best available evidence will be confirmed.
But the viewer’s response to the “97% consensus” video is measuring something else.
The video has in fact forced her to be become another version of herself. After watching it, she will now deploy her formidable reason and associated powers of recognition to correctly identify the stance to adopt toward the “97% consensus” message that accurately expresses who she is in a world in which the answer to “whose side you are on?” has a much bigger impact on her life than her answer to the question “what do you know?”