A Washington Post Poll found that “70%” of Americans support regulation of green house gases.
The first thing to do, always, is take a close look and see if one accepts that a survey item of this sort is indeed worded in the manner that supports how its results are being characterized.
After that, the question to ask is, “What is the survey item actually measuring?”
The answer is usually “nothing,” or nothing of consequence.
If the item refers to a policy that members of the public don’t know or think about–something that doesn’t figure in their everyday interactions with other ordinary people–then the survey is not modeling anything going on in the world we live in.
Consider: About half the respondents in a general population survey won’t know— or even have good enough luck to guess– the answer to the multiple-choice question “how long is the term of a U.S. Senator?”
Are we really supposed to base an inference about what people with that level of political engagement are thinking from the responses of the 1,000 who get weirdly transported out of their everyday worlds and asked (by aliens, it must seem) to indicate whether they “approve” of, say, “the NSA’s metadata collection policy?” (“hell, I’m for it– college athletes are there to be educated!”)
The Washington Post tells us that an “overwhelming majority of Americans” support regulating CO2 emissions.
But given that only 58% (+/- 3%) of a general population telephone sample know that “carbon dioxide” rather “hydrogen,” “helium,” or “radon” is the “gas … most scientists believe causes temperatures in the atmosphere to rise,” what exactly did the “overwhelming majority” of the weird-lottery winners called by “Langer Research Associates” understand “greenhouse gases” to be?
There’s zero correlation between responses to the “which gas” question and party affiliation. That’s how one knows that it isn’t measuring the same thing as a survey item asking ordinary people whether they believe in human-caused global warming.
That’s a question fraught w/ meaning — not as a matter of policy or science, necessarily — but as an element of their social world.
It’s one of those things — along with abortion and gun control — that separate “us” from “them.”
Indeed, the issue has split people right down the middle — about 50% have answered “yes” & 50% “no” to the “human-caused global warming” item –for many years. And no, that has not changed recently: 48%, in a nationally representative Cultural Cognition Project survey conducted last month.
What are we supposed to think when told that an “overwhelming majority” of Americans said they “support” a policy that regulates human activities responsible for climate change even though Americans are divided 50-50 on whether human behavior is even causing global warming?
But even that takes responses to the the Washington Post survey item waaaaaaaaaay too seriously.
What valid public-policy survey items measure is an affective orientation — a feeling that is either positive or negative, strong or weak.
Such orientations can be of extreme importance.
They can propel people to disregard serious health risks (e.g., lung cancer from smoking) & shrink in terror from non-existent ones (autism from vaccines).
They can determine who they vote for for President (or for the House of Representatives or for anything else)–or who they marry or try to kill.
Often, such affective sensibilities are an expression of a vital element of the respondent’s self-conception, one more convincingly seen as a cause than as a consequence of how that person makes sense of all manner of evidence and information, from raw data to brute sense impressions.
But that’s what the response is–a register of an affective orientation. It’s not an argument or an idea — not anything like what you would make of a statement that a person in a conversation uttered on his or her own accord.
Same for hypothetical “willingness to pay” measures: if they are genuinely connected to something meaningful for people, responses to them express an attitude– but are not a valid predictor of the willingness to pay for anything.
Psychometrically speaking, survey responses are “indicators” or indirect measures of a “latent” or unobserved “variable” or influence. They need to be validated — i.e., shown by independent means (including coherence with other survey items) to be measuring what one thinks they are measuring–and even then must be regarded as only noisy or imprecise approximations.
Or practically speaking, someone who is genuinely motivated to understand public opinion treats responses to any particular public-policy survey item as one of many ambiguous pieces of evidence.
When connected to other pieces of evidence including responses to other items that cohere with one another, the responses often support inferences — solid ones — on the basis of which we can predict behavior and explain states of affairs.
But a particular item by itself supports no reliable inference.
Likewise, if an outlier survey item (or even a collection of them) invites an interpretation contrary to the ones borne by other, conventional ones known to be valid and to support reliable inferences, the “hey look” item more than likely isn’t measuring the affective sensibility that genuinely motivates the real-world attitudes the pollster is purporting to model.
As I’ve explained before, the “industrial strength” risk-perception item –the rating of a putative risk source on a multi-point scale — elicits a straightforward pro- or con- expression of attitude that can vary within a meaningful but relatively constrained range. At least where the putative risk is something members of the general population have had experience with in the world, responses to ISM can be expected to correlate highly with particular perceptions or factual beliefs relating to that same object. It can be expected to be correlated with forms of behavior that fit or express the sensibility that it elicits.
But note the emphasis on correlation.
By itself, responses to ISM are meaningless; the scale is arbitrary (“OMG, the public’s perception of the risk of ‘private ownership of guns’ is only 3.4!”).
Its interpretive utility lies in its covariance with other characteristics — e.g., w/ “cognitive reflection” or “numeracy” in the case of some risk perception hypothesized to be connected to over-reliance on heuristic reasoning.
Or w/ “ideology.” Here is the ISM for “global warming” in relation to political outlooks:
The correlation displayed is quite strong. Indeed, the correlation between the respondents’ ISM rating and their right-left political outlooks is as high as the correlation between the two items — partisan self-identification (on a 7-point measure) and liberal-conservative ideology (5-point) –that were combined to form the left-right outlook scale itself (r = – 0.64, p < 0.01).
Well, believing in human caused global warming, unsurprisingly (the two together support the inference that each measures what it seems to).
It also correlates with positions on very familiar, very strongly contested issues like gun control and abortion.
ISM also correlates with individual characteristics, like being white & male, and hierarchical and individualistic, that are known to be indicators, too, of a group disposition that generates strong negative reactions to the issue of global warming (or “climate change”; changing the labe doesn’t have any material effect — which is to say, “believe in climate change?” and “believe in global warming?” are both valid, if noisy, indicators of the same latent affective disposition).
Look: The affective sensibility that motivates cultural polarization on climate is real.
It can’t be exorcised by magic words.
It won’t abate if people rely on lab experiments to justify “messaging campaigns” that have been shown decisively not to work by over a decade of real-world evidence.
Yrs of advocacy polls designed to create “overwhelming majority” support for “action on climate change” by insisting it already exists have proven their valuelessness too.
The only way to ameliorate the destructive impact that the climate conflict is having on our capacity for enlightened self-government is to extricate the scientific issues it turns on from the ugly, illiberal form of status competition that now engulfs them.
And we won’t figure out with wishful thinking & meaningless measures.