My paper Ideology, Motivated Reasoning, and Cognitive Reflection was published today in the journal Judgment and Decision Making.
I’ve blogged on the study that is the focus of the paper before. In those posts, I focused on the relationship of the study to the “asymmetry thesis,” the view that ideologically motivated reasoning is distinctive of (or at least disproportionately associated with) conservativism.
The study does, I believe, shed light on (by ripping a fairly decent-sized hole in) the asymmetry thesis. But the actual motivation for and significance of the study lie elsewhere.
The cultural cognition thesis (CCT) holds that individuals can be expected to form risk perceptions that reflect and reinforce their connection to groups whose members subscribe to shared understandings of the best life and the ideal society.
It is opposed to various other accounts of public controversy over societal risks, the most significant of which, in my view, is the bounded rationality thesis (BRT).
Associated most prominently with Kahneman’s account of dual process reasoning, BRT attributes persistent conflict over climate change, nuclear power, gun control, the HPV vaccine, etc. to the public’s over-reliance on rapid, visceral, affect-laden, heuristic reasoning—“System 1” in Kahneman’s terms—as opposed to more deliberate, conscious, analytical reasoning— “System 2,” which is the kind of thinking, BRT theorists assert, that characterizes the risk assessments of scientists and other experts.
BRT is quite plausible—indeed, every bit as plausible, I’m happy to admit—as CCT. Nearly all interesting problems in social life admit of multiple plausible but inconsistent explanations. Likely that’s what makes them interesting. It’s also what makes empirical testing—as opposed to story-telling—the only valid way to figure out why such problems exist and how to solve them
In my view, every Cultural Cognition Project study is a contribution to the testing of CCT and BRT. Every one of them seeks to generate empirical observations from which valid inferences can be drawn that give us more reason than we otherwise would have had to view either CCT or BRT as more likely to be true.
In one such study, CCP researchers examined the relationship between perceptions of climate change risk, on the one hand, and science literacy and numeracy, on the other. If the reason that the public is confused (that’s one way to characterize polarization) about climate change and other risk issues (we examined nuclear power risk perceptions in this study too) is that it doesn’t know what scientists know or think the way scientists think, then one would expect convergence in risk perceptions among those members of the public who are highest in science literacy and technical reasoning ability.
The study didn’t find that. On the contrary, it found that members of the public highest in science literacy and numeracy are the most divided on climate change risks (nuclear power ones too).
That’s contrary to what BRT would predict, particularly insofar as numeracy is a very powerful indicator of the disposition to use “slow” System 2 reasoning.
That science literacy and numeracy magnify rather than dissipate polarization is strongly supportive of CCT. If people are unconsciously motivated to fit their perceptions of risk and comparable facts to their group commitments, then those who enjoy highly developed reasoning capacities and dispositions can be expected to use those abilities to achieve that end.
Now enter Ideology, Motivated Reasoning, and Cognitive Reflection. The study featured in that paper was aimed at further probing and testing of that interpretation of the results of the earlier CCP study on science literacy/numeracy and climate change polarization.
The Ideology, Motivated Reasoning, and Cognitive Reflection study was in the nature of experimental follow up aimed at testing the hypothesis that individuals of diverse cultural predispositions will use their “System 2” reasoning dispositions opportunistically to form culturally congenial beliefs and avoid forming culturally dissonant ones.
The experiment reported in the paper corroborates that hypothesis. That is, it shows that individuals who are disposed to use “System 2” reasoning—measured in this study by use of the Cognitive Reflection Test, another performance based measure of the disposition to use deliberate, conscious (“slow”) as opposed to heuristic-driven (“fast”) reasoning—exhibit greater motivated reasoning with respect to evidence that either affirms or challenges their ideological predispositions.
The evidence on which subjects demonstrated motivated reasoning concerned how “closed-minded” and “unreflective” individuals of opposing ideologies are.
Closed mindedness” is a very undesirable trait generally.
It’s also what those on each side of politically polarized debates like the one over climate change identify as the explanation for the other’s refusal to accept what each side sees as the clear empirical evidence in favor of its own position.
One might thus expect individuals who have a stake in forming perceptions of facts congenial to their cultural commitments to react in a defensive way to evidence that those who share their commitments are less “open-minded” and “reflective” than those who harbor opposing commtiments.
So I tested that. I advised subjects that psychological evidence suggests that the Cognitive Reflection Test measures “open-mindedness” (some psychologists take that position; I actually think they are wrong—as I’ll explain in a moment!). Members of a control group were told no more than this. But subjects in two other groups were told either that climate change “skeptics” score higher than climate change “believers” or vice versa.
I found that subjects displayed motivated reasoning with respect to the evidence of the “validity” of the Cognitive Reflection Test as a measure of “open mindedness.” That is, they credited the evidence that the CRT is a “valid” test of “open-mindedness” and “reflection” much more readily if they were advised that individuals who hold the climate-change position consistent with the subjects’ ideologies scored higher, but rejected that evidence when they were informed that those same individuals score lower, than individuals with the opposing position on climate change.
Moreover, this tendency was highest among individuals with the highest Cognitive Reflection Test scores.
That finding is highly inconsistent with BRT, which assumes that a deficit in System 2 reasoning capacities explains the failure of the members of the public to converge on conclusions supported by the best available decision-relevant science.
But it very much consistent with CCT, which predicts that individuals will use their System 2 reasoning capacities strategically and opportunistically to reinforce beliefs that the their cultural group’s positions on such issues reflect the best available evidence and that opposing groups’ positions do not.
It’s consistent, too, with a growing collection of findings in political psychology. This research shows not only that ideologically motivated reasoning drives political polarization (generating perverse effects, e.g., like hardening of commitment to mistaken beliefs when “fact checkers” try to correct false claims), but also that this effect intensifies as individuals become more sophisticated about politics.
Some could have attributed this effect to a convergence between political knowledge and intensity of partisanship. But the result in my study makes it more plausible to see the magnification of polarization associated with political knowledge as reflecting the tendency of people who simply have a better comprehension of matters political to use their knowledge in an opportunistic way so as to maintain congruence between their beliefs and their ideological identities. (I’ve addressed before how “cultural cognition” relates to the concept of ideologically motivated reasoning generally, and will even say a bit more on that below.)
As for the asymmetry thesis, the study also found, as predicted, that this tendency was symmetric with respect to right-left ideology. That’s not what scholars who rely on the “neo-authoritarian personality” literature—which rests on correlations between conservativism and various self-report measures of “open-mindedness”—would likely have expected to see here.
Interestingly, I also found that there is no meaningful correlation between cognitive reflection and conservativism.
The Cognitive Reflection Test is considered a “performance” or “behavioral” based “corroborator” of the self-report tests (like “Need for Cognition,” which involves agreement or disagreement with statements like “I usually end up deliberating about issues even when they do not affect me personally” and “thinking is not my idea of fun”) that are the basis of the neo-authoritarian-personality literature on which “asymmetry thesis” rests.
It has also been featured in numerous studies that show that religiosity, which is indeed negatively correlated with cognitive reflection, predicts greater resistance to engaging evidence that challenges pre-existing beliefs.
Accordingly, one might have expected, if the “asymmetry thesis” is correct, that Cognitive Reflection Test scores would be negatively correlated with conservativism. Studies based on nonrepresentative samples—ones consisting of M Turk workers or of individuals who visited a web site dedicated to disseminating research findings on moral reasoning style—have reported such a finding.
But in my large, nationally representative sample, scores on the Cognitive Reflection Test were not meaningfully correlated with political outlooks.
Actually, there was a very small positive correlation between cognitive reflection and identification with the Republican Party. But it was too tiny to be of any consequence for anything as consequentially large as the conflict over climate change.
Moreover, there was essentially zero correlation between cognitive reflection and a more reliable, composite measure of ideology and political party membership.
Because I think the only valid way to test for motivated reasoning is to do do experimental tests that feature that phenomenon, I don’t really care that much about correlations between cognitive style measures and ideology.
But if I were someone who did think that such correlations were important, I’d likely find it pretty interesting that conservativism doesn’t correlate with Cognitive Reflection Test scores. Because this test is now widely regarded as a better measure of the disposition to engage in critical reasoning than are the variety of self-report measures on which the “asymmetry thesis” thesis literature rests—and, as I said, has been featured prominently in recent studies of the cognitive reasoning style associated with religiosity—the lack of any correlation between it and conservative political outlooks raises some significant questions about exactly what the correlations reported in that literature were truly measuring.
For this reason, I anticipate that “asymmetry thesis” supporters will focus their attention on this particular finding in the study. Yet it’s actually not the finding that is most damaging to the “asymmetry thesis”; the experimental finding of symmetry in motivated reasoning is! Indeed, I obviously don’t think the Cognitive Reflection Test—or any other measure of effortful, conscious information processing for that matter—is a valid test of open-mindedness (which isn’t to say there might not be one; I’d love to find it!). But it has been amusing—a kind of illustration of the experiment result itself—to see “asymmetry thesis” proponents, in various responses to the working paper version of the study, attack the the Cognitive Reflection Test as “invalid” as a measure of the sort of “closed mindedness” that their position rests on!
One final note:
The study characterizes differences in individuals’ predispositions with a measure of their right-left political leanings rather than their cultural worldviews. I’ve explained before that “liberal-conservative ideology” and “cultural worldviews” can be viewed as alternative observable “indicators” of the same latent motivating disposition. I think cultural worldviews are better, but I used political outlooks here in order to maximize engagement with those researchers who study motivatated reasoning in political psychology, including those who are interested in the “asymmetry thesis,” the probing of which was, as indicated, a secondary but still important objective of the study. I have also analyzed the study data using cultural worldviews as the predisposition measure and reported the results in a separate blog post.