Misinformation that originates on social media sometimes transitions to much larger audiences when it is picked up by traditional media outlets.
By Dr. Gordon Pennycook
Adjunct Professor of Psychology
University of Regina
By Dr. David G. Rand
Erwin H. Schell Professor
Professor of Management Science and Brain and Cognitive Sciences
Massachusetts Institute of Technology (MIT)
Highlights
- Recent evidence contradicts the common narrative that partisanship and politically motivated reasoning explain why people fall for ‘fake news’.
- Poor truth discernment is linked to a lack of careful reasoning and relevant knowledge, as well as to the use of familiarity and source heuristics.
- There is also a large disconnect between what people believe and what they will share on social media, and this is largely driven by inattention rather than by purposeful sharing of misinformation.
- Effective interventions can nudge social media users to think about accuracy, and can leverage crowdsourced veracity ratings to improve social media ranking algorithms.
We synthesize a burgeoning literature investigating why people believe and share false or highly misleading news online. Contrary to a common narrative whereby politics drives susceptibility to fake news, people are ‘better’ at discerning truth from falsehood (despite greater overall belief) when evaluating politically concordant news. Instead, poor truth discernment is associated with lack of careful reasoning and relevant knowledge, and the use of heuristics such as familiarity. Furthermore, there is a substantial disconnect between what people believe and what they share on social media. This dissociation is largely driven by inattention, more so than by purposeful sharing of misinformation. Thus, interventions can successfully nudge social media users to focus more on accuracy. Crowdsourced veracity ratings can also be leveraged to improve social media ranking algorithms.
Toward a Psychology of False and Misleading Online News
Overview
Fabricated news is nothing new. For example, in 1835 The Sun newspaper in New York published six articles about purported life on the moon which came to be known as the ‘Great Moon Hoax’. During the 2016 US Presidential Election and UK Brexit Referendum, however, a different form of fake news (see Glossary) rose to prominence: false or highly misleading political ‘news’ stories, primarily originating on social media [1.]. Concern about fake news was redoubled in 2020 in the face of widespread misinformation and disinformation [2.] on social media about the coronavirus disease 2019 (COVID-19) pandemic [3.] and the 2020 US Presidential Election [4.]. Misleading hyperpartisan news, as well as yellow journalism [5.], are related forms of problematic news content that are likely sources of political polarization [6.]. What is it about human psychology – and its interaction with social media [7.,8.] – that explains the failure to distinguish between accurate and inaccurate content online? Apart from being of theoretical interest, this question has practical consequences: developing effective interventions against misinformation depends on understanding the underlying psychology.
Prevalence of Fake News
Various analyses of social media and web browsing data have been used in an attempt to determine the prevalence of fake news, often with a focus on the 2016 US Presidential Election. For example, using web browsing data, archives of fact-checking websites, and a survey, Allcott and Gentzkow [19.] estimated that a particular set of news stories that are known to be false were shared on Facebook at least 38 million times in the 3 months leading up to the 2016 election (30 million of which were for news favoring Donald Trump). This estimate represents a lower bound since it only reflects that specific set of known false news.
Other analyses have focused on fake news publishers (i.e., websites) rather than on individual articles. Based on data from Twitter [117.], Facebook [77.,118.], and web browsing [89.], these studies concluded that content from known fake news sites represents a small proportion of most people’s media diets, and that the average social media user was exposed to little fake news during the 2016 election.
These analyses have important limitations, however, because the only available data concern what people are sharing and what they visit when they click through to visit news sites off-platform. But, of course, the vast majority of the time that people are exposed to news on social media, they simply read the post without sharing it or clicking on the link to visit the actual source website. Furthermore, so-called ‘fake news’ only represents one category of misinformation, and misleading content from sources such as hyperpartisan news websites likely represents a much larger proportion of people’s media diets [6.,119.]. Thus, the actual on-platform exposure of the average user to misinformation remains an open question [120.]. We feel it is premature to conclude that exposure rates are minimal, and thus that false and misleading news online is not a problem (also [7.,8.]). This is especially true when looking beyond the 2016 election because new misinformation threats – such as false claims about COVID-19 [3.,44.] and fraud in the 2020 US Presidential Election [4.] – have gained widespread traction through amplification by (mostly Republican) political elites.
Accordingly, exposure to fake news (and misinformation more broadly) is not equally distributed across all users. In particular, political conservatives and older adults were far more likely to visit fake news websites or share fake news articles during the 2016 Presidential Election [19.,89.,117.,118.]. Studies have also found associations between political conservatism and belief in misinformation in the USA [20.,44.], Chile [121.], and Germany [122.], but not in Hungary [24.], and users who engage in less reasoning have been found to share content from lower-quality news sites on Twitter [71.]. Thus, even if it was true that the average social media user was not exposed to that much misinformation, exposures rates are substantially higher in subpopulations that may be particularly vulnerable to believing inaccurate content. Finally, misinformation that originates on social media sometimes transitions to much larger audiences when it is picked up by traditional media outlets – either via direct repetition or debunking (which may result in inadvertent amplification).
We focus here primarily on online content that is presented in the form of news articles. However, false and misleading claims come in many forms, and there are several literatures that are clearly related, but outside the scope of our review (although we will draw some connections throughout). These include work on conspiracy belief [9.], superstition [10.], rumors [11.], bullshit receptivity [12.], and misperceptions[13.], among others. Furthermore, our focus is on individual examples of misinformation, and not on organized disinformation campaigns (e.g., by the Russian Internet Research Agency, or campaigns relating to global warming or fraud in the 2020 US Presidential Election).
Why Do People Fall for Fake News?
Overview
When considering the factors that may influence what people believe, it is essential to distinguish between two fundamentally different ways to conceptualize belief in true and false news. One common approach is to focus on truth ‘discernment’, or the extent to which misinformation is believed ‘relative’ to accurate content. Discernment, typically calculated as belief in true news minus belief in false news (akin to ‘sensitivity’ or d’ in signal detection theory [14.]) captures the ‘overall’ accuracy of one’s beliefs – and thus gives insight into failures to distinguish between true and false content (‘falling for fake news’).
Another approach is to focus on overall belief, or the extent to which news – regardless of its accuracy – is believed (calculated as the average or sum of belief in true news and belief in false news, akin to calculating ‘bias’ in signal detection theory [14.]). Critically, factors that alter overall belief need not impact people’s ability to tell truth from falsehood [15.]: increasing or decreasing belief in true and false headlines to an equivalent extent has no effect on the overall accuracy of one’s beliefs (i.e., does not affect truth discernment).
Political Motivations
A popular narrative is that the failure to discern between true and false news is rooted in political motivations. For example, it has been argued that people are motivated consumers of (mis)information [16.] – that they engage in ‘identity-protective cognition’ when faced with politically valenced content, and this leads them to be overly believing of content that is consistent with their partisan identity and overly skeptical of content that is inconsistent with their partisan identity [17.]. A related theory argues that people place loyalty to their political identities above the truth – and thus fail to discern truth from falsehood in favor of simply believing ideologically concordant information [18.]. These accounts contend that a strong causal influence of political motivation on belief is thus the dominant factor explaining why people fall for fake news.
It is clearly true that partisanship is associated with overall belief: People are more likely to believe news content that is concordant (versus discordant) with their political partisanship [19., 20., 21., 22., 23., 24., 25.] (Figure 1B ). It is important to note, however, that the effect of political concordance is typically much smaller than that of the actual veracity of the news [20.,21.,26.]. In other words, true but politically discordant news is typically believed much more than false but politically concordant news – politics does not trump truth. Furthermore, greater overall belief in politically consistent news does not necessarily indicate politically motivated reasoning. Such differences could even arise from unbiased rational (e.g., Bayesian) inference built on prior factual beliefs that differ across party lines (e.g., owing to exposure to different information environments) [27., 28., 29., 30., 31., 32., 33.]
Challenges in Identifying Politically Motivated Reasoning
The observation that people are more likely to believe information that is consistent with their political ideology/partisanship (and are less likely to believe information that is inconsistent with their ideology/partisanship) is often taken as evidence for politically motivated reasoning [22.,123.,124.]. Critically, however, this pattern does not actually provide clear evidence of politically motivated reasoning because partisan identity is likely confounded with other relevant variables [27.,125.]. Most notably, partisans differ in what they believe about the world, even when it comes to so-called ‘factual beliefs’ – beliefs that relate to facts or empirical evidence [28.], such as in the case of global warming [126.]. This is critical because a large body of evidence from entirely nonpolitical contexts shows that what people believe to be true about the world influences their reasoning (a phenomenon known as belief bias [127.]). Indeed, once prior beliefs are accounted for, the apparent impact of political concordance on processes of belief formation is typically greatly reduced or eliminated [28.,29.,128.]. Thus, observing a difference across ideological lines is not sufficient evidence to conclude that partisan identity or political motivations are themselves responsible for the difference [27.,125.].
To clearly disentangle the impact of partisan identity or motivations versus prior beliefs, further experimental work is needed – for example, studies that manipulate prior factual beliefs and/or political motivations [27.,30.]. Relatedly, understanding the origins of partisan differences in prior factual beliefs is also of the utmost importance. Exposure to different information streams is a promising candidate: entirely rational (e.g., Bayesian) and truth-seeking (i.e., nonpolitically motivated) people who obtain their information from conservative (e.g., Fox News) versus liberal (e.g., MSNBC) news sources would naturally wind up with very different factual beliefs about the world. Evaluating new information in light of how well it aligns with one’s priors – although often referred to as ‘confirmation bias’ – is not, in fact, necessarily evidence of bias in a normative sense. When there is uncertainty about the reliability of information sources or data-generating processes, it can be entirely consistent with Bayesian inference to be skeptical of information that is inconsistent with one’s prior factual beliefs [30.,129.]. In such cases, Bayesian agents may infer that the source or the data-generating process is unreliable rather than that their prior belief was incorrect – and doing so would not be evidence of bias per se (where bias is defined as deviating from some normative, e.g., Bayesian, benchmark) [130.].
Finally, the critique described in this box also applies to inferring ‘motivated System 2 reasoning’ from the observation that cognitive sophistication is sometimes (but not always or even frequently [131., 132., 133.]) associated with polarization rather than accuracy [17.]. The association between cognitive sophistication and polarization (e.g., in the context of climate change) disappears entirely once prior factual beliefs are accounted for [28.]. Instead, it appears that more cognitively sophisticated individuals may place more weight on their prior factual beliefs when evaluating new evidence, instead of placing more weight on concordance with their political identities. Of course, people’s prior factual beliefs may themselves be caused by politically motivated reasoning – but they need not be, and most study designs cannot determine (but must assume) a causal connection.
We now turn to the impact of political concordance on truth discernment. Greater overall belief in politically concordant news might seem to suggest that people are more inaccurate when assessing politically concordant news – in other words, that political concordance (and the associated motivations) interferes with truth discernment. In fact, the data reveal the opposite pattern: People are somewhat better at discerning truth from falsehood when judging politically concordant news compared with politically discordant news (Figure 1A). Taken together, the evidence therefore suggests that political identity and politically motivated reasoning are not the primary factors driving the inability to tell truth from falsehood in online news.
Reasoning
Overview
Another perspective on the (in)ability to differentiate between truth and falsehood comes from the field of reasoning. Work in this vein has a particular focus on dual-process theories stipulating that analytic thinking can override automatic, intuitive responses. The key question this perspective asks is – what is the role of reflective reasoning in the ability to discern fake news from truth?
Dual-Process Models of Reasoning and the Consequences of Deliberation
Dual-process theories are a core component of research on the cognitive science of reasoning. These theories argue that human cognition can be partitioned into two fundamentally different types of processes that differ in terms of their characteristics [40.,134.,135.]: Type 1 (or System 1) processing that is characterized primarily by automaticity such that Type 1 outputs (‘intuitions’) come to mind directly as a response to the stimulus, and Type 2 (or System 2) processing that is characterized by the deliberation that may or may not arise given a particular intuitive output (or set of outputs).
Consider the following problem from the Cognitive Reflection Test [136.]: ‘A bat and ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?’ The incorrect intuitive answer – 10 cents – comes to mind intuitively for most people, whereas the correct answer – 5 cents – emerges (for most) only with an additional deliberation process [137.]. Performance on tasks such as the Cognitive Reflection Test are associated with a wide range of beliefs and behaviors [138.], including actual social media behaviors observed on Twitter ([71.]; also [109.]), as well as conspiracy ideation [139.], politics [140.], bullshit receptivity [12.], and endorsement of a variety of epistemically suspect beliefs [138.]. Importantly, although dual-process theories typically emphasize the importance of overriding incorrect intuitions via analytic thinking, this should not be taken to imply that intuitions are always incorrect or that analytic thinking is always accurate [135.].
One potential answer, which follows from the earlier-referenced work on political identity, argues that deliberative (‘System 2’) reasoning is often motivated by political identity, and that people engage in ‘identity protective cognition’ [16.]. This account predicts that engaging in more deliberation should lead to more politically polarized beliefs – and, most importantly, to greater belief in politically concordant but false claims. Thus, more deliberation should be associated with worse truth discernment.
By contrast, more ‘classical’ reasoning accounts (i.e., accounts that are more consistent with work on dual-process reasoning in other domains) portray System 2 reasoning as being responsible for correcting faulty intuitions. This perspective therefore predicts that people who deliberate more will simply be less likely to believe false content – and are better able to discern between true and false content – regardless of the political concordance of the news that they are evaluating.
Across numerous recent studies, the evidence supports the classical reasoning account over the motivated System 2 reasoning account. People who are more reflective are less likely to believe false news content – and are better at discerning between truth and falsehood – regardless of whether the news is consistent or inconsistent with their partisanship [20.,26.,34.,35.] (Figure 1A). The same pattern is evident with respect to discernment between biased and misleading hyperpartisan news and true (mainstream) news [35.] and when judging full news stories as opposed to just headlines [36.], and using measures beyond the Cognitive Reflection Test (Figure 1), such as thinking disposition questionnaires [34.] and the Berlin Numeracy Test [35.]. Belief in fake news is also associated with delusionality [34.], dogmatism [34.], religious fundamentalism [34.], bullshit receptivity [37.], and overclaiming [37.] (all factors associated with analytic thinking; Box 3). Furthermore, experimentally manipulating participants’ level of deliberation demonstrates a causal effect whereby deliberation reduces belief in false (but not true) news, regardless of partisan alignment (and has no effect on polarization) [26.]. Research also shows that overconfidence may contribute to susceptibility to false information [38.], perhaps because it stops people from slowing down and engaging in reflective reasoning [39.,40.].
How, then, do people determine news veracity? The correlation between cognitive reflection and disbelief in fake news is stronger in cases where the content is more obviously implausible (and vice versa for true news) [20.]. This suggests that, in cases where people actually do stop and think, relevant prior knowledge is likely to be a critical factor. Indeed, political knowledge is positively associated truth discernment for political news content [23.,41.], as is media literacy [42.] and general information literacy [43.]. Similarly, basic science knowledge is positively associated with truth discernment for (mis)information about COVID-19 [44.]. This implies, unfortunately, that reasoning may not improve accuracy in contexts where prior knowledge is heavily distorted (e.g., by partisan media consumption or misinformation campaigns by political elites – climate change being a prime example) [28.]
Thus, when it comes to the role of reasoning, it seems that people fail to discern truth from falsehood because they do not stop to reflect sufficiently on their prior knowledge (or have insufficient or inaccurate prior knowledge) – and not because their reasoning abilities are hijacked by political motivations.
Heuristics
Prior work in judgment and decision making [45.] indicates that people are likely to use heuristics or mental shortcuts when judging news headlines. What, then, are the specific features of fake news that influence people’s intuitions or cause them to make mistakes when reasoning?
One key route to intuitive belief in news is familiarity [37.]. The influence of prior exposure on judgments of truth – sometimes referred to as the illusory truth effect – is well documented [46.,47.]. Indeed, wartime rumors during WWII that were more familiar were more likely to be believed [48.]. Consistent with this, a single prior exposure to a fake news headline increases later belief in the headline [49.,50.]. Remarkably, this is evident even if the headline is extremely implausible (also [51.]) and inconsistent with one’s political partisanship [49.]. Thus, feelings of familiarity and, possibly, processing fluency per se ([52.,53.]; (but see [54.]) likely contributes to increased belief in false claims.
The source is another important cue that may be used when evaluating news. Participants are more likely to believe information provided by people whom they view as being credible (reviewed in [55.]), and a large literature from political science has robustly demonstrated the impact of elite messaging, in particular, on public opinion [56.]. For example, attributing a false claim to President Trump increased Trump supporters’ belief in the claim while reducing Democrats’ belief in the claim [57.]. Furthermore, social feedback provided by social media platforms (e.g., ‘likes’) also increases belief in news content, particularly for misinformation [58.] – a factor that is likely to covary with elite messaging (given that political elites, such as Presidents and people in Congress or the media, often have many social media followers).
Finally, a salient feature of fake news headlines also seems to be that they are often emotionally evocative. That is, fake news is often geared toward provoking shock, fear, anger [8.,59.], or (more broadly) moral outrage [60.]. This is important because people who report experiencing more emotion (positive or negative) at the outset of the task are more likely to believe false (but not true) news; and instructing people to rely on emotion increases belief in false (but not true) headlines [61.].
Believing versus Sharing Fake News
One might expect that people share news on social media because they believe it is true. Accordingly, the widespread sharing of false content is often taken as evidence of widespread false beliefs [62.,63.]. However, recent work has shown that social media sharing judgments can actually be quite divergent from judgments about accuracy [21.,44.]. For example, participants who were asked about the accuracy of a set of headlines rated true headlines as much more accurate than false headlines; but, when asked whether they would share the headlines, veracity had little impact on sharing intentions – both in the context of political headlines [21.] (Figure 2A ) and headlines about COVID-19 [44.]. As a result, sharing intentions for false headlines were much higher than assessments of their truth (e.g., 91% higher in Figure 2A), indicating that many people were apparently willing to share content that they could have identified as being inaccurate.
To shed light on this disconnect between accuracy judgments and sharing intentions, a recent study examined the impact of asking participants to rate the perceived accuracy of each headline immediately before deciding whether they would be willing to share it on social media [21.]. This experiment helps to distinguish between three distinct explanations for the accuracy–sharing dissociation. The confusion-based account posits that people genuinely (but mistakenly) believe that the false claims they share are probably true. Consistent with this proposal, of the false headlines that were shared in the baseline condition of [21.], 33% were both believed and shared when participants were asked directly about accuracy – however, this leaves the remaining 67% of sharing unexplained by confusion.
The preference-based account is rooted in the idea that people place their preference for political identity (or related motives such as virtue signaling [64.]) above the truth, and thus share politically consistent false content on social media despite recognizing that it is probably not true. This purposeful sharing could be motivated, for example, by an effort to further one’s political agenda [18.], to sow chaos [65.], or to share news that would be interesting if it turned out to be true [66.]. Of the false headlines that were shared in the baseline condition of [21.], 16% of the headlines were shared despite being identified as inaccurate. Thus, although purposeful sharing occurs, it seems unlikely to explain the bulk of false or misleading content that is shared online.
Finally – and most consistent with the earlier focus on a lack of reflective thinking being a source of misjudgments – the inattention-based account argues that people have a strong preference to only share accurate content, but that the social media context distracts them from this preference. Consistent with this account, asking participants to rate the accuracy of each headline before deciding whether to share it decreased sharing of false headlines by 51% relative to the baseline condition [21.] – suggesting that inattention to accuracy was responsible for roughly half of the misinformation sharing in the experiment.
In a similar vein, work on social media behavior often emphasizes the importance of the ‘attention economy’ where factors relating to engagement (likes, shares, comments, clicks, etc.) are selected for in social media environments [8.,60.,67., 68., 69.]. Accordingly, sharing of low-quality news content on Facebook is associated with ideological extremity [70.] and ideological concordance is a much stronger predictor of sharing than it is of belief [21.]. Furthermore, analytic thinking is not only associated with more truth discernment, as described earlier, but is also associated with having more discerning sharing intentions in survey experiments (both in terms of false [35.,44.] and hyperpartisan content [35.]) and with the actual sharing of more reliable news outlets on Twitter [71.]. One striking possibility is that the social media context itself distracts people from prioritizing the truth when they decide what to share – and, in some cases, may actively promote antisocial behavior [72.] and hamper the exercise of analytic and critical thinking. Social media may be both a cause and a consequence of increased political engagement, both good and bad [73.].
What Can Be Done? Interventions To Fight Fake News
Overview
We now turn to the implications of these findings for interventions intended to decrease the spread and impact of online misinformation.
Current Approaches for Fighting Misinformation
As social media companies are, first and foremost, technology companies, a common approach is the automated detection of problematic news via machine learning, natural language processing, and network analysis [74., 75., 76.]. Content classified as problematic is then down-ranked by the ranking algorithm such that users are less likely to see it. However, creating an effective misinformation classifier faces two fundamental challenges. First, truth is not a black-and-white, clearly defined property: even professional fact-checkers often disagree on how exactly to classify content [77.,78.]. Thus, it is difficult to decide what content and features should be included in training sets, and artificial intelligence approaches run the risk of false positives and, therefore, of unjustified censorship [79.]. Second, there is the problem of nonstationarity: misinformation content tends to evolve rapidly, and therefore the features which are effective at identifying misinformation today may not be effective tomorrow. Consider, for example, the rise of COVID-19 misinformation in 2020 – classifiers trained to detect largely political content were likely unequipped to be effective for novel false and misleading claims relating to health.
Another commonly used approach involves attaching warnings to content that professional fact-checkers have found to be false (reviewed in [80.,81.]). A great deal of evidence indicates that corrections and warnings do successfully reduce misperceptions [41.,81., 82., 83.] and sharing [49.,84.,85.]. Despite some early evidence that correction checking could backfire and increase belief in false content [86.], recent work has shown that these backfire effects are extremely uncommon and are not a cause for serious concern [87.,88.].
There are, however, other reasons to be cautious about the sufficiency of professional fact-checking. Most importantly, fact-checking is simply not scalable – it typically requires substantial time and effort to investigate whether a particular claim is false or misleading. Thus, many (if not most) false claims never get fact-checked. Even for those claims that do eventually get flagged, the process is often slow, such that warnings are likely to be absent during the claim’s period of peak viral spreading. Furthermore, warnings are typically only attached to blatantly false news, and not to extremely misleading or biased coverage of events that actually occurred. In addition to straightforwardly undermining the reach of fact-checks, this sparse application of warnings could lead to an ‘implied truth’ effect where users may assume that (false or misleading) headlines without warnings have actually been verified [84.]. Fact-checks often also fail to reach their intended audience [89.], and may fade over time [90.], provide incomplete protection against familiarity effects [49.], and cause corrected users to subsequently share more low-quality and partisan content [91.].
Another potential approach that is commonly referenced is emphasizing the publishers of news articles, seeking to leverage the reliance on source cues described earlier. This, in theory, could be effective because people (at least in the USA) are actually fairly good at distinguishing between low- and high-quality publishers [92.]. However, experimental evidence on emphasizing news publishers is not very encouraging: Numerous studies find that making source information more salient (or removing it entirely) has little impact on whether people judge headlines to be accurate or inaccurate [37.,93., 94., 95., 96., 97.] (although see [98.,99.]).
New Approaches for Fighting Misinformation
One potentially promising alternative class of interventions involve a more proactive ‘inoculation’ or ‘prebunking’ against misinformation [8.,100.]. For example, the ‘Bad News Game’ uses a 10–20 minute interactive tutorial to teach people how to identify fake news in an engaging way [101.]. An important limitation of such approaches is that they are ‘opt in’ – that is, people have to actively choose to engage with the inoculation technique (often for a fairly substantial amount of time – at least in terms of the internet attention span [102.]). This is particularly problematic given that those most in need of ‘inoculation’ against misinformation (e.g., people who are low on cognitive reflection) may be the least likely to seek out and participate in lengthy inoculations. Lighter-touch forms of inoculation that simply present people with information that helps them to identify misinformation (e.g., in the context of climate change [103.]) may be more scalable. For example, presenting a simple list of 12 digital media literacy tips improved people’s capacity to discern between true and false news in the USA and India [104.].
Both fact-checking and inoculation approaches are fundamentally directed toward improving people’s underlying knowledge or skills. However, as noted earlier, recent evidence indicates that misinformation may spread on social media not only because people are confused or lack the competency to recognize fake news, but also (or even mostly) because people fail to consider accuracy at all when they make choices about what to share online [21.,44.]. In addition, as mentioned, people who are more intuitive tend to be worse at distinguishing between true and false news content, both in terms of belief (Figure 1A) and sharing [35.,71.]. This work suggests that interventions aimed at getting people to slow down and reflect about the accuracy of what they see on social media may be effective in slowing the spread of misinformation.
Indeed, recent research shows that a simple accuracy prompt – specifically, having participants rate the accuracy of a single politically neutral headline (ostensibly as part of a pretest) before making judgments about social media sharing – improves the extent to which people discern between true and false news content when deciding what to share online in survey experiments [21.,44.]. This approach has also been successfully deployed in a large-scale field experiment on Twitter, in which messages asking users to rate the accuracy of a politically neutral news headline were sent to thousands of accounts who recently shared links to misinformation sites [21.]. This subtle prompt significantly increased the quality of the new they subsequently shared (Figure 2B). Furthermore, survey experiments have shown that asking participants to explain how they know whether a headline is true or false before sharing it increases sharing discernment [105.], and having participants rate accuracy at the time of encoding protects against familiarity effects [106.]. Relatedly, metacognitive prompts – probing questions that make people reflect – increases resistance to inaccurate information [107.].
A major advantage of such accuracy prompts is that they are readily scalable. There are many ways that social media companies, or other interested parties such as governments or civil society organizations, could shift people’s attention to accuracy (e.g., through ads, by asking about the accuracy of content that is shared, or via public service announcements, etc.). In addition to scalability, accuracy prompts also have the normative advantage of not relying on a centralized arbiter to determine truth versus falsehood. Instead, they leverage users’ own (often latent) ability to make such determinations themselves, preserving user autonomy. Naturally, this will not be effective for everyone all of the time, but it could have a positive effective in the aggregate as one of the various tools used to combat misinformation.
Finally, platforms could also harness the power of human reasoning and the ‘wisdom of crowds’ to improve the performance of machine-learning approaches. While professional fact-checking is not easily scalable, it is much more tractable for platforms to have large numbers of non-experts rate news content. Despite potential concerns about political bias or lack of knowledge, recent work has found high agreement between layperson crowds and fact-checkers when evaluating the trustworthiness of news publishers: the average Democrat, Republican, and fact-checker all gave fake news and hyperpartisan sites very low trust ratings [92.] (Figure 3A ). This remained true even when layperson raters were told that their responses would influence social media ranking algorithms, creating an incentive to ‘game the system’ [108.]. However, these studies also revealed a weakness of publisher-based crowd ratings: familiarity with a publisher was necessary (although not sufficient) for trust, meaning that new or niche publishers are unfairly punished by such a rating scheme. One solution to this problem is to have laypeople rate the accuracy of individual articles or headlines (rather than publishers), and to then aggregate these item-level ratings to create average scores for each publisher (Figure 3B). Furthermore, the layperson ratings of the articles themselves are also useful. Analyzing a set of headlines flagged for fact-checking by an internal Facebook algorithm found that the average layperson accuracy rating for fairly small crowds correlated equally well with that of professional fact-checkers as the fact-checkers correlated with each other [77.]. Thus, using crowdsourcing to add a ‘human in the loop’ element to misinformation detection algorithms is promising.
These observations about the utility of layperson ratings have a strong synergy with the aforementioned idea of prompts that shift users’ attention to accuracy: periodically asking social media users to rate the accuracy of random headlines both (i) shifts attention to accuracy and thus induces the users to be more discerning in their subsequent sharing, and (ii) generates useful ratings to help inform ranking algorithms.
Concluding Remarks
The spread of misinformation online presents both a scientific puzzle and a practical challenge. The research we have synthesized here shows that the common narrative, whereby failing to differentiate false or misleading news from truth is a symptom of political polarization in a ‘post-truth’ world, is not an appropriate characterization. Although people do preferentially believe news that aligns with their politics, this occurs as much or more for true headlines compared with false headlines – and thus people are actually more accurate, not less, when judging headlines that are politically concordant. Rather than being bamboozled by partisanship, people often fail to discern truth from fiction because they fail to stop and reflect about the accuracy of what they see on social media. Accordingly, simple prompts that shift people’s attention to accuracy increase the quality of news that people share on social media. Approaches of this nature, including providing digital literacy tips, are not hindered by the same issues of scalability related to strict fact-checking approaches – and, in fact, can be combined with crowd-sourced fact-checking to maximize efficiency. Human reasoning, when applied appropriately, can be a powerful salve against the lure of misinformation. Nonetheless, numerous important questions remain unanswered (see Outstanding Questions).
There is also much to be gained for psychology and cognitive science per se from investigating online misinformation. Fake news and other forms of online misinformation represent a powerful (and ecologically valid) testing ground for evaluating theories from cognitive, social, and political psychology. More so than many other domains, when studying fake news and misinformation it is often possible to combine traditional laboratory experiments with large-scale social media data [71.,109.] – or even to conduct actual field experiments on-platform [21.,110.,111.]. Furthermore, these topics also motivate theory development by highlighting a new class of problems in need of further explanation: only a small fraction of online content draws sufficient attention and interest to be shared on social media. Understanding the psychology that underpins the dynamics of social media sharing is an important emerging subfield in psychology [21.,67.,112.] (in addition to other areas such as computer science [76.], political science [113.], communication [114.], and public health [115.], among others [116.]), driven in large part by concerns about misinformation. Tackling these questions will give psychologists the opportunity to demonstrate the power and real-world impact of psychological theory.
Outstanding Questions
What are the impacts of exposure to misinformation? How does exposure affect belief in the specific claims made in the misinformation, more general attitudes (e.g., support for political candidates or policies), and relevant behaviors (e.g., vote choice or health behaviors)? Do these effects differ for misinformation versus reliable news? How much misinformation must be encountered to shift attitudes, beliefs, and behaviors?
Does the design of social media platforms actively promote the spread of misinformation? For example, by inducing distraction and incentivizing or directing attention to factors other than veracity?
How do times of crisis (e.g., the COVID-19 pandemic) affect susceptibility to misinformation and the dynamics of social media behavior?
To what extent do findings about misinformation and social media from the USA and other Western countries generalize to other cultures and other social contexts (e.g., messaging apps such as WhatsApp)?
What are the similarities and differences between the psychology of online misinformation relative to related constellations of beliefs, such as conspiracy ideation, political misperceptions, rumors, and bullshit? How does the psychology of online misinformation compare with misinformation distributed by traditional media such as television and talk radio, or by political elites?
How will findings about current forms of misinformation generalize in the face of the continually evolving nature of online content?
Will social media corporations be willing to invest in, and implement, major changes to fight misinformation, given that their business models are premised on maximizing engagement? How can advocates and policy makers most effectively incentivize them to do so?
How can academics and social media platforms collaborate to study misinformation, given the platforms near-total control of relevant data, and widespread concerns about privacy?
See endnotes and references at source.
Originally published by Trends in Cognitive Sciences 25:5 (May 2021, 388-402) under the terms of a Creative Commons Attribution 4.0 International license.