This post too long for email. Click on the header to open it in a browser and read the full thing

Out last week, A registered report megastudy on the persuasiveness of the most-cited climate messages. The study tested the effects of 10 well researched climate change messaging strategies on Americans’ pro-environmental attitudes and behaviour.

Science is a tapestry of evidence, where we shouldn’t put too much weight on a single study. This study, however, is worth paying some attention to, for the following reasons:

• It takes evidence from a lot of people. 13,544 to be precise, which is approximately 13,500 more than some studies of persuasion from the dark, unreliable days of psychology, pre-replication crisis.

• It has a large, diverse authorship. 25 authors by my count. Most importantly, these researchers come from different research groups, ensuring that the study design has input from people who don’t necessarily agree, and who have different favoured theories they want to support.

• I don’t know what exactly qualifies something to be a “megastudy”, but I don’t think many participants or many authors alone should count - the crucial thing is the attempt to provide something comprehensive. For this study, they first reviewed the literature (157 research papers on climate messaging), and from them selected the ten most cited messaging strategies for which there was evidence of effectiveness. By testing them in the same study they allowed, first, agreement on how each strategy should be implemented, and second, direct comparison of the effects. Here’s the description of the strategies from the paper (individual strategies bolded by me):

Six selected messaging strategies described supporting climate change mitigation as consistent with moral values and ideologies, such as care and considerateness, free market beliefs, belief in a just world, patriotism, purity and system preservation. Two messaging strategies emphasized that climate change is scientific consensus. The remaining two strategies described the costs for a socially distant group or gain associated with climate change mitigation.

• The study has a number of credibility indicators. Most importantly, and in the title, is that it is a registered report, a mechanism whereby the methods for a study are reviewed by the journal - which agrees to publish regardless of how the results turn out. This shifts the focus of journal review onto the importance of the research question, and the solidity of methods proposed for addressing it, rather than the results. You realise exactly how revolutionary the improvement this makes to the credibility of published results is when you appreciate the range of biases generated by the “review when you know the results” convention (basically positive existing results get published, negative results get file-drawered, and authors are incentivised to write up their studies as though they show a positive result, and what happened was what they predicted all along). So, basically, the registered report format meant the authors had every incentive to design a strong study, and reduced incentive to manipulate the analysis to present positive results, leaving us, the readers, with more faith that what they claim they show is representative1. They also share the anonymised data and analysis code, which is an important transparency measure and credibility signal.

So, worth spending some time with. What do they show?

1. There is bipartisan agreement on climate issues (in many ways)

This first thing isn’t emphasised by the authors of the study, but it is revealed by their results and worth comment. The study polled a sample of American adults, recruiting approximately as many Democrats and Republicans, so in this sense at least representative2

These plots show the volume of responses from participants of different political alignment. Here’s the one from belief in climate change. Although there is a famous partisan divide on this in the US, I’d argue that there is as much similarity as difference here. Most Republicans have more belief that climate change is happening than it is not (they are over 50% on the belief scale), the same as independents and Democrats (admittedly more Democrats express 100% certainty on this one).

Both Democrats and Republicans are similar in this intention to take “non-political” pro-environmental actions, such as “Eat less meat”, “fly less”, “compost”. Or, at least, they are similar in that many who identify as Democrat and many who identify as Republican have intentions to take pro-environmental action, and some from both groups have the intention not to.

The partisan groups really pull apart in how they interpret climate change and how they regard political choices in response to climate change. Republicans, for example, just seem a great deal less worried about climate change than Democrats:

You can check out the full results in the paper, but my takeaway was that - despite elite messaging questioning the reality of climate change - most of the US population doesn’t reject the reality of climate change, they just differ in what they think should be done about it.

Consequently, we shouldn’t assume that disagreements about climate policies are disagreements about the science of climate change (and if we do, that could be misdirection). There’s a similar story in the UK, where the reality of climate change and the need to do something about it has cross-partisan consensus.

2. Message targeting doesn’t matter

“Persuasiveness varied little across party lines”, report the study authors, “inconsistent with theories predicting heterogeneous effects for targeted messages.”

Put another way, messages were persuasive or not, it didn’t seem to matter who was seeing them, a Democrat or a Republican.

This chimes with what I reported political scientist David Broockman arguing in my newsletter from December, and what Alex Coppock argued in his book Persuasion in Parallel: How Information Changes Minds about Politics.

The idea of targeted messaging fulfils our hope for a magic bullet which will persuade those who think differently from us (and surely also plays well for the business model of campaign consultants), but the reality may be that the differences between different messages are larger (and easier to take advantage of) than the differences between how the same message lands with different demographics.

Although the study authors don’t rub this in, six of the ten messages they tested used a moral value reframing, inspired by the Moral Foundations school of thought. The lack of difference across Democrats and Republicans should specifically dampen our enthusiasm for how practical Moral Foundations is as a way of reaching people of different political persuasions. (That said, many good theories contain deep truths and are also entirely impractical, or impractical until applied right, but at the least this result shows that simplistic translation of Moral Foundations into persuasive messaging doesn’t allow special access to one partisan group or the other).

3. Persuasion is possible, just hard

The third thing I’d take from this study is that, consistent with a lot of the research I review in this newsletter, persuasion is possible. All groups of participants responded to some of the persuasion messages, but the response was typically small - a few percentage points.

Here’s just one of the outcomes they measured, attitude (on a scale of 0 to 100) towards climate mitigation policies (things like “Ban construction of new coal-fired power plants”).

The top strategy - a purity framing message (“Participants read a text arguing that mitigating climate change is consistent with values such as protecting the purity of American land.”) - looked like it had an average effect of moving people 1.5 points on the 100-point scale. In an election, that can be enough to matter, but in an election you also have the other side trying to push voters in the other direction. Does it matter in other contexts? Well you could say that it is impressive to get any movement in beliefs or attitudes from a short persuasive message. It proves people haven’t completely made up their minds on the issue.

That said, my native optimism about how reachable people are was given pause by what was perhaps the most interesting question the researchers asked: would participants be willing to give up a proportion of their payment for taking part as a donation to a pro-environmental organisation? No messaging strategy - not one - increased donations relative to the control condition (which was reading a non-persuasive message on a non-climate related topic, such as “the history of neckties”).

In fact, it looked like several messaging strategies decreased participants’ willingness to donate. I don’t read too much into this backfire effect. The donation measure wasn’t a planned part of the study. The most I’d take from it is the results are incompatible with any persuasion effect on expressed belief flowing through directly to behaviour (willingness to donate). Simply put, what people say is easier to shift than what they do (especially when cash is on the line).

In the words of the study authors, these results highlight “the limits of short-form messages for increasing Americans’ support for action to address climate change.”

Overall

We need more collaborative studies which test different approaches head to head. These particular results should shift our expectation down that it is possible to find bespoke messages for different political groups, without doing much to shift our prior expectation that, while always possible, persuasion is hard.

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for the references, more on persuasion, a recommendation for a great podcast and links to some recent things published elsewhere by me.

Reference & More

Voelkel, J.G., Ashokkumar, A., Abeles, A.T. et al. A registered report megastudy on the persuasiveness of the most-cited climate messages. Nat. Clim. Chang. (2026). https://doi.org/10.1038/s41558-025-02536-2

Related, by me

• Our intuitions about political advertising are poor. Notes on a great interview with David Broockman

• Language models are persuasive - and that’s a good thing. Two new studies provide insights into exactly how LLMs persuade, and what that means.

• The truth about digital propaganda. Reasonable People #55: Our piece in New Scientist bring evidence to worries about online manipulation

• How persuasive is AI-generated propaganda? Reasonable People #51: Bullet review of a new paper suggesting LLMs can create highly persuasive text and will supercharge covert propaganda campaigns.

Amy Mount: Political Heat

If climate policy and politics is your thing, please read Amy Mount’s reflection the last 12 months: 2025: When the UK’s climate consensus crumbled

The short version: emissions went down. Renewables and electrification went up. The climate narrative was muted. Wider politics got harder. Voters stayed volatile and parties were polarised. The general election in 2029 will be a defining moment for the UK’s climate progress. But I remain hopeful – because the public is onside. The question is, will it stay there?

And for deep dives on everything from the UK energy market to the Conservatives’ environmental legacy, Amy has you covered in the Political Heat podcast archives

Political Heat

• Podcast

• Substack

Full disclosure: Amy is my cousin, so I have some CoI

The Transmitter: The 1,000 neuron challenge

By me, in neuroscience magazine The Transmitter

A competition to design small, efficient neural models might provide new insight into real brains—and perhaps unite disparate modeling efforts.

Link: The 1,000 neuron challenge

WonkHE: A new model moves research in a more democratic direction – it could be faster and fairer too

By myself and Anna Butters. This is about research funding, which is an interesting and important decision problem. I’m proud of the way we wrote this to start very zoomed out on the big picture, then take the reader through to the specific mechanism - distributed peer review - about which we have new evidence on.

Here’s the start:

Research quality is the dark matter of the university sector. It is hard enough to assess research after it has been done, research funders must find some way to evaluate proposals for projects which don’t exist yet. The established model for this is external expert review, combined with a panel stage where proposals, and their reviews are discussed, and hard choices made.

UK researchers will be familiar with this via our own UKRI, and everyone who has had a funding application rejected will recognise that the reviews received may be partial or mis-directed. This speaks to the idiosyncrasy and variability in individual judgments of what makes a good project, and has downstream consequences for what ultimately gets funded.

Link: A new model moves research in a more democratic direction – it could be faster and fairer too

Preprint: Applicants as reviewers: A mixed methods evaluation of the risks, benefits, and potential of distributed peer review for grant funding allocations

Speaking of which, the preprint of our work on this has just been published. It’s a shorter (by 50%) version of the working paper, so not strictly new. Here’s the conclusion

This trial demonstrates feasibility of DPR [Distributed Peer Review] for an interdisciplinary funding programme. Implementing a DPR process requires funders to accept trade-offs, but can offer valuable benefits, including shortening time to funding decision. Building on established peer review principles, it has the potential to contribute to the democratisation of funding evaluation.

Citation:

Butters, A., Marshall, M. B., Pinfield, S., Stafford, T., Bondarenko, A., Neubauer, B., … Denecke, H. (2026, January 6). Applicants as reviewers: A mixed methods evaluation of the risks, benefits, and potential of distributed peer review for grant funding allocations. https://doi.org/10.31222/osf.io/t2p56_v1

…And finally

Tom Thomson. Northern River, 1914-15. (via @tarnport)

END

Comments? Feedback? Urban legends? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

‘The Volvo Story’ has a special place in the history of research on judgement and decision making. It’s a thought experiment. Imagine, it asks:

Let us suppose that you wish to buy a new car and have decided that on grounds of economy and longevity you want to purchase one of those solid, stalwart, middle class Swedish cars―either a Volvo or a Saab. As a prudent and sensible buyer, you go to Consumer Reports, which informs you that the consensus of their experts is that the Volvo is mechanically superior, and the consensus of the readership is that the Volvo has the better repair record. Armed with this information, you decide to go and strike a bargain with the Volvo dealer before the week is out. In the interim, however, you go to a cocktail party where you announce this intention to an acquaintance. He reacts with disbelief and alarm: “A Volvo! You’ve got to be kidding. My brother-in-law had a Volvo. First, that fancy fuel injection computer thing went out. 250 bucks. Next he started having trouble with the rear end. Had to replace it. Then the transmission and the clutch. Finally sold it in three years for junk.”

How likely is it that you would proceed as planned and buy a Volvo?

Most people share the intuition that they would rethink the Volvo purchase.

This is despite, as the authors put it, “the logical status of this information is that the N of several hundred Volvo-owning Consumer Reports readers has been increased by one and the mean frequency of repair record shifted up by an iota on three or four dimensions.”

In other words, this guy at a party is just one guy. Why do we weight his report more than the hundreds compiled in the Consumer Reports consensus?

For years I swallowed the standard line from the Heuristics and Biases literature, that this story highlighted something flawed in our thinking.

It’s been called The Anecdotal Fallacy (which is really just providing a name, something which ‘dulls the ache of incomprehension’ but isn’t really an explanation). It has been called an example of the Availability Heuristic, where vivid information dominates our judgements of risk (in this case, the personal testimony of the man at the party is more vivid than the hundreds anonymously compiled in the report. Another frequently invoked example of the Availability Heuristic is that people fear flying, with its extremely vivid, but rare, accidents, more than they fear being driven to the airport, which is actually the higher risk activity).

The Volvo story has also been used to illustrate Base Rate neglect, which is an example of a failure of Bayesian rationality. The thought here is that the Consumer Report provides a prior expectation of faults among Volvos, which should be integrated with the new information (one additional Volvo with many faults). If you do this integration mathematically your estimate of the rate of faults should move from “very small” to “very small + a small fraction” (i.e. still very small).

One cause of the success of the heuristics and biases programme, for which Daniel Kahneman got the Nobel Prize, and which gave us “Nudge”, behavioural economics and the rest, was that Kahneman and Tversky worked very hard on refining their experiments, which often relied on exactly the kind of vignettes represented by the Volvo Story, so that they precisely captured some quirk of our intuitions.

Fans of the literature will recognise this approach in the (increasingly unfortunately named) Asian Disease Problem, The (increasingly anachronistic) Linda Problem and others.

These vignettes bring into view two ‘obvious’ answers which are in contradiction. With the Volvo story, obviously - ‘logically’! - we should ignore the party guy. Also obviously, we wouldn’t.

This is the basic contradiction which requires an explanation. The authors of the chapter which introduced the story were aware that recognising the contradiction was only half of the issue: if, ‘logically’, the man should ignore the party-goer, only someone ignorant of human nature would expect that he would. What, they ask, explains why some information (like the Consumer Report) is ignored, and some (like the party-goer’s anecdote) is prioritised?

Now, many years after first hearing this thought experiment, I look at it differently. Rather than seeing the intuitive reaction (to abandon Volvo-purchase plans) as resulting from one of the many cognitive biases I was taught about, I see it as showing us something important about the nature of trust.

Trust, fundamentally, isn’t transmissible in the same way as information. We believe the people we trust, but it isn’t reasonable to expect others to believe them in turn.

Here’s a way of thinking about it: trust fades quickly with transmission. The half-life is short, so that if A tells B, and B tells C, and C tells D, the credibility that D can reasonably assign to what C has told them (that B told after hearing it from A) is close to nothing.

This is why it is reasonable to discount the Consumer Reports conclusion. A bunch of people (Who? How selected?) told Consumer Reports which collated this information (How? Under what biases?) and then published it. The chain of transmission contains too many points where biases could creep in. Maybe Volvo pay bribes to Consumer Reports? Maybe Volvo owners are too proud to report faults? The point isn’t that any bias is plausible, only that there are just too many unknown unknowns, and with each point in the transmission chain these multiply, rather than add.

The party-goer’s anecdote is vivid, but it also appears immediately verifiable. The party-guy is standing right there, ready to defend or explain his story. The mechanisms of credibility are tangible, so trust is greater than for the distant and anonymous evidence of Consumer Reports1.

It sounds obvious, but far too many important communications - from news, to health information about vaccines - ask people to take things on trust without seeming to recognise how fragile trust is, how poorly it travels across social distance.

One notable example: Consensus messaging has been promoted as the way to convince people about about climate change (e.g. “97% of scientists believe in human-caused climate change”). Science is a massive, anonymous, aggregator. A Consumer Reports for evidence on everything. Trusting science is reasonable if you have some understanding of how science works, but if you don’t then it is the equivalent to “97% of people different from me who do things in ways and for reasons I don’t understand believe something”. No wonder some of us prioritise vivid information from people we meet at parties instead.

When we recognise that my trust doesn’t convert 100% into your trust we can communicate more effectively - giving reasons to believe what we’ve come to believe, rather than hoping that the strength of our conviction will carry the day.

What of our hypothetical Volvo-purchaser? I’m arguing that it isn’t unreasonable to rethink his choice after the party, just as it would also be reasonable to press on and follow the Consumer Reports recommendation. Reasonableness in the real world often doesn’t have correct answers. Choices are usually matters of which risks you want to take, and what (or who) you put your trust in. We’ll understand choice better when we take trust seriously as a problem for each individual to solve for themselves, rather than something any of us can, or should, take as a given.

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for the references and more on persuasion by LLM, design thinking for chatbots and a quote from Mary Wollstonecraft.

References

The Volvo story originally (as far as I can tell) appears in:

• Richard E. Nisbett, Eugene Borgida, Rick Crandall & Harvey Reed, “Popular Induction: Information is Not Necessarily Informative”, in Judgment Under Uncertainty: Heuristics and Biases, Daniel Kahneman, Paul Slovic & Amos Tversky, editors, (1985), pp. 112-113.

There’s a side-note here about using LLMs for research. I had misremembered the source of the quote, so was trying to track it down. Gemini persistently hallucinated that it was in Kahneman’s Thinking Fast and Slow. Despite having a hardcopy, this was hard to definitely disprove in the time I was willing to spend looking through the book. An internet search also turned up many people who attributed it to Kahneman or Tversky, although I couldn’t immediately find a source where they discuss it (or attribute it).

Eventually, I got to Gary N. Curtis’ Fallacy Files which provided the reference (which I verified by picking up my own copy of Judgment Under Uncertainty):

• The Anecdotal Fallacy

Talking LLM blues

I’ve always been interested in how psychological science is laundered to support grander narratives of human failing and irrationality. Normally this happens via popularisation in the mass media, but you can also catch it happening in the scholarly literature.

Here’s Rogier et al:

“LLM Systems appear capable at exploiting cognitive biases that make humans vulnerable to persuasion and manipulation”

Following the claim there are citations. This is from the abstract of the first one

“Arguments that included factual knowledge, markers of trust, expressions of support, and conveyed status were deemed most effective”

(Breum et al, 2024).

Now the Rogier et al claim isn’t flat wrong - the Breum study did show an effect of status, and relying on arguer status is a plausible cognitive bias - but it is definitely selective to read this study and see evidence of manipulative power of LLMs.

If arguments that include factual knowledge are more persuasive that isn’t exploiting a cognitive bias, at least in the way that most people would understand it.

Here’s another study of those cited by Rogier et al to support the claim: Griffin et al (2023). One thing they report, which really jumps the shark in my opinion, is that simulating participants with an LLM shows that LLMs can be persuasive (because they ‘persuade’ simulated participants). To be fair that isn’t all the paper reports, but this part at least seems very minimal evidence for the claim that LLMs are exploiting cognitive biases to manipulate people (and the wider implication that we should be worried about their persuasive powers).

People need support if “do your own research” is going to help rather than hurt

A story from early 2024:

Nieman Labs “Asking people to “do the research” on fake news stories makes them seem more believable, not less”

Original paper, Aslett et al. 2023 Online searches to evaluate misinformation can increase its perceived veracity

Part of this must be what I called (in 2021) The Epistemic IKEA effect - we put more faith in beliefs we’ve ‘assembled’ ourselves. The study here also highlights the role of skill - those who are most inept at using search are most likely to find and trust low quality sources and confirmation of false beliefs. Here’s their abstract:

Considerable scholarly attention has been paid to understanding belief in online misinformation with a particular focus on social networks. However, the dominant role of search engines in the information environment remains underexplored, even though the use of online search to evaluate the veracity of information is a central component of media literacy interventions. Although conventional wisdom suggests that searching online when evaluating misinformation would reduce belief in it, there is little empirical evidence to evaluate this claim. Here, across five experiments, we present consistent evidence that online search to evaluate the truthfulness of false news articles actually increases the probability of believing them. To shed light on this relationship, we combine survey data with digital trace data collected using a custom browser extension. We find that the search effect is concentrated among individuals for whom search engines return lower-quality information. Our results indicate that those who search online to evaluate misinformation risk falling into data voids, or informational spaces in which there is corroborating evidence from low-quality sources. We also find consistent evidence that searching online to evaluate news increases belief in true news from low-quality sources, but inconsistent evidence that it increases belief in true news from mainstream sources. Our findings highlight the need for media literacy programmes to ground their recommendations in empirically tested strategies and for search engines to invest in solutions to the challenges identified here.

Design thinking about chatGPT

Amelia Watternberger makes the case that “Chatbots Are Not the Future”. It’s a fantastically presented argument, making the case that text inputs have no affordances and the responses don’t aggregate, so chat is the wrong interface for many tasks. Ironically, the affordances of chat are the reason I think chatbots are *right* for some of the tasks I’m interested in: deliberation, articulating your own position, understanding the position of others, and so on.

Amelia Watternberger: https://wattenberger.com/thoughts/boo-chatbots

Quote: Mary Wollstonecraft

Truth must be common to all, or it will be inefficacious with respect to its influence on general practice

A Vindication of the Rights of Woman (1792)

… And finally

From poorlydrawnlines.com by Reza Farazmand

END

Comments? Feedback? Urban legends? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

This is great: The Realities of Political Persuasion with David Broockman, from the always listenable Opinion Science by Andy Luttrell.

Broockman argues that, in the context of campaigning, too much emphasis has been put on message personalisation - identifying particular segments of voters and designing ads which particularly appeal to them - and not enough on empirically testing message effectiveness. You can understand why - for consultants and strategists personalisation seems like the kind of magic which might allow big breakthroughs, and there are a range of ways of categorising people from demographics or psychology which superficially seem promising.

Broockman’s research pours cold water on this idea of personalisation. This fits with what I’ve learnt - when I’ve looked at studies which purport to show that advertising according to personality is more successful, there are either major confounds, or - when you look closely - the personalisation may arguably have an effect, but the size of the effect is not significantly larger than a non-personalised message.

Important context for this is that there are big differences between individual adverts in terms of effectiveness. This means that better and worse adverts could be deployed in terms of persuasive effect - the variance is there, if we can work out how to take advantage of it (and if we did, it wouldn’t threaten our vision of human reasonableness, because microtargeting is not mind control).

The point is that personalisation is, in practice, very very far from being some kind of terrifying ‘manipulation machine’. The difference between different ads is typically far larger than between a personalised and non-personalised version of an ad. I try and show this by reworking the data from this paper into a summary visualisation:

To understand this plot you just need to know that the up-down/vertical axis shows the persuasive effect. The green line, as it changes left to right, shows the difference between individuals in how persuasive they found the ads - a huge difference. The change in the vertical position of the orange triangles shows the differences between different adverts used in the study - a smaller difference, but still much much bigger than the difference due to personalisation (blue dots). The full context is in this post: AI-juiced political microtargeting. The point is that a non-zero effect of personalisation (the highest blue dot is higher than the lowest blue dot) can still be non-important compared to the other sources of variation (the differences due to advert or individual are far larger).

This is from a paper which says in the abstract : “Recent technological advancements, involving generative AI and personality inference from consumed text, can potentially create a highly scalable “manipulation machine” that targets individuals based on their unique vulnerabilities without requiring human input.”

The “highly scalable” part is definitely true, but I question the degree to which this is manipulation, or to which it is worth worrying about given the size of the demonstrated effects.

From the other side of the coin, the implication is that political campaigners should worry far more about being persuasive for everyone than trying to take advantage of minuscule fine-tuning that might be possible for different groups.

In the Opinion Science pod, Broockman goes on explain that despite the large differences between different messages these differences are hard to predict. They aren’t predicted by the categories that political scientists typically use, such as whether the ads are positive or negative in tone.

Further, expert political practitioners are not better at predicting the most persuasive adverts compared to the general public. Not only were the experts not better than the public at predicting which campaigning ads would be most persuasive, they weren’t consistent with each other (which is important because it suggests there isn’t even some kind of shared conventional wisdom which, although wrong, is shared by the experts).

I have a memory of a study done of the 2016 Democratic campaign which shows that the ads predicted to be most persuasive by campaign staffers were least persuasive with the voters who most needed to be swayed. Sadly I can’t find this now, but I did find this very telling article from pre-election 2016: How the Clinton campaign is slaying social media. This gloating piece neatly illustrates the potential mechanism - highly political, young, cosmopolitan and progressive campaign staffers (and journalists) mistake what works for them as what works for the generally less political, less young, less cosmopolitan and less progressive swing voters.

My take-away is that our intuitions in this area are bad.

We’re bad at discounting our own reaction to an ad to work out how it will affect people unlike us (meaning that campaigners probably leave a lot of variation in persuasive effect on the table when deciding which ads to deploy).

We’re drawn to think that microtargeting and personalisation will unlock extra persuasive power (when the evidence is that extant models of personalisation are not strongly effective, and the degree to which they are is probably in line with how normal persuasion works - information containing facts and evidence is persuasive, just as we’d hope).

And we’re constantly tempted to think of persuasion in general, and advertising in particular, as a form of manipulation. This is extremely limiting. It implicitly denies legitimacy to electoral campaigning, which is a core part of democracy, and denigrates the voters who are persuaded.

Link: The Realities of Political Persuasion with David Broockman

Paper: Broockman and colleagues: Political practitioners poorly predict which messages persuade the public.

Related, from me:

• Language models are persuasive - and that’s a good thing

  Two new studies provide insights into exactly how LLMs persuade, and what that means.

• The truth about digital propaganda

  Reasonable People #55: Our piece in New Scientist bring evidence to worries about online manipulation

• AI-juiced political microtargeting: Reasonable People #53 looking carefully at the claims in one study which uses generative AI to customise ads to personality type

• Propaganda is dangerous, but not because it is persuasive

  Reasonable People #52: I pick at the claim that propaganda “doesn’t work”.

• How persuasive is AI-generated propaganda? Reasonable People #51: Bullet review of a new paper suggesting LLMs can create highly persuasive text and will supercharge covert propaganda campaigns.

• Microtargeting is not mind control

  Reasonable People #22 exaggerated beliefs about the effectiveness of microtargeted ads obscure real risks, and real opportunities to foster public trust in politics

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for more on fact-checking, AI, and a very cool job opportunities.

In case you missed it

I’m spending more time in the UK capital. Let me know if you want to get coffee :

London calling / Tools for Thought

PODCAST: Gordon Pennycook on Unthinkingness, Conspiracies, and What to Do About Them

From Sean Carroll’s Mindscape, an interview with Pennycook about his work, including on pseudo profound bullshit and using chatbots to debunk conspiracy theories. Something which came out, which I missed from the papers and seems really important, is that people like the experience of being debunked by the chatbot.

And people usually actually like it.

They’re not mad at the AI.

The AI gives them information I think is useful.

And evidence matters more than we thought it was.

Link: Gordon Pennycook on Unthinkingness, Conspiracies, and What to Do About Them

Share

JOB: Research Scientist at Wikimedia

We’re hiring a Research Scientist strongly committed to the principles of free knowledge, open source, privacy, and collaboration to join the Research team. As a Research Scientist, you will conduct applied research on the integrity of Wikipedia knowledge, its communities and their work, and the Wikipedia model.

Examples of recent research questions you may contribute to include::

How does the platform and its community navigate election times?

What is the role of Wikipedia in the landscape of online disinformation?

What guidance can we provide to researchers studying neutrality on Wikipedia?

Closes: January 15th, full remote with some geographic limitations

Link: https://job-boards.greenhouse.io/wikimedia/jobs/7484474/

….And finally

END

Comments? Feedback? Takes from 2016 which aged really badly? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

Two new studies were published last week, in two of the most prestigious scientific journals, and sharing an overlapping authorship list. Persuading voters using human–artificial intelligence dialogues (Lin et al, 2025) was published in Nature, and tested the effect of AI dialogues on voters’ preferences in the context of elections in the US, Canada and Poland. The levers of political persuasion with conversational artificial intelligence (Hackenburg et al, 2025) was published in Science, and tested the effect of AI dialogues, when driven by different models and different prompting strategies, on attitudes of UK adults across British political issues.

Both reports show that language models are persuasive. This means that they shift participants’ position on the issues they set out to persuade on, measured using the shift in before-after difference of how participants self-report their beliefs on a 100 point scale1. Not only do the models persuade, there’s some evidence they can be even more persuasive than a baseline of a static persuasive message.

There are sure to be some whom this worries. It raises the spectre of a lumpen population dragged around by artificially generated propaganda. It doesn’t worry me. These studies are good news. Not only should we expect language models to be persuasive, the details in the papers about exactly how they are persuasive are good news for human rationality, and good news for the future.

Both studies show that the crucial ingredient to persuasion is evidence and argument.

In their US election persuasion experiment, Lin et al used an automated analysis to detect which persuasion strategies were deployed by their models. Note that this was 2024 and the Trump-Harris presidential race. If we believe the common story, this was an acrimonious election campaign with supporters of both candidates firmly entrenched and hard to reach. Lin et al found their participants listened to the models, and shifted their attitudes. A small amount, but they shifted. Here’s a detail from their Figure 3, which shows the most frequently used strategies by the model:

Top persuasion strategies: politeness and civility, evidence and facts!

The Lin et al study prompted the models to be persuasive, giving detailed instructions in the prompt on how to make a respectful, argument-based, case (so perhaps less surprising that the model displayed these features). The other paper, Hackenburg et al, explicitly prompted the models to adopt different persuasion strategies, including various techniques suggested by recent decades of psychology research such as moral reframing or deep canvassing. The single most persuasive strategy from this set? “information-based argumentation (in which an emphasis is placed on providing facts and evidence)”

(The other persuasion strategies used, listed in this plot, are “Debate” - parliamentary style debate; “Mega” - prompting the model to use all strategies; “Norms” - focusing on demonstrating that others agree with the position; and “None” - just prompting the model to “be persuasive”).

Not shown here is that the authors also analysed persuasion across all the dialogues coming out of the models and found that, regardless of which explicit strategy the models were prompted with, the thing that predicted persuasive effect was the extent of the information content.

This is great! Language models are not a kind of magical persuasive sauce - there are not Words of Power which unlock the unsuspecting citizen’s mind. Instead, language models persuade using the same interface that any of us persuade - citing evidence and making arguments. What’s more, these results show that the whole grab-bag of persuasive tricks from psychology is less important than backing up relevant claims with facts. This includes personalising the message. The Hackenburg study tested the persuasive effect of individual personalisation and found it was small, although non-zero (this fits with other studies, which have found a zero or small effect of personalisation).

Zooming out, both studies show the power of the interactive mode for carrying persuasive information. Dialogue is more persuasive than broadcast. If language models present any kind of disruption to the dynamics of human persuasion it is not because they can do anything like a ‘superhuman’ levels of persuasion. Rather, it is because they can scale the standard mode of human persuasion, talking. Hitherto, the most persuasive technology we knew required a human on the other side of the conversation.

That said, the models tested in these studies weren’t even that persuasive. They didn’t unambiguously outperform a static message, and the measure of persuasion - self-report of position on a topic before and after - might not last that long, or translate that strongly into action. The results fit with a range of other studies which show that persuasion is generally hard, while acknowledging also that people tend to adjust their views to new evidence.

We can add to this that any persuasion by a language model is unlikely to happen in a vacuum - anyone persuaded in one direction is likely to experience persuasion in the other direction (especially in the context of an election campaign). The factors that led someone to a belief in the first place are likely to reassert themselves after an encounter with a model. Altogether I’m more and more reassured that language models aren’t going to disrupt the human information environment any more than it already is. In terms of disruption, the effects of social media - the takeover of short form, low context, monetised, acutely public information in the common media diet - are surely greater.

Extrapolating out, we can think about the possible downstream effects of persuasive models being widely deployed, but incorporating what these two recent studies show about how they would work their persuasive effect. What does it mean if language models can persuade best with facts and evidence?

The pessimistic view is that there will then be an arms race to recruit different facts to support different political poles. If the facts can’t be found they’ll be made up, and if they can’t be made up directly then the fact generating machinery of society will need to be co-opted so the ‘right’ facts get into the language models. The value of facts will become devalued, as everyone (and their language models) tries to find, misrepresent or invent, facts which support their favoured view.

Writing this out, it feels worryingly like it happens already, regardless of language models being widely deployed for persuasion. Language models might make a bad situation worse, but as you’d expect from this glass-half-full newsletter, I don’t think the value of facts has dropped, or will drop, to zero. Some corruption isn’t complete corruption. There’s more consensus and more trust about than the media represents, still.

If models are poisoned by made up facts then people will come to trust them less. The pessimist account assumes that people will become trapped in some kind of epistemic free-fall where there are no corrective mechanisms or filter of credulity, so the trust in what models say remains even as the reliability of what they say degrades. The two results I’ve written about here show that information presented by language models is persuasive but that doesn’t mean that all models, for all time, will be persuasive - especially if models demonstrate themselves to be unreliable and start pushing out lies.

What this makes clear is that a key issue for persuasion is that people believe and trust model output. Participants in these experiments bring an assumption of trust, or at least benefit of the doubt, to their interaction with the models. If that disappears, so does the persuasive power.

This puts some limit on the scare story that language models will release a ‘manipulation machine’ on society. Imagine retelling this story with books in the place of models. “Books are persuasive!” Because they contain facts and arguments. “But what if the books were made up of lies? Then you could use books to persuade anyone of anything!” We’ve had the ability to create books full of lies for thousands of years. I know enough about the religious wars of the 17th century not to lightly dismiss the risks of new forms of persuasion, but my argument is that a realistic understanding of the risks of persuasive language models needs to be understood as a continuation of the history of human persuasion, not a disjuncture.

That history includes the faltering, imperfect, but persistent power of human reason. The optimistic extrapolation of the results on the persuasive effect of language models recognises that it is possible to agree on some features of reality, despite conflict and context around all claims. This common ground is supported by the epistemic institutions of society: discussion one-to-one between individuals and small groups, the media, fact-checkers and the brute appearance of facts on the ground. As long as this holds, then persuasion in all forms - from demagogues, to books, to language models - has to work with the grain of reality, meaning that persuasion against the facts may be possible temporarily, but the advantage will be on the side of those making a case which best fits the truth. We should have no special fear of language models. If the models have to work with the same facts as spin doctors, influencers, pundits, politicians and conspiracy theorists then we already have the equipment to deal with them. The diagnosis must be the same: rigour, nuance, factfulness, dialogue.

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Upgrade to paid

Keep reading for more on fact-checking, AI, and updates from my life

A side note on accuracy

Hackenburg et al did an analysis of the accuracy of the facts deployed by their persuasive models. They found that the more persuasive, more advanced, models were less likely to be accurate, but they also found that (in)accuracy didn’t correlate with persuasive effect. Further, explicitly instructing the models to be as persuasive as possible regardless of the truth - i.e. to encourage dishonesty from the models - didn’t increase the amount of persuasion. Their conclusion is that this is compatible with a “side-effect” account, where greater inaccuracy doesn’t cause greater persuasion, but can result from other changes which the models makes in the pursuit of great persuasion. To me this sounds a lot like the formal definition of bullshit. It isn’t just models which can decide what to say based on how things sound, regardless of whether they are true or not.

I still maintain that long-term, if models sacrifice accuracy to persuasion people will learn not to trust them and so their persuasive effect will erode.

References

Lin, H., Czarnek, G., Lewis, B. et al. Persuading voters using human–artificial intelligence dialogues. Nature 648, 394–401 (2025). https://doi.org/10.1038/s41586-025-09771-9

Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., ... & Summerfield, C. (2025). The levers of political persuasion with conversational artificial intelligence. Science, 390(6777), eaea3884. https://doi.org/10.1126/science.aea3884

Related from me:

• The truth about digital propaganda

• AI-juiced political microtargeting

• How persuasive is AI-generated propaganda?

• Microtargeting is not mind control

Mike Caulfield: We’re not taking the fact-checking powers of AI seriously enough. It’s past time to start.

A walk-through of using AI for fact-checking which is illuminating enough on its own, but Mike also draws out three important take-aways:

#1 Stop assuming AI errors are hallucinations : the usefulness of the term “hallucination” stops when you get to investigating the different kinds of errors possible

Was it conflating? Leaning too heavily on social media sources? Misreading a primary document? If it was “muddled”, where did that muddle likely originate? Or was it truly a “hallucination” in the classic sense, bearing no discernable relation to any set of known claims or documents.

#2 Add AI to your toolkit to answer the questions you didn’t think to ask

This means that AI won’t just give you the answer to the question you thought to ask, it will also provide the context you *should* have asked about. This seems to be a fairly decisive advantage for AI, moving it beyond the simple Question-Answer / spicy autocomplete mode that many people assume when criticising it.

#3 Pay for a model already

I remain shocked at how many reporters and even fact-checkers do not use a paid model of AI — or even know it makes a difference.

It makes a huge difference!

I remain convinced that there is no future of verification and contextualization that doesn’t involve both better understanding of LLMs and more efficacious use of them. The three simple suggestions here — don’t prematurely dismiss errors as hallucinations, do use LLMs to surface the “unknown unknowns”, and do pay for the better models — are all pleas to engage more fully with this tech, and develop better (and more up-to-date) understandings of what they do well and what they do poorly, going beyond whether they fail or succeed to how they fail and how they succeed. This is also what we need to do in education. If we can start from there I’m confident our effort and engagement will be well-rewarded.

Link: We’re not taking the fact-checking powers of AI seriously enough. It’s past time to start.

Talks

I’ve updated my Talks page, so if you’d like to hear me speak about some of the topics covered in this newsletter (and some others) look out for upcoming events online, in London and Liverpool.

Related: I’m in London (and Sheffield) weekly, so let me know if you’d like to meet up.

CATCH-UP

Recently from me:

Life news: London calling / Tools for Thought An update on my career break, plus a strong endorsement for Advait Sarkar’s vision of AI. Original career break news: Some personal news

New work :The Ideological Turing Test: Do you truly understand those you disagree with? and Community Notes require a Community: How we used a novel analysis to understand what causes people to quit the widely adopted content-moderation system

Commentary: When less (communication) is more (collective intelligence): Team work, complex problems, and preserving diversity in the ecosystem of ideas. Listening to Tommy Robinson: What did I learn by giving the right-wing activist my ears for 90 minutes? Helping people spot misinformation: And the greatest gift psychology gave the world

… And finally

Strong early contender for snowperson of the year (via)

END

Comments? Feedback? Reasoned persuasion? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online