{"id":302,"date":"2013-12-19T07:00:26","date_gmt":"2013-12-19T12:00:26","guid":{"rendered":"http:\/\/datacolada.org\/?p=302"},"modified":"2020-11-18T23:17:41","modified_gmt":"2020-11-19T04:17:41","slug":"11-exactly-the-most-famous-framing-effect-is-robust-to-precise-wording","status":"publish","type":"post","link":"https:\/\/datacolada.org\/11","title":{"rendered":"[11] \u201cExactly\u201d: The Most Famous Framing Effect Is Robust To Precise Wording"},"content":{"rendered":"

In an intriguing new paper<\/a>, David Mandel<\/a> suggests that the most famous demonstration of framing effects \u2013\u00a0Tversky & Kahneman's (1981)<\/a> \u201cAsian Disease Problem\u201d \u2013 is caused by a linguistic artifact. His paper suggests that eliminating this artifact eliminates, or at least strongly reduces, the framing effect. Does it?<\/p>\n

This is the perfect sort of paper for a replication: The original finding is foundational and the criticism is both novel and fundamental. We read Mandel\u2019s paper because we care about the topic, and we replicated it because we cared about the outcome.<\/p>\n

The Asian Disease Problem<\/span><\/p>\n

Imagine that an Asian disease is expected to kill 600 people and you have to decide between two policies designed to combat the disease. The policies can be framed in terms of gains (people being saved<\/i>) or in terms of losses (people dying<\/i>).<\/p>\n

\"\"<\/div>\n

Tversky and Kahneman found that whereas 72% of people given the gain frame chose Program A\u2019s 200 of 600 certain lives saved, only 22% of people given the loss frame chose Program C\u2019s 400 of 600 certain deaths. This result supports prospect theory<\/a>: people will take risks to avoid losses but will avoid risks to protect gains.<\/p>\n

Just a Linguistic Artifact?<\/span><\/p>\n

David Mandel <\/b>argues that when people read \u201c200 people will be saved\u201d or \u201c400 people will die\u201d they interpret it as \u201cat least <\/i>200 people will be saved\u201d or \"at least<\/i> 400 people will die\u201d. That small difference in interpretation would switch the finding from irrational to sensible, as the certain option would potentially save more people in the gain frame (>200 of 600 will be saved) than in the loss frame (>400 of 600 will die). Mandel resolves the ambiguity by adding the word \u201cexactly\u201d (\u201cexactly 200 people will be saved\u201d). Because the word \u201cexactly\u201d makes it clear that no more than 200 will be saved, he predicts that including that word will eliminate the framing effect.<\/p>\n

In Study 2 (the study most faithful to the original), Mandel used Tversky and Kahneman\u2019s wording and replicated their result (58% chose to save 200 people for certain; 26% chose to kill 400 people for certain). When he added \u201cexactly,\u201d that difference was reduced (59% vs. 43%). [1<\/a>]\n

We replicated Mandel\u2019s procedure. We showed mTurk workers the same scenario and asked the same questions. We collected ~2.5 times Mandel\u2019s sample size; Mandel had ~38 per cell and we had ~98 per cell<\/a>. (Following Mandel, we also included conditions with \u201cat least\u201d as a modifier;\u00a0here<\/a> are those results).<\/p>\n

Unlike Mandel, we found a strong framing effect even with the use of the word \u201cexactly\u201d (p<.001) (materials<\/a>;\u00a0data<\/a>):<\/p>\n

\"\"<\/div>\n
\n

For completeness, we should report that Mandel emphasized a different dependent variable \u2013 a continuous measure of preference. We measured that too and it also failed to replicate his result<\/a>.<\/p>\n

In sum, our replication suggests that Tversky and Kahneman\u2019s (1981) framing effect is not caused by this linguistic artifact.<\/p>\n

We Could Have Just Asked Uri<\/span><\/p>\n

When we told Uri about all this, he told us that he conducts this experiment in his class each year and that he uses the word \u201cexactly\u201d in his materials<\/a>. The experiment has replicated every single year. For example, in the past two years combined (N=250), he observed that 63% chose to save 200 lives for sure whereas only 25% chose to let 400 die for sure.<\/p>\n

David Mandel Responds<\/span><\/p>\n

As is our policy, we sent a draft of this post to David Mandel to offer him the chance to respond. Please check out David's response below. We are very thankful he took the time to do this:<\/p>\n

I welcome this replication experiment by Joseph Simmons and Leif Nelson. I think we all agree proper replications are important regardless of how the results turn out. They have kindly offered me 150 words to reply, but that would hardly get me started. There are many points to cover, both about the replication and some of the broader issues it sparks. Joe contacted me on the Friday before the post went live with the \u201cunwelcome news\u201d and it\u2019s been a weekend of changed plans, but I wanted to have a reply ready when the post goes live. Here it is <\/a>on my website [and here<\/a> it as a pdf]. I hope you read both. Feel free to email me<\/a> if you have comments. Lastly, Joe invited me to comment on their post, but I haven\u2019t since I cover what I might have otherwise recommended they alter in my reply. Readers can make up their own minds. 151, 152\u2026<\/p><\/blockquote>\n

Subscribe to Blog via Email<\/h2>\n\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t\t

Enter your email address to subscribe to this blog and receive notifications of new posts by email.<\/p>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t