{"id":4535,"date":"2019-11-20T07:00:33","date_gmt":"2019-11-20T12:00:33","guid":{"rendered":"http:\/\/datacolada.org\/?p=4535"},"modified":"2020-02-11T23:59:25","modified_gmt":"2020-02-12T04:59:25","slug":"80-interaction-effects-need-interaction-controls","status":"publish","type":"post","link":"https:\/\/datacolada.org\/80","title":{"rendered":"[80] Interaction Effects Need Interaction Controls"},"content":{"rendered":"

In a recent referee report I argued something I have argued in several reports before: if the effect of interest in a regression is an interaction, the control variables addressing possible confounds should be interactions as well.<\/em> In this post I explain that argument using as a working example a 2011 QJE paper (.htm<\/a>) that examined domestic violence following NFL games.<\/p>\n

I chose that paper because it provides an intuitive setting to explain the need for interaction controls, and because it is the paper that first prompted me to think of this issue (several years ago).<\/p>\n

The 2011 QJE paper
\n<\/strong>This excerpt from the abstract summarizes the key finding:
\n\"\"<\/a>
\n\u00a0 \u00a0 \u00a0 \u00a0Codebook.<\/u><\/em><\/span>
\n\u00a0 \u00a0 \u00a0 Pregame point spread<\/em>: Number of points the favorite team is expected to win by.<\/span>
\n\u00a0 \u00a0 \u00a0Local viewing audience<\/em>: How many locals watched the game on TV.<\/span><\/p>\n

Table IV in the paper has the key result:
\n\"\"<\/a><\/p>\n

Each column is a different (Poisson) regression with domestic violence after an NFL game as the dependent variable. As we move right the regressions include more controls.<\/p>\n

The first row shows the estimated effect of a surprising vs. expected loss. The point estimate is about 0.10, corresponding to that 10% increase in violence mentioned in the abstract (scroll up).<\/p>\n

Viewership: a confound that worried the original authors
\n<\/strong>As the QJE paper notes, one concern with the finding is that more people watch games when their team is expected to win (see footnote for details of the authors\u2019 original discussion of this issue [1<\/a>]). This means that more people watch surprising losses than watch expected losses. We thus have two explanations for greater violence after surprising losses:<\/p>\n

Explanation 1: surprising losses are more enraging.
\nExplanation 2: surprising losses are watched by more people.<\/p>\n

Column (5) in Table IV above intends to tease these two apart by controlling for viewership (the \"Nielsen rating\" row). But the point of this post is that such control is not sufficient.<\/p>\n

If the confound we were worried about was that the bigger the number of fans watching any game<\/span>, the more violence there is, then controlling for the Nielsen rating as a main effect would be the right solution.\u00a0 But we are concerned with something different. The confound we are worried about is that the bigger the number of fans watching a loss<\/span>, the more violence there is.<\/p>\n

More generally, because the effect of interest is an interaction, any alternative explanation for it must also involve an interaction. In this particular case, the effect of interest involves an interaction with the team losing, and therefore, any confound must also involve an interaction with the team losing. So we want viewership to potentially have a different effect for games won vs lost, so we need to add the term Nielsen Rating*Loss<\/span> <\/em>as a predictor.<\/p>\n

I make the same point with simple regression equations in this footnote [2<\/a>].<\/span><\/p>\n

This is a relatively common issue.
\n<\/strong>I browsed the last two issues of some top economic journals (AEJ:Applied, AER, & QJE) and found several papers with regressions where an interaction effect was of interest, and where controls to address confounds were included. But I did not find any that included interaction-controls. The need for interaction-controls does not seem to be an issue applied researchers are generally aware of.<\/p>\n

Punchline<\/strong>.
\nWhen the coefficient of interest is an interaction, and confounds are a concern, interacted<\/em> controls are needed.<\/p>\n

\"Wide<\/p>\n

\n

Author feedback<\/strong><\/span>
\nOur policy (.htm<\/a>) is to share drafts of blog posts that discuss someone else's work with them to solicit suggestions for things we should change prior to posting. I shared a draft with David Card and Gordon Dahl. They did not suggest any changes but Gordon reported the results with the suggested interaction controls, writing [I added the approximate p-values in [ ] for readers who may be interested in them]:<\/span><\/p>\n

\"adding these interactions does not appreciably change the point estimate on the upset loss variable: it becomes .095 (.056) [approx. p-value=.09].\u00a0 This compares to the estimates reported in the QJE paper of .100 (.0310)\u00a0 [approx. p-value <.001] when we only control for Nielsen rating and .096 (.031) when we don't control for Nielsen rating at all.\u00a0 Interestingly, the coefficients on rating*loss and rating*win are almost identical: .0031 versus .0034, respectively.\u00a0 Moreover, our test for loss aversion (upset loss = – upset win) has a p-value of .02 now, whereas before it was .01.\u00a0 So our conclusion is that adding in the interaction terms with ratings results in less precise estimates, but doesn't change the estimates appreciably<\/em>.\"\u00a0<\/span><\/p>\n

I thank David and Gordon for their time, and for dusting off the STATA code from nearly 10 years ago for this post.<\/span><\/p>\n

Shortly after this post went live, Dominique Muller (@dom_muller<\/a>) alerted me to this JESP (.htm<\/a>) paper dealing with this very issue and documenting that psychology papers don't include interaction controls.<\/span><\/p>\n

Subscribe to Blog via Email<\/h2>\n\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t\t

Enter your email address to subscribe to this blog and receive notifications of new posts by email.<\/p>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t