The purpose of this research is to experimentally test whether counter-narratives are effective to reduce people’s support and willingness to join Islamic State of Iraq and Syria (ISIS). Integrating psychological reactance theory (Brehm, 1966) and need for closure (NFC; Kruglanski, 2004), we predicted that exposing people to counter-narratives when they are at greater risk of radicalization (high NFC individuals) would be counterproductive and enhance their support for ISIS. Participants (N = 886 American Muslims) were randomly assigned to a 3 × 3 factorial experimental design varying the source (United States Government, Imam, ISIS defector), and the content (social, political, and religious) of the counter-narrative while comparing these groups to a control message. Results show an overall small positive effect of counter-narratives (β = −0.107, p = 0.043), but also evidence for greater support for ISIS in individuals at greater risk of radicalization (β = 0.154, p = 0.005). Results also show that the content was more important than the source: A political narrative was the most effective, and this result is consistent across different sources although an ISIS defector is the most effective messenger. These findings challenge the widespread assumption that counter-narratives are effective against violent extremism. In fact, they accelerate the very phenomenon that governments and policy makers are trying to undermine. Therefore, policy makers should avoid including them in their armamentarium to tackle violent extremism.