Why We’re Preemptively Banning A Pollster — And Not Banning Another

FiveThirtyEight is in the business of grading and reporting on polls — so when a pollster allegedly engages in misconduct, we take it seriously. In November 2022, an anonymous interviewee in The Tartan, Carnegie Mellon University’s student-run newspaper, alleged that Sean McElwee, the then-executive director of polling firm Data For Progress, was betting on elections. We immediately began our own investigation, led by reporter Kaleigh Rogers; today, we published the results of that investigation — the full story of McElwee and Data For Progress as best we could determine.

The allegations also prompted us to reconsider McElwee’s and Data For Progress’s inclusion in our polling aggregates. Based on the results of our investigation, we have decided to ban any future polls conducted under McElwee’s direction from appearing in FiveThirtyEight’s database, averages and forecasts. However, Data For Progress — which recently severed ties with McElwee — will remain a pollster in good standing.

Here’s why we arrived at these decisions. FiveThirtyEight is intentionally permissive about which polls we include in our database and averages; polls must meet only the bare minimum methodological or qualitative standards. But our polls policy does state that “we will permanently ban any pollster found to be … engaging in betting markets that may be directly impacted by their survey work.” 

That’s because a pollster betting on politics can be akin to insider trading: Polls can significantly influence the prices of these markets, so pollsters who have conducted a poll but not released it yet have foreknowledge of how a market will react. More importantly for our purposes, pollsters having a financial stake in these markets raises questions about their polls’ intent and the integrity of their design and results. If a pollster has $1,000 riding on what President Biden’s average approval rating will be on a specific day, we cannot trust that he or she will put out unbiased numbers.

To be clear, we uncovered no evidence that McElwee manipulated Data For Progress’s polls for financial benefit. However, we found ample evidence that McElwee did indeed bet on politics. He tweeted about doing so as far back as 2020, and he acknowledged wagering $20,000 on the 2020 election in a podcast interview in August 2022. That alone is sufficient for a ban, in the event he returns to the polling industry. (In an email, McElwee told FiveThirtyEight he has “no plans to conduct polling that would be ingested by the 538 model in my future work.”)

That ban will not extend to Data For Progress, however. Simply put, it has reacted exactly as we would expect a professional organization to do in this situation. Staffers there told FiveThirtyEight that they are conducting an internal investigation into McElwee’s behavior, and they are confident he did not improperly influence their polls. They also told us that all staffers had to sign new employment agreements that prohibit betting on any topics Data For Progress works on. And most importantly, senior staff at Data For Progress ousted McElwee as executive director, and he is no longer employed at the firm. As a result, we are confident in the integrity of Data For Progress’s current polling operation, and we will continue to use its polls on our site.