As the football season gets underway every season, two things are certain: There will be some headline-grabbing concussions, and we'll hear the National Football League (NFL) tell us that player health and safety remain a top priority.

The contrast between appearances and that claim is striking. Few fans could forget Miami Dolphins quarterback Tua Tagovailoa stumbling on the field after a huge hit, being cleared to play and then sustaining another scary injury a few days later. The scene of a wobbly player being escorted off the field after a head impact is all too familiar on high school fields, in college stadiums, and, of course, in the NFL. Yet we hear claims that the game has gotten safer, as the NFL regularly reports progress in the fight against concussions, citing new rules and equipment, greater awareness, improvements in its concussion protocol, and continued research into the issue.

Can we be confident that all of these efforts are making a difference? As we’ll see, the answer may well be no.

We’ll get to that answer by looking at the data underlying the NFL’s persistently optimistic interpretation of concussion counts and show how those numbers parallel the year-to-year fluctuations in other injuries. We’ll also look at long-term trends—recent concussion tallies surpass those of past decades—and discuss why there are reasons to question the accuracy of the underlying data. With all these nuances in mind, we’ll explore whether the NFL might be mistaking random variation for cause and effect and then justifying policy changes using this unconvincing evidence.

Heads I win, tails I also win

According to its officials, the NFL has made continual progress in its fight against concussions in the past decade, basing that claim on the number of concussions reported. In 2014, the league announced an impressive 25 percent decrease in concussions—a trend that NFL SVP of Health and Safety Jeff Miller described as “heading in the right direction.” That four-year low of 202 concussions (combined preseason and regular season) was attributed to improvements in culture and rule changes. Following this, NFL Commissioner Roger Goodell declared that “Football has never been safer than it is right now” in the league’s 2015 NFL Health and Safety Report. Miller delivered a similar message following the 2018 season (214 concussions), saying that the NFL was “pleased with the progress on concussion reduction.” Progress was cited yet again in 2020.

If things improve that regularly, you’d expect that concussion counts would be on a downward trend. They’re not. And paradoxically, you get similar statements about improved player safety in years where the concussion count increases.

The NFL has attempted to improve concussion awareness in athletes, medical providers, and management (which is a good thing), so it attributes a higher concussion count to evidence that concussions are now recognized more often through increased screening and self-reporting. This was a major talking point when regular season game concussions increased by 58 percent in 2015. When concussions jumped 13.5 percent in 2017, the NFL emphasized this was “amid a rise in self-reporting of symptoms.” In response to an 18 percent increase in concussions in 2022 (213 total), NFL Chief Medical Officer Dr. Allen Sills explained how a broader definition of "concussion" was the root of the rise.

Even when the league recognized that a 5 percent increase in 2019 (224 concussions) was “statistically similar” to the previous year (214 concussions), that was said to have served as validation of its safety efforts.

Could it be that players have gotten better about recognizing and reporting symptoms, so more concussions are being diagnosed? Or are there years where there are simply more concussed players and/or players experiencing more severe symptoms? We don’t know—and in fairness, the NFL mentions both as a possibility. Yet the league touts a message of improved player safety regardless of which direction the numbers go. If concussions decrease, it’s because the new safety measures are working (rather than under-reporting). If concussions increase, it’s because everyone is more astute in recognizing concussions (rather than more injuries). Any change in numbers translates to a “safer game”—which seems like a textbook case of confirmation bias.