Three years after Congress pressured the NFL to overhaul its concussion program, the league effort remains marked by inconsistencies in how it tracks, manages and even describes serious head injuries, making it difficult to assess the league's progress on the issue, an analysis by ESPN's "Outside the Lines" and PBS "Frontline" shows.
The analysis found that NFL officials this season have released conflicting data about head injuries, and medical personnel have sent some injured players back into games -- possibly in violation of new league guidelines.
The "Outside the Lines"-"Frontline" analysis was based on interviews with concussion experts inside and outside of the NFL and an examination of weekly injury reports over the past four years. Among the findings:
-- The NFL has released two different sets of statistics this year about the number of concussions that occurred in 2010 and 2011, making it impossible to evaluate the league's claims that brain injuries decreased significantly during that period.
-- Teams use different and imprecise terms to describe concussions on injury reports, obscuring the prevalence and severity of head injuries from week to week.
-- On some occasions this season, team medical personnel have sent players back into games despite apparent symptoms that are listed under league guidelines for declaring a player "No Go," or unable to return.
"I'm gonna be blunt here: We're so primitive," said Dr. David Dodick, a neurologist who examines some NFL players as director of the concussion program at the Mayo Clinic in Scottsdale, Ariz. "Twenty years from now, we'll look back at ourselves and laugh."
Dodick said one NFL player recently told him: "Concussion is a four-letter word in the locker room right now."
The "Outside the Lines"-"Frontline" analysis highlights the challenges facing the NFL as it grapples with an expanding lawsuit involving nearly 4,000 former players and growing public pressure to take the lead against an injury that concussion experts have described as cryptic and elusive.
After congressional hearings in 2009, the NFL disbanded its Mild Traumatic Brain Injury Committee, which had denied the link between football and chronic brain damage, and initiated a series of measures, including mandatory guidelines for removing injured players from games.
Those measures, combined with significant research grants made by the NFL to the National Institutes of Health and other organizations, have won praise for the league and commissioner Roger Goodell.
"I think it's getting better," said Dr. Richard Ellenbogen, a Seattle neurosurgeon who co-chairs the league's Head, Neck & Spine Committee, the NFL's new policy-making group. "Is it perfect? No. We've got a lot of education to do. We're trying to build a better mousetrap."
In February, the NFL publicly announced a 13 percent decline in concussions from the previous year, attributing the drop from 218 concussions in 2010 to 190 in 2011 to increased awareness and the decision to move kickoffs from the 30- to the 35-yard line.
Yet last week, one of the league's brain-injury experts provided a different set of statistics to "Outside the Lines"-"Frontline," showing a dramatically higher number of concussions than in the February report, with only a nominal decrease between 2010 and 2011.
Dr. Hunt Batjer, co-chair of the NFL's Head, Neck & Spine Committee, referred questions about the discrepancies -- a gap of 50 reported concussions in 2010 and 73 in 2011 -- to Greg Aiello, a league spokesman. Aiello said the first set of statistics released by the league did not include concussions that occurred in the postseason or practice.
Batjer said almost all the concussions occurred during games, not practice; if true, Aiello's explanation suggests that more than one-quarter of all reported concussions in 2011 occurred during the playoffs -- 11 games involving 12 teams -- an average of nearly seven concussions a game.
The NFL's initial data suggested that less than one concussion occurred per game.
Dustin Fink, a certified athletic trainer in Illinois who has been tracking NFL head injuries on "The Concussion Blog," described the explanation on the discrepancies as "beyond ridiculous."
"What this information tells me is that the league does not have a solid/reliable way to account for concussions all season long," Fink said in an email to ESPN. "… Their numbers the past two years show they do have an internal set that is beyond the official injury reporting system the public sees."
The inconsistency of the NFL also can be seen on weekly injury reports, which show discrepancies in how often individual teams report concussions and how they describe head injuries.
The Cleveland Browns, for example, listed 27 players with concussions from 2009 to now, the most in the NFL during that period. The Cincinnati Bengals and Miami Dolphins each reported six. The Bengals reported no concussions last year; the Dolphins reported none in 2009. Cincinnati has had three players listed with concussions this season, while Miami has reported just one (running back Daniel Thomas, concussed twice), the fewest in the league through the first 14 weeks.
In addition, terminology varies widely throughout the league; some teams use the term "concussion" on the injury report; others use the generic term "head." Five teams -- Buffalo, Chicago, Houston, Kansas City and St. Louis -- have used both this season. Since 2009, a few teams appear to have identified a single player as having both a head injury and a concussion while describing the same injury from week to week.
Ellenbogen said the terms are "synonyms," adding that the Head, Neck & Spine Committee should address the matter to make reporting "more uniform."
In 2011, Ellenbogen and Batjer, a renowned Dallas neurosurgeon, sent a memo to team doctors, trainers and neurological consultants outlining new measures for "game-day care."
Those measures included strict guidelines on sideline examinations and the implementation of "The Madden Rule," which requires a player diagnosed with a concussion to leave the field immediately and remain in the locker room under the observation of the team's medical staff.
The memo concludes: "WHEN IN DOUBT LEAVE THEM OUT: If you have any suspicion about a player being concussed, remove him from the game."
But some instances this year raise questions about whether those rules are being followed.
On Sept. 16, early in the second quarter of a game in Week 2, New York Jets running back Shonn Greene was blasted on an illegal helmet-to-helmet hit by Steelers safety Ryan Mundy. Greene returned to the huddle, dazed and wobbly, until quarterback Mark Sanchez effectively removed him from the game, pushing him toward the sideline.
The NFL's new Standardized Concussion Assessment Tool, or SCAT, lists six "obvious signs of disqualification (i.e., No Go)" from play. They include confusion and amnesia. The Jets said Greene was "dizzy" after the collision but that his symptoms quickly cleared. Later, Greene said he did not remember Sanchez pushing him to the sideline.
The Jets said during the game that Greene had sustained a head injury and was questionable to return.
A Jets doctor, Damion Martins, administered a memory and balance test in the locker room after the injury. Greene said he was asked to recite the months of the year backward and the score. He also was given a balance test in which he was required to stand on one leg and close his eyes.
"Out of respect for the injury, we took him to the locker room to perform a thorough evaluation to be sure," a Jets spokesman wrote ESPN in response to questions last month. "We were concerned enough to perform the testing, but all signs and tests suggest that he did not have a concussion."
The Jets said Greene was allowed to return to play in the second half because he passed every test.
Greene told reporters the next day that he was "fine" and didn't have a concussion.
Last month in an interview with ESPN, Greene said: "I think what it was, when it first happened, I kind of got up too fast from the hit. I wasn't all the way there. It was kind of shaky."
Two weeks later, Lions receiver Calvin Johnson took a helmet-to-helmet hit from Vikings linebacker Chad Greenway, who was later fined $21,000. Johnson was slow to get up and went to the sideline, where he was tested for a concussion. He returned about six minutes later.
Johnson was not listed on the team's injury report when the Lions returned from a bye week after the Vikings game, yet, according to the Detroit Free Press, he told reporters at least three times that he had suffered a concussion.
"Yeah, yeah, he knocked me good," Johnson said. "You could tell. It was obvious."
The Lions disputed Johnson's claims about his injury. More than a month later, the team and the player issued a joint statement in which Johnson retracted his claim and said he had misused the term "concussion."
Experts say the challenges in identifying concussions are compounded by players being reticent to self-report symptoms because they want to stay on the field and fear losing their jobs. That issue came into sharper focus recently with the case of Alex Smith. The 49ers quarterback admitted he was having double vision after taking a hit to the head in a Nov. 11 game against the Rams, was diagnosed with a concussion and then, despite recovering, he lost his starting spot to Colin Kaepernick.
Dodick, of the Mayo Clinic, said he believes the NFL protocols are "reasonably sound, but they're not necessarily being followed."
The pressures on the sideline are too great, he said. Players want to play, and trainers and doctors, even those with the best intentions, wind up torn.
"The pressures must be enormous," Dodick said. "Doctors want to keep their patients happy; that's the bottom line."
Dodick said he supports a proposal by the NFL Players Association to have an independent neurologist on the sideline. The league has rejected the idea, arguing that team medical personnel and doctors are most familiar with the players and thus best suited to treat them.
Ellenbogen said he doesn't support the idea of independent doctors on the sideline, and he believes players might be even less inclined to self-report a concussion to someone they don't know.
"Because there is no rapid test or X-ray for concussion, one depends on the sideline medical team to know their players and be able to recognize when they are different than the baseline," he said. "I think an independent doctor would miss the nuances of the exam and behavior that someone who knows the player would not."
Without addressing specific cases, Ellenbogen acknowledged that teams are still learning and that concussions are missed. But he said it is unreasonable for people to make judgments watching on television.
Margot Putukian, director of athletic medicine at Princeton University and chair of the NFL's Return to Play subcommittee, agreed. "It's a challenge; there's an elusiveness to the injury," she said. "Signs and symptoms can be delayed. The sideline assessment can be 'normal.' And the athlete can still have a concussion. It's easy to be the Monday morning quarterback."
ESPN researcher Rachel Eldridge, ESPN reporter Rich Cimini, "Frontline" reporter Sabrina Shankman, and "Frontline" researcher Jason Breslow contributed to this report.