Nielsen Confirms Undercount of Pandemic TV Viewership Following Standoff With Networks

The Media Rating Council, a nonprofit organization that audits Nielsen ratings, says that Nielsen’s Total Usage of Television time (or TUT) from February was lowballed by between 2% and 6% among viewers in the adults 18-49 demo most coveted by advertisers, and that people using the TV among that same demo (known as Persons Using Television, or PUT) were understated by between 1-5% in the same month—undercounts that could translate to hundreds of millions of dollars in lost advertising spend.  

Nielsen’s data defects, the Media Rating Council alleged Monday, were caused by how Nielsen managed its in-home panel, which comprises the backbone of the media ratings system that media’s sell-side and buy-side use to set rates and transact on, and which has been at the center of a months-long controversy between the measurement firm and one of TV’s biggest industry groups.

“While certain programs showed no change in their rating, and some even had lower ratings in the simulated results, MRC believes these may have been partly a function of the analytical approach used, where homes were removed for the purpose of the simulation, resulting in greater variability, as well as the fact that the analyses were conducted prior to the completion of maintenance procedures for all suspect panel homes,” the organization said in a statement late Monday. “In short, MRC believes changes to program estimates are largely one-directional, and that there was some degree of understatement overall in the C3 Persons 18-49 estimates that were originally reported by Nielsen in the February 2021 period.”

The unprecedented statement was soon followed by confirmation from Nielsen itself, which reviewed its own data after being contacted by the organization. But the measurement firm says that most ratings saw only miniscule ratings changes due to the undercount.

“As a result of some of the Covid measures we implemented, we found that there was some understatement of audience estimates,” a Nielsen spokesperson said in a statement. “The variance differed by daypart, demographic and program. At a high level, the February 2021 simulation analysis showed: 2-6% change on Total Usage of Television (TUT) ratings and a 1-4% change on Persons Using Television (PUT) ratings. 93% of C3 P18-49 rating changes for major networks saw no more than a 0.02 change in rating points.”

‘The tip of the iceberg’

The development marks the latest in an ongoing salvo between Nielsen and the industry group the Video Advertising Bureau, which represents the major television networks. A month ago, the VAB furiously charged Nielsen with under-counting pandemic television measurement, citing the measurement firm’s pause in its maintenance of its in-home panel households in the early days of the pandemic. The networks had demanded that Nielsen undergo a third-party audit, which Nielsen declined; after that, the VAB and individual networks began speaking to the MRC, which was undergoing its own review into Nielsen’s figures.

The results of that review from both Nielsen and the MRC were presented Monday in a closed-door MRC TV committee meeting, comprised of TV buy-side and sell-side clients. While the contents of the meeting were not made public, VAB CEO Sean Cunningham told press after the meeting concluded that there was information suggesting far more analysis would be required to determine just how exactly the figures should be revised.

“Both Nielsen’s analysis and the MRC statement essentially contradicts months of Nielsen’s denials on any significant problem and calls for much deeper scrutiny into the accuracy of their Covid-period data,” Cunningham said. “What we saw was, to say the least, the tip of the iceberg.”

The VAB began ringing the alarm publicly about Nielsen’s pandemic audience figures a month ago, but for months, network leaders had been frantically meeting with Nielsen behind the scenes asking for information about discrepancies and seeking clarity about how Nielsen was managing its panel. Separately, the MRC had begun examining 9,400 homes that comprised part of its panel data set, which the group believed were impaired in some way due to a technical malfunction or change in living conditions.

Publicly and privately, Nielsen had defended its figures and its handling of in-home panel maintenance throughout the pandemic, saying that its guidance—including a white paper on the subject—has been thorough.

“As requested by the MRC, Nielsen has implemented a thorough analysis of the estimated impact of changes in consumer viewing behaviors and its panel maintenance protocols during the Covid-19 pandemic, and we will continue to work with the MRC on other analyses in the future,” a Nielsen spokesperson said in the statement. “Throughout the pandemic, Nielsen has been fully transparent in collaborating with the MRC and focused on procedural changes to support its panelists, people and the integrity of currency metrics used by the industry.”

Some members of the VAB disagree, and Cunningham charges that Nielsen withheld information that could have been crucial or could have addressed audience figures prior to a busy upfront season. Cunningham applauded the VAB’s members for speaking out about their own data discrepancies as a way to hasten the review process.  

“There was a deep amount of immersion from our members to the leaders of the MRC, going through a lot of specific data defects,” Cunningham said. “Some of the defects that the MRC wound up exploring and outing into the initial analysis were brought to their attention by my members … Had not Nielsen’s largest clients done these exercises and challenged them with these analyses, these would have never come to light.”