Another Academic Data Manipulation Scandal: US News Rankings

Evers has bragged about US News’ ranking of Wisconsin’s failing schools among the best in the nation

US News does not verify or vet self-reported data

Elite colleges have been involved in data manipulation scandals on multiple fronts this year

We called out Governor Evers for touting a US News ranking showing our K12 schools in the top 10 in the US, a rating that as an educator, Evers knows is deeply flawed.

It doesn’t take the greatest minds in academia to catch the disconnect:  DPI data shows that over half (53%) of students are failing math and English, and just over a quarter of students are performing at grade level in those basic subject areas, but US News says that makes Wisconsin #8 in the nation.

There is clearly something amiss with their rankings.

US News recently pulled the 2nd place ranking of Columbia University after faculty there raised questions about the data used.  Nine other colleges also lost their rankings this year as well. And it’s not the first time elite schools have been caught self-reporting phony data that US News turned into phony rankings.

While US News’ spin team has done an admirable job directing blame at the schools, the scandal has revealed the mainstream media outlet doesn’t bother to have independent verification or vetting of the information provided to them by schools – or even do so internally.  US News says they don’t have the resources to check out the accuracy of the data they’re reporting, so they just print whatever they’re given.

In other words, this major mainstream news outlet simply regurgitates self-reported, unverified data from sources with a very specific agenda, never questioning, or caring, if what they publish is accurate, trustworthy, or reflective of reality.

College leaders have long questioned the accuracy of the US News rankings but continue to cooperate with US News because parents and students put stock in them.  And because higher rankings bring more tuition dollars, there is ample incentive – without risk – to fudge the data.

New UW Chancellor Jennifer Mnookin mirrored this sentiment in a tweet Monday about UW-Madison’s move up the US News rankings to #10 in public institutions.  She bragged about the improvement while hedging a little on the value of rankings.  Is she simply acknowledging awareness of the scandal, or does she know something we don’t about the UW data?

 

It’s worth noting that #1 ranked Princeton was recently involved in a data manipulation scandal we reported on where their own staff at the Princeton Gerrymandering Project reported the lab’s data was doctored to benefit democrats in state redistricting cases where they posed as ‘neutral’ advisors.  Princeton launched a cursory internal investigation and a couple of days later declared ‘there’s nothing to see here, carry on.’  The Gerrymandering Project staff who came forward found out exactly what happens to Princeton employees who complain about lapses in integrity, so it’s unlikely anyone there will risk reputation and career by coming forward to question their US News ranking.

And US News is never going to check.

It should be a fundamental expectation that our educational institutions, at every level, uphold standards of integrity and honesty.  When DPI secretly rigged the school report card accountability scoring ranges to disguise how many schools were underperforming, we called them out on it.  They should be held accountable for cooking the books.

But the media is supposed to be responsible for confirming the stories and the data that they’re given before reporting it as fact.  US News created a ranking system where lies are rewarded with high rankings and financial gain.  Honesty and accuracy don’t matter and the repeated controversies represent only the fraction of data manipulation that is discovered.

For the US News Best High Schools rankings, proficiency scores only count for 20% of a school’s ranking.  Another 20% is based on entirely subjective “US News’ expectations” of performance. Another 10% is based on a comparison of the proficiency scores of minority and poor students with non-underserved students “with parity or higher being the goal.”

The graduation rate, which has become an increasingly manipulated data point, accounts for another 10%. Curiously, US News lists the graduation rate of every high school in Wisconsin at 93% and lists that 93% variously as “somewhat above the state median,” and “well above the state median.”

The remaining 40% is apparently based on college readiness measures that don’t apply to many Wisconsin schools. There is no way a ranking based on 20% on student performance and 80% on subjective expectations and comparisons, suspect data, and categories where data is missing from large numbers of participants can possibly show the “Best High Schools.”

Some have called for US News to suspend their college rankings until they are able to institute a system that cannot be manipulated and is based on data that can be verified.  With both the college and high school rankings clearly flawed, we believe US News should suspend all their rankings until they can develop an objective, accurate, verifiable measures that don’t mislead the public.  And Governor Evers – who once claimed to believe that ignoring the problems in our K12 system wrecked credibility – should stop using US News’ scandal-plagued, unsound rankings to mislead Wisconsinites about the dire situation facing our schools.