School Rankings Database Methodology

Data on public school student test results, and graduation and dropout rates had already been requested by Times database reporter Tom Saul in early 1998, which gave us a leg up on the project. Tom left The Times that summer to go to the Quad City Times, and I inherited the project from him. It was in the rudimentary stages. Some of the data had been keyed in, but much of it was still coming in from school authorities and some necessary information, particularly rates of reduced meal participation, a key indicator of the relative prosperity or poverty of student bodies, was totally lacking.

So began six months of the hardest work I've ever undertaken for The Times. Dozens of faxes and letters flowed to and from the paper and the school systems in 10 parishes, as well as the state Department of Education. Reams of data got keyed in, was tested, crunched, tested and crunched again. I printed data, showed it to the bosses, tinkered with it as needed, printed it out again, tinkered with it, ad infinitum. My only memoir today: a 25-pound box of mailings, printouts, floppies and CD-ROMs burned with the information gleaned.

We had to establish the formula to rank schools. The state had described its basic process, but this had to be turned into a formula that would work under Excel. So we paid a consultant at a nearby university to do this. In retrospect, the formula closely followed the state description and could have been devised by our team with no problem. Remember one of the basic rules of database reporting, though: Trust nothing. The formula we purchased was flawed, in that it treated a school's dropout rate as a POSITIVE rather than a negative. I realized this only after the first sets of data had been given to the reporter on the project. Luckily, the fix was easy -- subtract the dropout rate from 100 to get a retention rate and then weight that into the overall formula -- and the corrected data was given to the reporter. That catch saved the project, though, for it affected the choice of schools to be examined and the entire "report card" for parish schools that was the direct result of all my database work.

Once we had the schools ranked, we had to quartile them, both by their performance levels and by their socioeconomic rankings. This allowed us to compare schools' performance within similar demographics, so we could compare academic oranges with oranges and apples with apples, so to speak. Obviously, comparing a well-performing school with an affluent student population with a lesser-performing one with a much poorer group of students would not break much ground. But if you have two schools within the same demographic, and one performs much better than the other, then you have hard evidence with which to approach administrators, PTA members, students and others to ask "Why?" The question applied just as much to the affluent schools as the poor ones. Why are there effective, well-performing schools in the poorer demographics? Why are there ineffective, failing schools with affluent student populations and ample education budgets? The data we collected told us which schools to focus on for the answers to those and other questions.

Verify, verify, verify! The reporter and I sat down on several occasions, calculators in hand, to verify figures given us by school authorities. You'd be amazed how often the figures just don't add up.

Don't trust your computers, and don't trust theirs! If you can't verify everything, draw random samples and verify them. If you encounter errors, find the time to crunch everything the old-fashioned way.

This page was created on January 30, 1999, and was posted on the World Wide Web on February 7, 1999. It was reposted to the Web June 23, 2000.

Return to John Andrew Prime's home page..