Data-driven decisions about student progress and teacher effectiveness may not be embraced by all across the educational spectrum, but those who do think they are leading the way to improving educational achievement across the board.

Tim Sass is an economics professor at Florida State University who, for most of the past 17 years, has been conducting in-depth research on issues related to municipal governance, electoral procedures, minority representation and market trends. Recently, however, he switched his focus to education, and that, he said, is because Florida has an unprecedented wealth of publicly available data about each of its 168,000-plus teachers and nearly 2.7 million students.

He calls the availability of data “revolutionary.” Researchers can now access the kind of detailed information that has been commonplace in other sectors—the business world, for example—where split-second, data-driven ATM transactions and nationwide inventory decisions are made at the press of a button.

For Sass, this means Florida has an abundance of data on its students and an identification code for each one; it also has an abundance of data on its teachers and an identification code for each one of them. And it has a longitudinal accumulation of this information dating back to the 1995-1996 school year.

One result, said Sass, is that it is possible to track the academic performance of any given student as he or she goes through school, and that’s vastly different from comparing one year’s class of students in a particular grade to subsequent years of students in that grade. Therefore, he adds, you are controlling for a lot of things that can’t be measured, and in the process, eliminating the impact of outside factors such as race, ethnicity or socioeconomic status.

Sass’ first foray into education research focused on Florida’s charter schools. After sorting through the academic records of 53,000 students enrolled in the state’s 258 charter schools, he found that student performance on standardized tests was typically lower during the first few years of a charter school’s existence, but by the fifth year, it was on par with public schools in math and slightly ahead of them in reading.

Sass has since added teacher data to his research on education. In one study he is looking at the correlation between the educational background of teachers—e.g., the college they attended and their ACT or SAT scores—and student test scores. In another he’s asking, “Does it matter if all the kids in your class are bright, high achieving students? Or is it better to have a mix?”

“Economists are fairly new to this game, according to Sass. “It’s been in the last dozen years or so that economists have gotten involved in education research [and that’s because] it’s become more quantitative. It’s been a sea change in terms of education.”

Driven in part by technology and in part by requirements of the No Child Left Behind Act (NCLB), the sea change is having an impact at every level of the educational spectrum. Some are embracing the opportunities afforded by the availability of measurements that allow for tracking the cumulative achievement of individual students and the performance of specific teachers. Some are not. And those that are moving forward are doing so with varying degrees of speed and enthusiasm. The result of these various efforts is a patchwork of progress.

Implementing the infrastructure to slice and dice the academic success of school children in every conceivable way is not an easy task. It requires funding, qualified personnel, and leadership. “Using data to inform decision-making at every level changes how institutions have to function,” said Michele Cahill, Vice President for National Program Coordination and Director of Urban Education for Carnegie Corporation of New York. “It requires changing expectations, changing the thinking of management at the leadership level, developing assessments that accurately measure learning gains, developing information technology systems that can make this data available accurately and in a timely way, and building capacity at every level of schooling to use data well.”

Above all else, it requires the political will to deal with what the numbers might expose. Teacher and student identification codes, for example, allow researchers like Sass to hone in on the impact any given teacher has on his or her students. Modern day numbers
crunching allows for administrators to determine with more clarity than ever before which teachers are doing well and which ones are not.

Student IDs also allow schools to track students with greater precision through to graduation, thereby making it ever more possible to come up with reliable graduation and dropout rates. But not everyone either agrees that the data is measuring well what children are learning or how to use the data appropriately, especially if more accurate measures are used in a punitive way.

“There are schools that are using data as a liberator and as a catalyst for change,” said Timothy J. Magner, who directs the Office of Educational Technology at the U.S. Department of Education. “And there are other places that are using data simply as a sledgehammer to beat people harder, and you can see where that would be a huge disincentive for people to want to buy into those systems. Nobody likes to get beat with a bigger stick or a more accurate stick.”

That’s why it’s important for there to be transparency about what the data is capturing so the numbers do not become misleading shorthand for success or failure. For example, is success or failure for a school determined by the proportion of students achieving at or above state standards for their grade in school or by how well and how quickly students are progressing academically in that school?

In the first instance, schools in which large numbers of the students enter the school with educational advantages will likely do very well and schools where the majority of entering students are far from academically proficient will do poorly. Measuring progress, as in the second example, will allow the school with substantial academic progress to show this. This dilemma is at the heart of using data for accountability purposes. Schools must bring all students to standards of achievement if the students are to have the opportunity for success in life, so transparency about how far the school is from this achievement is essential. However, measuring progress both makes the challenges and gains more transparent and can expose important policy choices, such as where resources should be targeted. Without this level of sophistication, the emphasis on data and accountability tends to generate a tremendous amount of confusion, mistrust and cynicism about the use of data.

The impetus behind the current momentum in education data collection is twofold: the rapid pace of technological developments coupled with the onset of the standards and accountability movement, most notably the implementation of the No Child Left Behind Act (NCLB) of 2002. With NCLB, “The federal government created a huge expectation for data,” said Deborah Newby, Director of Data quality for the Council of Chief State School Officers.

It also shifted the focus from compliance, or simply making sure all the data cells were filled with the right information, to performance, meaning making sure that all students from every level—state, district, school and classroom—were meeting academic requirements. “Now performance and differentiating performance,” Magner said, “and bringing kids up who are falling behind as well as making sure that other kids can move ahead have really become much more a part of the conversation.”

Magner works with Ross C. Santy, Deputy Assistant Secretary for Data and Information. The Department of Education duo are now part of the policy and planning division and that, in and of itself, is indicative of the new status bestowed upon education data. “More and more folks are realizing this isn’t just an IT project,” Santy said, referring not only to the federal level but the state level as well.

“The states,” he said, “have really put a great amount of effort into improving their own data capacities in the past couple of years and are really moving fast.”

Some states like Florida and Texas started holding schools accountable for their performance on standardized tests long before it was mandated by NCLB. For years, they’ve been keeping track of student test scores on a systematic basis and shining the spotlight on various racial, ethnic and other subgroups. But for many states, these tabulations are relatively new.

A recently formed group called the Data Quality Campaign monitors each state’s ability to collect education data. Formed in 2005 and housed out of the National Center for Educational Accountability in Austin, Texas, it has a checklist of ten things each state should do if it wants to collect the kind of information that is needed to improve instruction at the classroom level and keep kids in school.

The Data Quality Campaign calls them the ten essential elements of a longitudinal data system, and they include a student ID; a teacher identification system that allows teachers to be matched with individual students; and student-level information that includes enrollment, demographics, performance on state assessments and college enrollment tests, graduation data, dropout data, courses taken and grades.

Educators often compare this trend to the kind of data collection that’s been taking place in the business world for years. “Wal-Mart makes decisions about what needs to be shipped to a store instantaneously,” Santy said, “and they’re basing that on a huge number of sales and inventory figures. So systems know how to do this.” Health care, a field more similar to education, is also being transformed by the use of data on mortality, survival and treatments in evaluating the results of drugs and other therapies as well as the performance of hospitals and doctors.

When it comes to technology, the education world is no different. The hardware and the software exist that would allow administrators, principals and teachers to look at test scores, attendance rates, graduation rates and other information at the click of a mouse. However, the IT systems are not yet in place nor the human capacity developed to make accurate data available and usable throughout school systems.

“We are just at the beginning of that transformation,” Magner said. “The availability of information systems, the availability of data, and the ubiquity of technology are creating the opportunity and also the challenge of entirely different modes of interaction—and the data system is a necessary foundation for that.”

The availability of data and the myriad possibilities it creates also comes with the need to build an entirely new value system around the data, Magner said, and that doesn’t happen overnight. “In our microwave society,” he said, “we expect that after firing up the software and dumping some data in it, the light is just going to come right out of the computer, fully formed from the head of Zeus or something.”

Once all the hard work of getting the data into the system is done, he said, “the even harder work is to understand what comes out of it, and whether we are really collecting the right stuff. And if we’re not, then we need to retool our system so that we are.”

Among the states, Florida is leading the way in terms of collecting education data. It is one of four that have completed all ten essential elements required by the Data Quality Campaign. And relatively speaking, it’s been in the data collection business for a very long time. It started putting a sophisticated system in place at least twenty years ago, long before it became a priority in most other states.

Among the districts, the New York City Department of Education is leading the way in terms of collecting and analyzing data on graduation and dropout rates that is more detailed, nuanced and revealing than in most other school systems around the country. In 2006, it conducted a thorough and unflinching probe of its dropout problem. With support from the Bill & Melinda Gates Foundation, New York City worked with the Parthenon Group, a consulting company with extensive quantitative capacity, to analyze the data on the hundreds of thousands of students who attended a New York City high school between 1999 and 2005.

Among the schools, Ware Elementary in western Kansas is one of many that have taken the lead in adopting diagnostic tests, or formative assessments, to gauge what their students are learning. At Ware, the principal implemented the tests on a routine basis well before the state adopted formative assessments as a standard practice.

Each of these places—Florida, New York City and Ware Elementary School—is a microcosm of what’s happening all around the country at the state and local level. At the national level, various governmental and nongovernmental efforts are underway to assist state and local school systems. The resulting progress is a wide range of advances, some taking place smoothly and some in fits and starts.