Lisa Murphy's research shows a link between income levels and school performance. Photo: Matt Suby
Lisa Murphy didn’t intend for her doctoral dissertation to focus on a political issue. After all, as a geographer, Murphy studies maps — albeit complex ones — and what could be more apolitical than that? But, using maps to analyze standardized testing is political, considering that the subject of standardized testing can’t be raised in Colorado without someone getting hot under the collar.
For the last two years, Murphy, BA social sciences ’90, MA anthropology ’97, has been using Geographic Information Systems (GIS) mapping technology to study correlations between neighborhood demographics and Colorado standardized test scores. Her aim was to identify social factors that might explain why students in some schools perform well on tests while others don’t.
In Colorado public schools, standardized testing is conducted through the Colorado Student Assessment Program (CSAP), which measures student progress in reading, writing, science and arithmetic. In 2003-04, only 115 of the 182 districts in the state made “adequate yearly progress” toward proficiency in reading and math. For parents and educators alike, results like these are frightening enough. But add federal funding pressure to the mix, and the subject of standardized testing can become downright explosive.
The 2001 No Child Left Behind Act requires that all students in grades 3-8 be proficient in reading and arithmetic by 2014. The act also requires that schools have “highly qualified” teachers in all core subjects by the end of the 2005-06 school year, or 2007 for rural areas. If schools don’t meet the act’s requirements, they are placed on probation until scores improve. Schools that don’t improve lose federal funding.
CSAP scores can vary widely between districts, neighborhoods and schools, and debate has raged for years about whether or not the test provides an accurate measurement of student and school performance. And, no one seems to agree on whether teachers, schools, administrators, parents or the students themselves should bear the blame for poor scores. Some believe that social factors strongly impact school performance, while others believe that educational success is independent of demographics.
A mother of four, Murphy originally sent two of her children to public schools before deciding that they all would have greater success in private academies. But, the perennial debate about CSAP got her thinking. Are schools and teachers primarily responsible for poor student performance? Her background in anthropology and political science had taught her that demographic qualities have social ramifications, and she suspected that demographics impact the CSAP, too.
“You can have good and bad teachers, sure, but there are other things that influence test scores,” says Murphy, who is an adjunct professor of GIS at DU’s University College.
Murphy challenged the theory that educational success is independent of social factors, hypothesizing that school scores are indeed influenced by the social characteristics defining the neighborhoods where students live. She set out to identify factors impacting CSAP scores and to learn whether factors at the regional, district or school level were more influential.
“I wondered, does everyone have the same chances, or does social equity need to be improved?” Murphy says.
Looking for correlations
To conduct her research and test her hypothesis, Murphy used GIS technology to view and analyze data from a geographic perspective. GIS layers information on a map, enabling users to model scenarios, test hypotheses and see how individual variables interrelate.
Murphy began by poring through 2000 Colorado census data and the 2001 and 2002 CSAP scores for fourth-, seventh- and 10th-grade students throughout the state.
She then mapped property values, the age of homes, whether housing in an area was occupied, family size and whether households had a single parent or married parents. Murphy also created map layers based on race, language, age, educational attainment, birth countries, income and the percentage of the population in the workforce.
At the school level, Murphy mapped out enrollment, the percentage of free lunches, teacher salaries, discipline reports, the amount of teaching experience and student-to-teacher ratios. She also added attendance boundaries and the distribution of CSAP scores for the state. Overall, she included 40 variables in her research.
Murphy then looked for correlations at the regional, district and school levels. At the regional level (the Colorado Department of Education divides the state into five regions based on population), Murphy found some significant correlations between demographic variables and test scores.
But, when Murphy looked at the district levels, she found even stronger correlations between test scores and six demographic variables. The percentage of free lunches, low educational attainment, the percentage of Spanish speakers and the percentage of Hispanic population showed strong correlations with lower test scores, while the percentage of white population and married-family households were correlated with higher test scores.
More intriguing still, when Murphy examined the school-level data, a single variable jumped out: free lunches.
Throughout Colorado, Murphy found that the percentage of free lunches accounted for 81 percent of the variability in CSAP scores from one school to another. And free lunches, she explains, are a good measure of a community’s socio-economic standing.
“I suspected it would be income related,” Murphy says. “But it’s indisputable — it’s clearly related. That was overwhelming.”
“Most people ‘in the know’ knew the basic gist of Lisa’s research even before her results were confirmed. Poor children do not fare as well on standardized exams as their middleclass and wealthy counterparts,” says Geography Prof. Paul Sutton, Murphy’s PhD adviser. “Lisa confirmed this point for vast areas of Colorado. It is important to repeat and reiterate this point because the public is not very aware of this fact.”
To test the validity of her findings, Murphy used free-lunch data to try to predict CSAP performance for a sample of 240 elementary, middle and high schools. Using an equation she created, Murphy accurately predicted the CSAP scores for 87 percent of the schools.
“I didn’t even need a computer,” Murphy says. “I could almost guess the scores by looking at the percent of free lunches.”
However, it’s the predictions she missed that interest Murphy the most. Some schools did much better than she predicted, and others did much worse, which makes her wonder why.
“Now that we know which schools are doing better despite the percent of free lunches, researchers can go to those schools and explore why they don’t fit the model,” Murphy says.
Using the data to make a difference
Murphy isn’t the first researcher to find a strong correlation between income and CSAP results. In 2003, DU’s Graduate School of Social Work and College of Education commissioned noted sociologist David Rusk to study the CSAP issue. He found a high correlation between a school’s socioeconomic profile and its students’ CSAP scores. He also found that on average, Denver’s neighborhoods and schools are more racially and economically segregated than many comparable American metropolitan areas.
“Educators have known about the correlation between a child’s economic status and a academic performance. Research has shown this over and over again,” confirms Ginger Maloney, dean of the College of Education.
Murphy notes that her findings could have important policy implications because they point out a potential flaw in education legislation. “Education policy initiatives are based on what is happening at the district level or higher,” she explains. “I’m showing that if you don’t address achievement problems at the school level, you lose all the kids that are truly left behind.”
Murphy’s research culminated in the successful defense of her dissertation in May; she graduated with a PhD in geography in June. She has presented her research at DU and at a recent conference of the Association for American Geographers.
Although Murphy doesn’t plan to continue her CSAP research, she would like to see similar analysis applied broadly and become part of the school-reform debate. “I would like to give this information to the Denver and Boulder school districts,” Murphy says. “I’d like Governor Owens to read it as well.”
“It is controversial material, and some of the conclusions one must come to regarding these results raise deep and troubling questions about the nature of public education in the country today,” Sutton says. “Lisa’s work should help Colorado in general, but that all depends on who pays attention to it.”
Regardless of the outcome, Murphy is proud of the work she’s done. “Even if you disagree with me, that’s okay,” she says. “If people are talking about it and asking questions, this has been successful.”