We finish up the last of our four part series (Part I, Part II, Part III) with “Giving Kids the Chaff; How to Find and Keep the Teachers We Need” by Marie Gryphon, Cato Policy Analysis No. 579, September 25, 2006. This entire paper is excellent and very much worth the time it will take you to read in full. As with all of Cato’s Policy Papers this is very easy to read and to the point. It’s less than ten pages if you take out graphs, footnotes, etc. However, we will paste some highlights below (we struggled with what important parts to leave out, not what to include).
Researchers who try to measure the impact that teacher quality has on student learning face profound difficulties. One difficulty is the presence of confounding factors: differences between student groups that have nothing to do with their teachers but can’t be neutralized using statistical techniques. For example, students attend schools not on a random basis but on the basis of neighborhoods that differ in terms of their socioeconomic and cultural character. Teachers do not choose schools randomly either. They tend to teach at schools close to their own homes, and more experienced teachers tend to teach in wealthier neighborhoods. Finally, students are not always randomly assigned to teachers; often they are placed in particular classes for nonrandom reasons such as parent request, student course selection, and academic record.
Economists Steven Rivkin, Eric Hanushek, and John Kain (the Rivkin Group) recently loosened this empirical logjam with a paper that analyzed an unusually comprehensive data set from Texas with a special methodology to measure the impact that teacher quality has on student performance.
The Rivkin Group looked at data collected over several years from multiple student cohorts at the same set of public schools in Texas. Comparing the educational outcomes of students in different classes at the same schools allowed the group to account for the concomitant effects of neighborhood and peer influence, socioeconomic status, the inherent qualities of the tested children, and different school management styles. Instead of attempting to determine which specific teacher attributes, such as experience and educational attainment, are important, the group simply sought to measure the effect that teachers have on student performance by calculating the variation in student performance among their students. The Rivkin Group’s work presents concrete evidence of the extent to which the best teachers can outperform the worst teachers under identical circumstances.
It turns out that parents have been right all along. Teachers matter a lot. “Results reveal large differences among teachers in their impacts on achievement and show that high quality instruction throughout primary school could substantially offset disadvantages associated with low socioeconomic background,” the group concludes. The group noted that good teachers matter more than smaller class sizes. Rivkin and his colleagues found that raising teacher quality by one standard deviation would improve student achievement more than a very expensive class-size reduction of 10 students per class.
Moreover, the Rivkin Group’s estimate of the teacher effect is probably too low, because it measures only the variation in the quality of teachers within schools. It is likely that there are large differences in teacher quality between schools as well, but the authors did not include between-school differences in teacher quality in their estimate because they could not measure them accurately. Hanushek believes that the true teacher effect is about twice as large as the Rivkin Group’s conservative estimate
After Rivkin and his colleagues found that teacher quality has a large effect on student achievement, they checked to see whether any specific, measurable attributes of teachers usually considered important—formal education, job experience, and academic ability—explained the teacher effect. Generally, they do not.
Like other researchers before them, Rivkin and his colleagues found that possession of a master’s degree makes no difference in teacher effectiveness—though it is nevertheless one of the principal determinants of public school teachers’ salaries today. Job experience improves teacher performance, but only for the first three or four years of teaching. A teacher with 10 years of experience is no more effective, on average, than a teacher with 5 years of experience.
The Rivkin Group did find that a teacher’s effectiveness in the classroom is related to high standardized test scores in high school and college. That is consistent with prior findings. Teachers’ scores can explain only a portion of the large difference in achievement that Rivkin and his colleagues attribute to teacher quality, but they remain the most useful proxy available for determining the overall quality of a school’s teaching staff. In addition, subject-matter competence is thought to be particularly important in math. While the Rivkin Group did not address math preparation specifically, other research suggests that teachers with good subject-matter knowledge in math, as measured by tests, are more effective math teachers.
All in all, these findings mean that most of the teacher effect remains a mystery. Outstanding teachers may have attributes that are very difficult to measure, such as diligence, charisma, and a love of teaching. The large differences in teacher quality that exist within schools suggest that those who hire teachers, such as principals, either can’t tell in advance which job applicants will be good teachers or are making hiring decisions on the basis of criteria other than teaching quality.
But a long line of initiatives designed to raise overall teacher quality, including large indiscriminate pay increases, elaborate teacher training regimes, and burdensome certification requirements, has failed to improve student outcomes. Moreover, in terms of the measurable characteristics that seem to matter—test scores and math and science preparation—teacher quality has actually declined in the past three decades.
The teacher quality problem is difficult to solve in part because of entrenched public school personnel practices. First, public schools systematically fail to hire the best applicants for teaching jobs. Second, they adopt compressed pay scales that entice low-ability workers while driving higher-ability workers away. Finally, they over compensate experienced teachers with funds that could be better used to lure teachers with in-demand math and science skills.
Part of the conventional wisdom about the teaching crisis is that the United States has too few willing teachers. But most measures suggest that the opposite is true, that the United States has a surplus of teaching applicants for a limited number of available positions.
In part, poor teacher quality is due to a weakness in the applicant pool as a whole. But the problem of poor teacher quality is also exacerbated by the perverse hiring practices of principals and school district administrators. Research shows that those gatekeepers systematically fail to hire the most capable applicants.
Economist Dale Ballou found that administrators were no more likely to hire high-ability teaching candidates than candidates of lower tested ability. That was the case despite substantial evidence that higher tested ability of teachers is one of the most reliable indicators of superior classroom performance. Ballou himself is not sure why school administrators hire the way they do.
Ballou also found that administrators were more likely to hire applicants who majored in education than applicants who majored in math or science, even when both were eligible to be employed in public schools, despite a recognized national shortage of teachers who bring those subject specific skills to the classroom.
As teachers’ unions gained size and strength during the latter half of the 20th century, teacher pay became increasingly uniform among individuals with similar levels of formal education and classroom experience. This uniformity, known to economists as “pay compression,” creates incentives for low-achieving graduates to enter the teaching profession while deterring their highly capable counterparts.
Pay compression dissuades the best potential teachers from entering the profession because their alternative career options will be more enticing than those of their less-capable counterparts, but they will be offered no additional compensation to teach. Because the public schools don’t reward them for their merit, potentially excellent teachers choose to work elsewhere.
If pay compression is responsible for low teacher aptitude, only a system of differential pay that offers unique rewards to highly capable teachers will reverse the trend. If alternative opportunities for women disproportionately drew highly capable women from teacher applicant pools, then across-the-board raises for teachers might improve average teacher quality.
Teacher attrition is widely considered a costly crisis in American education. People believe that attrition reduces quality because they assume that more experienced teachers are more effective in the classroom.
It is the character of teacher attrition, not the amount of attrition, which lowers the quality of teaching in public schools. Higher-ability teachers tend to leave the profession. Lower-ability teachers tend to stay in order to obtain the relatively steep salary increases associated with seniority. Higher-ability teachers are more likely to leave teaching in favor of nonteaching professions where their skills are apt to bring greater compensation.
While the conventional wisdom is that attrition is “too high,” attrition is actually too low among the least-capable teachers. Hanushek observes that a policy change that simultaneously caused the best teachers to teach for two years longer and caused the worst teachers to quit two years sooner would substantially improve average public school teacher quality.
Hanushek has found that teachers reach full effectiveness after four years. Beyond that point, experience, per se, is not associated with student achievement gains; a 15 year classroom veteran will produce the same results, on average, as a teacher with 5 years of experience. But the more senior teacher is paid far more for his services solely as a function of seniority, because union-negotiated pay scales systematically over compensate teachers with many years of experience. That salary pattern is not replicated in private schools, where teachers receive little seniority pay after the first few years.
Across-the-board salary increases are a favorite teachers’ union remedy for the problem of poor teacher quality.
But Ballou shows that wholesale pay hikes may actually have the opposite effect. High salaries can reduce teacher quality if administrators are making poor hiring decisions: “Drawing more applicants into a recruitment process that does not screen well may only make matters worse, particularly if the career choices of better candidates are more sensitive to changes in the probability of getting a job,” Ballou explains.
In a poorly screened system, in which applicants are chosen without regard to ability, the most capable candidate is no more likely to be hired than any other applicant. Therefore, if raising salaries by 10 percent increased the number of applicants by one-third, the high-ability teacher’s expected value from teaching would actually go down, because the probability of not being hired in that school year would more than cancel out the benefit of the higher salary.
Ballou’s research suggests that teaching is a very poorly screened labor market, a finding shared by other researchers. Therefore, across-the-board salary increases—without dramatic changes in hiring practices—are unlikely to raise teacher quality much, and might actually lower it.
Across-the-board salary increases can also slow the pace of reform by reducing teacher turnover. Higher salaries reduce attrition, and Ballou’s work suggests that lower-quality teachers stay in their jobs the longest. It is more difficult to raise the quality of teaching through new hiring practices if fewer open positions are available to fill.
Merit Pay Isn’t Enough Merit pay is an increasingly popular idea for reforming public schools. The theory is simple and sound: rewarding better teaching will result in better teachers, both because all teachers will try harder and because more capable potential teachers will expect to be compensated for their skills. It seems to work in the private sector.
But rewarding merit in the public sector is very difficult. One problem with a highly regimented merit pay system is that it yields exactly what it measures and no more. These highly targeted improvements often come at the expense of other important aspects of educational quality. Economist Randall Eberts and his associates studied the effects of a merit pay system in a Michigan high school in 1996. The high school studied sought to reduce dropout rates by financially rewarding teachers for keeping more students enrolled in their courses each semester. Although course retention rates went up as a result of the new system, evidence suggests that little, if any, additional learning took place. Average daily attendance rates, test performance, and course passage rates all declined after the retention policy took effect. “Merit” is a difficult thing to quantify. Regimented programs like the one in Michigan are intended to reward it, but they too often generate more of what they happen to measure—retention, in this case—without achieving the underlying goal of increasing learning.
Moreover, merit pay programs are designed through a political process that is heavily influenced by the teachers’ unions. As a result, past and present merit programs are designed to improve performance exclusively by raising teacher effort despite evidence that teacher preparation regimes are expensive and ineffective. In other words, without better training, increasing effort will not necessarily improve results. To truly raise teacher quality, a merit system would have to change the character of the teacher labor force by attracting more qualified new hires and removing lower-quality teachers from their jobs.
School choice is not frequently suggested as a way to change the makeup of the teaching profession. Too often, debates about choice envision reallocating students among existing schools with existing personnel. But research suggests that private and charter schools have very different hiring practices than do traditional public schools. The introduction of market forces on a broader scale could thus change the way teachers are chosen and compensated, transforming the teaching profession by attracting new and different workers.
To show that school choice can change teaching in a positive way, advocates must offer evidence that school administrators respond to competition by hiring better teachers. Demonstrating that this is true is hard for many of the same reasons that it is hard to show that teachers matter.
Once the Rivkin Group had found a way to show that teachers matter, Hanushek and Rivkin turned their attention to whether they could use similar methods to determine whether competition increases teacher quality. Using the same data set they used as part of the earlier Rivkin Group study, they compared the within-school variation in teacher quality in districts that were subject to substantial competitive pressures (due to Tiebout choice) with the quality variation within schools in less-competitive districts. Their approach builds directly on Ballou’s finding, discussed earlier in this paper, that public school administrators do not systematically prefer job applicants with attributes (such as high test scores or a math degree) that we know are related to teaching performance. Ballou reasoned that if administrators don’t prefer to hire teachers with those qualities, they probably also will not prefer to hire teachers with more-difficult-to-measure performance-enhancing characteristics. That would explain why the variation in teacher quality measured by the Rivkin Group is so large.
Hanushek and Rivkin examined the link between competition and the variance in teacher quality with the idea that competition should, if Ballou is right, lead to less variance in teacher quality. That would occur because administrators responsible for hiring and retaining teachers would respond to competitive pressures by hiring applicants likely to be high performing, rather than hiring on the basis of attributes loosely related or unrelated to performance, such as popularity among fellow teachers.
Competition affects public school personnel practices significantly, the researchers found. Greater competitive pressures were systematically related to a smaller variation in teacher quality, suggesting that administrators in competitive districts gave quality higher priority in their hiring and retention processes.
Competition improves teacher quality the most in school districts that serve large numbers of low-income students. Hanushek and Rivkin found that improvements were strongest in schools in which 75 percent or more of the students had family incomes low enough to qualify them for subsidized lunches. Policies that increase competition should therefore reduce the current educational disparities between wealthy and poor students.
Hanushek and Rivkin’s work on competition and teacher quality is useful because, like the Rivkin Group’s work, it captures and measures changes in the presence of the “teacher spark”—those hard-to-measure attributes of great teachers. But as does the literature relating teacher attributes and student achievement, the research relating competition and teacher characteristics provides some more specific clues about school hiring practices and priorities.
Harvard’s Caroline Hoxby analyzed survey data from public, private, and charter schools to find specific ways in which the presence of choice changes the characteristics of teachers. She found that schools subjected to competition hire more teachers who have the specific qualities that have been tied to performance by past research: high tested ability and experience with math and science.
Public schools subject to the highest level of public school, or Tiebout, competition, had teachers from colleges whose average SAT scores were 4.391 percentiles higher than the alma maters of teachers in less-competitive districts. Teachers in competitive schools were also more likely than others to have majored in math and science or to have taken significant coursework in those subjects.
Personnel policies of charter schools reflect similar preferences. About 30 percent of charter schools offer higher pay to teachers with expertise in hard-to-staff subject areas such as math and science. Teachers at charter schools are even more likely than their private school counterparts to hail from colleges with high average standardized test scores, indicating relatively high tested ability. Charter schools were also more likely than traditional public schools to consider salary at a previous nonteaching job and evidence of superior performance when determining compensation.
A seemingly magical property of private and charter schools is their ability to simultaneously keep teaching quality high and student/teacher ratios low, all while spending less on salaries per teacher than the public system. Recent research points to a likely explanation for this seeming impossibility: these schools are less likely to offer their teachers annual raises based solely on years of experience. By refusing to provide large raises solely on the basis of seniority, private and charter schools maintain a cheaper, younger but highly capable workforce.
Because private and charter schools do not reward seniority as richly as public schools do, they have more resources available to reward high-performing teachers. Hoxby argues that increased school choice would cause less skilled and less-motivated incumbent teachers to receive smaller raises than many of their colleagues. As a result, she suggests, they would be more likely to quit, thus reversing the current, unfortunate pattern of higher-ability teachers exiting the profession, leaving behind their weaker colleagues. Under a system of widespread school choice, the demographics of public school teachers would more closely track those of their counterparts in schools already exposed to substantial competitive pressures.