Tag Archives: Value Added Model

Department of Education’s New “Report Card” for Teachers

For the first time a pilot program by the Ohio Department of Education is ranking the abilities of individual teachers in what is being called “a landmark measure in the school reform battle.” This new “grade card” which was recently issued for some of Ohio’s public school teachers supposedly shows which educators made a measurable difference in the classroom last year. 

Currently, the reports are only for about 30% of Ohio’s reading and math teachers who teach fourth through eighth grade, which means that about 7,500 teachers received a report with details as to what effect they had on their students’ learning last year. Thirteen Ohio districts and two charter schools were participants in this first round of effectiveness reporting. Using the “value-added measure,” these reports link data regarding student growth to the teachers who worked with these students.

The value-added system, which has been rejected by many analysts who have studied it, determines how much growth students have shown within a school year after first determining where they started. It is considered to be an “equalizer” because it assumes teachers will make progress with each student no matter what their ability level might be. 

In spite of the fact that unions have opposed using student data to judge teacher effectiveness, Ohio is required, under the federal Race to the Top initiative, to change the way principals and teachers are evaluated. Additionally, the state budget bill now requires Ohio’s Education Department to devise an evaluation tool that would base half of a teacher’s job evaluation on data regarding student growth.

The Columbus Dispatch pointed out the following implications of this new evaluation system:
* Students could be assigned to classrooms based on teachers’ abilities — by placing low-, middle- or high-performing students with the educators best able to help them learn. Data showing the effect that teachers had with different types of students are included in each report.
* It will distinguish good teachers from great ones, and mediocre ones from good ones.
* Over time, schools will use the effectiveness ratings to weed out teachers who aren’t making the grade.

Matt Cohen, who oversees policy and accountability at the Ohio Department of Education, said, “This will help confirm good teaching. It will help identify in an objective way some of the issues that people are very uncomfortable about in terms of trying to characterize poor teaching from average teaching.”

Interestingly, officials who helped produce the new ratings said that they shouldn’t be used as a way of labeling teachers either good or bad. Mary Peters, senior director of research and innovation at Battell for Kids, a Columbus-based nonprofit organization helping the Department of Education to develop the evaluation system, said that this year’s rating is nothing more than a statement of a teacher’s effectiveness with their students for last year.  

“We need to be careful about making judgments about one year of data. These measures were intended for diagnostic purposes, to provide information to help teachers reflect on their practice and determine with whom they are being successful,” Peters explained.

And while officials agree that the data should primarily be used to improve schools, as more years of data becomes available, they admit that teachers consistently earning “least effective” ratings will be scrutinized closely by their administrators.

Cohen admitted as much when he said, “Our hope, anyway, is that what you end up with is a better work force. And when you do have teachers who are really consistently doing poorly with results for kids, that they might not belong there.”

Rhonda Johnson, president of the Columbus teachers union, one of the districts that were included in this first wave of evaluations, said that teachers in Columbus already use data to help determine how much they are accomplishing with their students, (as I think most school systems do) but she stated that valued-added data should not be the only tool in judging teacher effectiveness.

“It doesn’t tell the whole picture. This is only a fraction,” Johnson said.

I am not necessarily opposed to long-term analyzing of a teacher’s ability to get most students to show a year’s progress each year, with all students starting at a variety of different levels. I see some inherent problems however, that will most definitely need to be addressed along the way to avoid misusing this effectiveness system.

First, I do not feel that the same criterion should be used to judge progress for students on IEPs, as they tend to progress at a slower rate. If that is not taken into consideration with this new evaluation system, very few brave souls will volunteer to work with these students, which would be a travesty.

Second, administrators must be open to looking at mitigating circumstances which may have affected student growth on a class-by-class basis in a given year. Anyone who has been a teacher for any length of time knows that there are some years that you remember with a shudder, when your classroom seemed to be the dumping ground for so many behavior and/or academic issues that instruction was a constant battle. In situations such as these, administrators must look beyond data to see the reality that teacher faced in his or her classroom.

Third, I fear that some administrators will misuse these reports to condone the firing of teachers who may have had one bad year, or who could have become more effective with proper mentoring and guidance.

Finally, I worry that this information, in the hands of the media, will be used to vilify teachers and hold them up to public scrutiny and ridicule. We all remember what happened when the L.A. Times published their article ranking teachers and 39-year old Rigoberto Ruelas, Jr. committed suicide shortly after receiving a less effective ranking based on his students’ English and math scores.

So, what do you think? Are these “report cards” a good idea, or do you predict problems?

Diane Ravitch Weighs in on Wisconsin and Teachers’ Rage

I have written to you about Diane Ravitch before; a powerful woman well-grounded in education and one-time staunch supporter of No Child Left Behind but now just as staunchly opposed to all that No Child Left Behind stands for. Well, my respect for this woman has grown as she has now spoken out about what is happening in Wisconsin. Here are some of her salient points.

First, she accuses conservative Republican governors like Chris Christie of New Jersey, John Kasich of Ohio, Mitch Daniels of Indiana, Rick Scott of Florida, and Scott Walker of Wisconsin of wanting “to sap the power of public employee unions, especially the teachers’ union, since public education is the single biggest expenditure for every state.”

Thousands of public sector workers have camped out in Wisconsin protesting Walker’s plan to reduce their take-home pay by increasing the amount they will have to contribute to their pension plan and their health care benefits at the same time that they plan to restrict their collective bargaining rights. Walker claims these cutbacks had to be imposed because the state is broke, but, Ravitch claims, “Teachers noticed that he offered generous tax breaks to businesses that were equivalent to the value of their givebacks.”

Ravitch goes on to enumerate the cause of the “simmering rage” felt by the nation’s teachers. “They have grown angry and demoralized over the past two years as attacks on their profession escalated. The much-publicized film Waiting for ‘Superman made the specious claim that ‘bad teachers’ caused low student test scores. A Newsweek cover last year proposed that the key to saving American education was firing bad teachers.”

Following this was the outrage felt by teachers everywhere when the leaders of the Central Falls School District in Rhode Island threatened to fire the whole staff of the town’s only high school due to poor performance on test scores. She points out that what really concerned teachers when they heard this news was the positive way it was received by both the Secretary of Education Arne Duncan and President Obama, who both felt it was a fine idea, even though there had been no evaluations of anyone at the high school.

Ravitch points to the Race to the Top program as another way that teachers have felt under attack. “The Obama administration’s Race to the Top program intensified the demonizing of teachers, because it encouraged states to evaluate teachers in relation to student scores. There are many reasons why students do well or poorly on tests, and teachers felt they were being unfairly blamed when students got low scores, while the crucial role of families and the students themselves was overlooked,” wrote Ravitch.

Finally, she points to the despair teachers felt in August when we read about the outrageous report in the Los Angeles Times in which this paper rated 6,000 teachers in Los Angeles as either effective or ineffective using the Value Added Model and students’ test scores. As you recall, the publishing of these ratings online led to the apparent suicide of one of these teachers who was rated ineffective, in spite of his consistently good evaluations. But as Ravitch points out, “Testing experts warn that such ratings are likely to be both inaccurate and unstable, but the Times stood by its analysis.”

Now, teachers are facing the latest and most demoralizing attack of all, the plan to abolish our right to due process, our seniority, and, in some states, the loss of collective bargaining rights. Ironically, Ravitch points out, “Actually, the states with the highest performance on national tests are Massachusetts, Connecticut, New Jersey, Vermont, and New Hampshire, where teachers belong to unions that bargain collectively for their members.”

She points out that the reason conservative governors want to reduce the power of the unions is because they actively lobby to increase funding for education while reducing class size. If they can shut down teachers unions they also shut down the biggest opposition to making cuts in education.

Ravitch eloquently and masterfully summarizes what those of us in public education are feeling when she concludes: “There has recently been a national furor about school reform. One must wonder how it is possible to talk of improving schools while cutting funding, demoralizing teachers, cutting scholarships to college, and increasing class sizes. The real story in Madison is not just about unions trying to protect their members’ hard-won rights. It is about teachers who are fed up with attacks on their profession. As the attacks on teachers increase and as layoffs grow, there are likely to be more protests like the one that has mobilized teachers and their allies and immobilized the Wisconsin Legislature. “

Value Added Model for Evaluating Teachers is “Junk Science”

Let’s talk about the Value Added Model or VAM, the newest method being pushed to evaluate teachers. It is based on the theory that a teacher’s effectiveness can be judged by measuring the progress that teacher’s students make on standardized tests over the course of the year. It is being suggested that this be the measure with which teachers are retained or let go. It’s the same method of evaluating teachers which was printed in Los Angeles by the L.A. Times, and we all know how that worked out.

Here’s the problem with VAM, according to neatoday’s January/February magazine: “Every respected, independent testing expert in the country agrees that VAM is not a valid or reliable measure for making high-stakes decisions about teacher effectiveness. It is junk science.” And it gives these examples to support their findings:

* The Board on Testing Assessment wrote an open letter to Department of Education Secretary Arne Duncan in October, 2009, stating that not enough research had been done on VAM’s validity to use it as a basis for determining teacher effectiveness. It also concluded that a student’s scores can be affected by various factors other than their teacher.
* The Department of Education’s Institute of Education Sciences published a 36-page analysis of VAM in July, 2010, in which it stated that “more than 90 percent of the variation in student gain scores is due to the variation in student-level factors that are not under the control of the teacher.”
* The Economic Policy Institute (EPI) assembled a panel of experts in August, 2010, who warned against giving substantial weight to VAM scores as a tool for measuring teacher effectiveness.
* Researchers for the RAND Corporation, a nonprofit institution that helps improve policy and decision making through research and analysis, concluded, “The research base is currently insufficient to support the use of VAM for high-stakes decisions about individual teachers.”
* And in the wake of the L.A. Times debacle, Rutgers Professor Bruce Baker concluded, after an analysis of the study, that its ratings of teachers are racially biased. He cited that the lowest VAM scores were earned by black teachers while the highest were earned by Asian teachers.

Additionally, a policy letter drafted by ten prominent education scholars, including Eva L. Baker, a UCLA professor and co-director of the National Center for Evaluation Standards and Student Testing, and Paul Barton, former director of the Policy Information Center of the Educational Testing Service; Stanford Professor Linda Darling-Hammond; education historian and New York University Professor Diane Ravitch; and several other prominent figures, warns of the danger of linking teacher evaluations to standardized tests. In this letter, they state: “Too many policymakers have recently adopted the misguided belief that improvements in students’ scores on standardized tests in mathematics and reading can be heavily relied upon to evaluate, reward, and remove the teachers of these tested students. However, even the most sophisticated use of test scores, value added modeling (VAM), is a flawed and inaccurate way to judge whether teachers are effective or ineffective.” They go on to cite evidence from recent research that concluded that VAM was too inaccurate to be used as the primary means of evaluating teachers.

Which brings us full-circle in asking whether VAM scores can be used as a reason for firing teachers. First neatoday states that tenured teachers cannot be fired without “just cause”, and using VAM as that “just cause” is not appropriate when its value has not been proven. Secondly, it explains that if Professor Bruce Baker is correct that VAM is racially disparate, a study would have to exist that proves the validity of using VAM as an accurate measure of teacher effectiveness, and again, no such proof exists.

As part of its report, EPI experts cited these two concerns about the use of VAM:

* It will likely lead to “expensive…litigation in which experts will be called to testify, making the district unlikely to prevail.”
* And it is also likely to “demoralize teachers.”

Shortly after the L.A. Times published its list of the most effective to least effective teachers in local schools in Los Angeles, Rigoberto Ruelas Jr., a fifth grade-teacher who was rated less effective according to the Value Added Model, in spite of his “great performance rating” at his school, jumped off a bridge to his death.

The awful tragedy is that a performance measure was used publicly here to demoralize teachers; a measure which has never been proven to be an accurate gauge of a teacher’s effectiveness, a measure which has actually been discounted by reliable sources, including one directly affiliated with the Education Department.  Which can’t help but leave teachers fearful of the future since proposed House Bill 21 is asking for Value-Added Data to be a requirement for renewing licensure which means using VAM as an evaluation tool.

Teachers need to get vocal with their legislators and demand that they vote this bill down, or we all will be held to a standard that has not even been proven to be a fair or accurate evaluative tool. Go to this site and sign a letter against VAM, but speak up now, before it is too late!