Austin Tedman read his classmate’s essay carefully and began meticulously grading the paper. Although the program doesn’t require it, he typed out a carefully worded justification for each one of his assessments.
Then, the program crashed.
He tried again, reconstructing his previous responses to assure his grades were fair and thorough.
The program crashed a second time.
“I try to write meaningful comments and then it crashes – at that point, you just want to get it over with and you don’t want to keep writing,” said Tedman, a third-year biochemistry student.
UCLA created Calibrated Peer Review in the 1990s to teach students in large classes how to write a scientific paper by making grading essays easier for teaching assistants and professors. However, many students do not think the program helps to improve their writing abilities or give them a deeper understanding of the content.
“I think the program was a pretty decent intro into scientific writing, but it didn’t allow for much creativity,” said Hannah Mark, a third-year undeclared physical sciences student. “You kind of check these things off a list.”
Students submit their written assignment to the program, grade three peer-written responses by answering multiple choice questions about the assignment – like “Does the essay have a descriptive topic sentence?” – and grade their own assignment according to the same criteria.
The instructor assesses the grades students receive from their peers and uses this to help determine their overall grade on that assignment. Writing the assignment is only a small part of the final grade, which is also based on whether or not they go through “training.” During training, they learn how to grade assignments by reading and practicing on past assignments. Their grade is also based on how closely their assessments match the assessments of other students.
Arlene Russell, a senior lecturer in the Department of Chemistry and Biochemistry and director of CPR at UCLA, said that while writing is essential to scientific education, assessment is often impractical in large classes. Adding writing assignments to a course could require twice as many teaching assistants. CPR is a way for students to learn how to write without having extra staff, Russell said.
But for many students, this program does little to teach scientific writing.
“You’re asking people who don’t know how to write to grade other people who don’t know to write,” Tedman said. “It’s the blind leading the blind – of course it’s not going to work.”
Tedman said he also thinks the grading criteria on CPR are often arbitrary and give little incentive to write well.
“CPR turns everything into a checklist. ‘Do you have a hypothesis? Do you have a conclusion?’ et cetera. … There is no room for quality,” he said. “It’s ‘Do you have a hypothesis?’ not ‘Do you have a good hypothesis?’”
Russell, who is also a senior lecturer in the Department of Education said the program is less about improving writing quality and more about learning the science material more in-depth. That is why a student’s essay usually constitutes less than 30 percent of the final grade while the remaining score is based on how well a student assesses themselves and others.
“Writing to learn and learning to write are not the same,” she said. “CPR is really focusing on writing to learn in the act of articulating and learning ideas.”
Julia Nakamura, a third-year psychobiology student, said she does not think CPR helped her improve her writing skills or learn the material better.
“They made it so clear exactly what you were supposed to include,” she said. “All you had to do was put in the information they specifically asked for in order to get a good grade.”
She also said she understands why they use the program, as TAs and professors often do not have time to grade papers themselves.
Gaston Pfluegl, a lab director in the Life Sciences Core Education department who uses CPR in his classes, said he typically has 800 students and only 12 TAs. Before CPR, some TAs would give short and undetailed feedback because they had to grade a large volume of papers. He said he thinks CPR gives students more constructive feedback.
“Students are always a little confused because (CPR) is a complicated thing,” he said. “But I think it’s worth doing it.”