Course evaluations flawed, unreliable

Ever since we were wee little schoolchildren with lunch pails
and Velcro buckle shoes, we students have been rated on our
academic performance. In truth, it’s an arduous path, wrought
with constant critiques and sometimes harsh judgments.

In the fourth grade, I misspelled “guard” (I still
don’t think the u should come before the a),
“receive” (no one had ever told me “i before e
except after c”), and “nonchalant” (exactly why
does a 9-year-old need to know that word). I ended up with a
C-minus on the test.

In the seventh grade, I wrote a haiku titled “Giant
Snowman,” and though I had exactly 17 syllables, somehow I
winded up with an 80 out of 100.

In high school, I failed many math tests. (I don’t really
have an excuse for this one other than stupidity.)

By the 10th grade, I was pretty tired of the constant judgment.
It angered me to think that teachers could rate their students
without ever being rated back.

So imagine my delight on the last day of class during my first
quarter at UCLA when I found out that professors are judged too,
and that content and methods used in future classes from beginning
to end might be based on my review.

For the first time, I filled out a course evaluation form. For
the first time, I graded my teacher.

That day, I embraced the saying “the pen is mightier than
the sword” and proceeded to laud the professors I liked and
rip apart the ones I hated.

Today, I’ve realized that exploited in a spiteful manner,
course evaluations are a waste of trees. However, even if you
aren’t drunk with newfound power, would the forms be
meaningful?

In evaluating the course evaluation, the verdict is out: It is a
flawed tool that does not accurately judge effective teaching.

In a study done by Wendy Williams and Stephen Ceci, they found
that the answers to the questions on course evaluation forms
“have more to do with style than substance.”

They came to this conclusion after Ceci, who is a professor at
Cornell University, took a teaching-skills workshop with a
professional media consultant. The consultant suggested that he
vary the pitch of his voice frequently and use hand gestures during
lectures.

After the workshop, Ceci taught the course exactly the same way
that he had taught it the semester before, but added more voice
modulation and hand gestures.

Though none of the content or methods had been changed, student
evaluations of Ceci improved significantly, increasing from a 3.08
to a 3.92 on a 1-5 scale.

Curiously, Ceci’s more enthusiastic manner was able to
color many other dimensions of his teaching that he had not
altered.

For instance, even the question “How knowledgeable is the
instructor?” elicited higher ratings, even though Ceci had
covered every idea and topic that he had covered the previous
semester, and not one thing more.

In the same vein, the higher ratings he received for the
question, “How organized is the instructor?” were also
unfounded. Students may have felt that Ceci was more organized, but
in truth, the exact same structure and even the exact same syllabus
were used.

Another question, which was “How accessible is the
instructor outside of class time?” also evoked higher ratings
despite the fact that Ceci kept identical office hours. Again,
students may have felt that Ceci was more accessible, but
that’s not what the question is asking. The question asked
how available the professor was outside of class, but the student
responses reflect more how welcome they felt in seeking him
out.

Even the textbook was rated a full point higher on the 1-5
scale.

But of course, who cares if students upped these ratings as long
as their learning improved, right? The thing is, their objective
learning did not improve.

Interestingly, in response to the question “How much did
you learn in this course?” student answers increased from a
mean of 2.93 to 4.05 ““ from average to very much.
Disturbingly, their perceptions were wrong, because scores on tests
and overall grades were virtually identical to those of the
previous semester.

It is important to note that all of the above mentioned
evaluation questions are similar to the forms used at UCLA.

Some corresponding questions on our forms are “To what
extent do you feel that class presentations were well prepared and
organized?” and “To what extent do you feel that
students felt welcome in seeking help in or outside of
class?”

Another flaw of course evaluations is statistical. Extreme
scores are more influential on the means of small-sized classes
than large ones. This means that the ratings for a professor who
teaches a group of 40 will be more susceptible to distortion caused
by an extreme score of 1 than a professor who teaches a group of
300.

Yet another problem with course evaluations is that they fail to
consider gender bias. Susan Basow, a psychology professor at
Lafayette College, has research that suggests that male students
give lower ratings to their female professors than to their male
professors.

This would certainly be a problem if a female instructor were to
teach any electrical engineering or computer science class at UCLA.
Last year, only 16 percent of first-year engineers were female.

What it boils down to is this: The course evaluation at its
present state is not a reliable tool for measuring teaching
effectiveness. Unfortunately, the form is unlikely to change, as
that would require a whopping amount of time and money, as well as
approval from the Academic Senate.

As students, there is a temporary Band-Aid that we can provide.
We can be more thoughtful when it comes to grading our teachers. Of
course, this won’t compensate for all the problems inherent
in course evaluations, but it can help.

Professors should try to make class as interesting as possible,
but when it comes down to the course evaluation, it’s
important to focus on what truly matters ““ how much they
taught you.

So, at the end of 10th week, I encourage you to take your new
privilege seriously. Remember that even if your professor
doesn’t speak in varying tones or gesture wildly, that
doesn’t mean that you didn’t learn just as much as if
your professor were Chris Rock.

While some professors will entertain us and other professors
will not, it’s important to keep in mind a teacher’s
primary objective. When filling out course evaluations, we should
remember that a professor’s ultimate intention is to
teach.

Contact Tao at atao@media.ucla.edu if you also have an
evaluation of the evaluation.

Leave a comment

Your email address will not be published. Required fields are marked *