Blogs > Cliopatria > Norming the grades

Jan 5, 2005

Norming the grades




I'm home from England (thanks, Ralph, for the link), where I spent some time talking with my brother (a lecturer in English Lit at Exeter) about the differences between American and English universities.

One difference that struck me as remarkable was the lack of autonomy my brother and his colleagues have over grading. In order to make sure that there is a uniform (or near uniform) standard of grading, my brother and his colleagues read samples of each other's students' papers. The objective is to ensure that all professors are giving similar marks for the same sort of work; in other words, it is expected that there will be widespread agreement about what constitutes a "first" or a "second" (in this country, an A or a B are the rough equivalents.)

When I was a teaching assistant at UCLA in the early 90s, I had one professor who asked her TAs to swap graded exams in the hopes of achieving a normative standard of what constituted an A. I can remember this leading to some real arguments! Most of the disagreements were over slight differences (such as a B- or a C+); in a few instances, however, we had TAs who disagreed as to the merit of a given paper by well over a whole grade's difference.

In my eleven years of community college teaching, I've never reviewed a colleague's grading, or had my own examined. I admitI would bristle if I were asked to have my grading supervised! I'm curious to know if any of my American colleagues, particularly those who have been teaching for a while, "norm" their grades with other faculty in their same discipline. If so, is this voluntary, or, as at Exeter, mandated by the institution? Does this sound like a useful practice?


comments powered by Disqus

More Comments:


Hugo Schwyzer - 1/6/2005

Excellent link, Jonathan; thank you!


Sharon Howard - 1/6/2005

And I can add that aside from internal second marking (I take it that's what's being described at Exeter? It's certainly not unusual), it's normal in Britain to have external examiners with this sort of supervisory role, and who have the final say on marks awarded.


Jonathan Dresner - 1/6/2005

http://www.educationnews.org/towards-a-unified-theory-of-grad.htm

I did norming exercises with fellow graduate students both as TAs and in writing/teaching seminars, and always found the exercises useful. Our perspectives were usually pretty close, as long as our disciplines were pretty close....

But I think it's nearly meaningless to have a quantitative grading system without a clear shared understanding of what those grades mean.


Jonathan Dresner - 1/6/2005

There will be exceptions, and class size should be noted along with grade norm. But generally, a lower level course will have a different distribution than a higher level course and most people looking carefully at a transcript will realize that.


chris l pettit - 1/6/2005

Here in South Africa, all professors use externals from different universities or who are leading scholars in the field to give two different ideas of what grade should be given. I have always found it to be a good thing because a) I have never minded a second opinion to help put my own thoughts in perspective and b) because we all know politically minded profs that discriminate against those who do not share their views (my C in International Law in law school is a perfect example of such a practice) and the external helps keep those biases in check because the external is not chosen by the professor of the course.

CP
www.wicper.org


Julie A Hofmann - 1/6/2005

KC, how would that happen, unless there were a mandatory curve? I have taught classes where there were no As, and classes where there were no Fs. If better students self-select into more difficult classes, you'd probably just get a narrower and higher grade distribution, wouldn't you?


Robert KC Johnson - 1/6/2005

Very interesting. One idea that I know has been floated at some institutions has been to couple a student's grade with the average GPA of the course, to give a sense of how the student fared against his or her peers. This approach has some merit, especially in introductory courses.

I'm somewhat ambivalent on the issue of grade inflation. I teach a lot of upper-division electives that tend to be somewhat self-selective, and worry about grade policies that would, in effect, punish students for taking more challenging classes.


Rob D. Priest - 1/5/2005

I know that you didn't ask for my opinion, but I'd like to give it anyway, just because as a UK undergrad I find it difficult to imagine work not being 'second-marked'. This is not so much because I like the idea of standardised grading, but because I dunno, it just feels <em>fair</em>.

Two examples:

1.
I had a guy (Mr. X) who was quite clearly and infamously a ludicrously high marker in my first year, and seemed to give almost everybody 'First' class marks for their essays. A lot of people ended up being 'marked down',* which, while initally obviously quite disappointing to them, did mean that
a) there was no risk of them getting 90 and some other student flippantly going "well, he's an easy marker"
b) students themselves didn't feel like they were cheating a grade.
To have your grade from Mr. X upheld by the second marker was a really nice sign of confidence. Now, admittedly, if this guy didn't have such a reputation that kind of effect wouldn't occur, and you have to wonder how effective the system is if people like that still give out ludicrously high marks. But my main point is that, as a student, I feel my grade is earned when it's been second-marked.

2.
Also, I don't know how it is in the US, but it's a fundamental part of degree and course guidelines at UK universities that every grade has a fairly specific description, so it's difficult to imagine grades being too far from standard. For example, with my own course, if you haven't argued a strong, well-read but original argument which shows awareness of course objectives in your essay, then theoretically you shouldn't get a First. And there's a bit of paper somewhere that says so. Which also gives you a bit more confidence that your grade 'means something'. Not that I'm suggesting US grades mean nothing to students, just trying to give a sense of how UK students feel about it, at least at times when the system appears to be working.

*Sorry for the repeated quotation marks, I was trying to give a sense of just how common these phrases are.


Julie A Hofmann - 1/5/2005

This is an interesting topic, and one that's come up for me a couple of times in the recent path, but in different ways. For example, when the topic of grading standards came up in our Faculty Senate Council recently, there was a lot of insistance that faculty absolutely maintain their autonomy. My view was that faculty should definitely be able to say what constituted a particular grade in their field, but that it seemed unfair to the students if faculty then equated those grades differently in regards to the 4 point system we have to use on our grade rolls. There was some support for his, but faculty still remained resistent to giving up any control.

However, I've also been working with an interdisciplinary assessment group (one of those "do we assess on the college's stated outcomes?" exercises), and what I've found is that, even across the disciplines, we all have very close notions of what an A essay looks like as compared to a B essay. This becomes even more true if we use rubrics for our grading. To be fair, I think that working in such groups also helps to norm the grades, because we all have to articulate what we are looking for to our peers and explain why we might look for some things in our disciplines that others don't consider as important.

In the last case, in terms of disciplinary norms, it is entirely informal. If a student questions a grade, I simply hand the paper over to an appropriate faculty member for a second look. Moreover, I think many of us -- especially those of us who make up half of a department -- tend to norm a lot via hallway conversations, listservs, and blogs, but in a very informal way.

As I understand the AP grading process, all of the grades are normed. It would be interesting to see how those grades compare to grades given by college faculty who don't norm.


David Lion Salmanson - 1/5/2005

I liked norming among TAs as a TA. I was/am pretty anti-grade inflation and usually I didn't have to change much. What was most helpful was if we had a rubric prior to grading. That helps speed things up and keeps me honest.