Adventures in teaching

Oct 31 2011 Published by under [Education&Careers]

As part of a graduate class assignment I gave students a pre-review draft of a manuscript* and asked them to review it. We had discussed review, but none had experience with it. To complicate the matter, the class draws a wide range of students with varying backgrounds. Some were obviously more familiar with the background and topic of the manuscript than others. I later gave them the actual reviews of the paper and the final published version. Overall, I think this was a useful exercise and they appear to have as well.

But now I am grading their reviews and realizing that they are incredibly difficult to evaluate. In retrospect I could have given them clearer expectations for this assignment, but their work on this assignment is far more scatter shot than any of the previous assignments. I have no idea what to do with these.

*The manuscript was a review, intended for a broad audience.

10 responses so far

  • Natalie says:

    I'd probably "mark it easy" so that you aren't punishing the less experienced students, improve your marking criteria for next year, and chalk up the experience as worthwhile for both you & the students.

  • pyrope says:

    What a great idea! I bet that would be really helpful for grad students who haven't reviewed before. I've never really thought about what makes for an effective review...just kind of feel my way based on what's been most helpful on the receiving end. Do you have a rubric of some sort now that you're grading them? One thing I've tended to notice with early career reviews (when I've solicited them) is that they can be ruthless and pedantic. I think I was overly harsh in early reviews too - and one piece of advice from grad school that helped me was to balance negative comments with positive ones in reviews (if the paper merits it).

  • That really is a great idea for an assignment!

  • Dan says:

    This doesn't help you for this year, but...

    I did something very similar with an undergraduate seminar I taught last spring. I was really impressed with the reviews the students wrote, and I think one key was that I showed students examples of reviews before they got to work on their own. We discussed what was effective and what wasn't effective in a review, and from this discussion, we generated some of the rubric that the students were graded on.

    The students then applied what they learned to reviewing manuscripts written by their peers. It was way more in-depth than the "peer editing" that they do in other classes and it was done in a professional context- they ate it up. I was blown away with the quality of the reviews (most students went back to the literature to fact check claims of the papers they were reviewing) and will definitely do this sort of thing in the future.

    BTW, my students were shocked at how "mean" the reviews were. (The reviews resulted in revise and resubmit, so they weren't that bad!) Were your's?

  • MM says:

    I co-taught an advanced undergrad class where we had a generally similar assignment. We ended up commenting heavily on each draft and then dividing our students' papers into A, B, and C piles, which corresponded to 95, 85, and 75% respectively. Ones that were in between an A and B, for example, we gave 90%. It was much easier doing the sort and then assigning the grade than trying to be consistent while doing all the individual comments.

  • gerty-z says:

    that sounds like a good assignment, but i can see how it would be really hard to grade. If you figure out a rubric that can make it work, I'd be interested to hear about it.

  • anon says:

    Can you go for pass/fail? Why grade them at all - if the student made an effort and participated in class discussion, who's to say whether they are right or wrong. It's a good assignment, but it seems that grading in this case might be too subjective.

    That said, if you must grade, I would lean toward rewarding those who may have caught something missing from the article (if it is a review) and was well-justified. Or, had a review that was well-justified regardless of whether something was missing, over-the-top, or over-interpreted.

  • Alex says:

    Grade it easy, but next time give them (1) a sample review (e.g. an insightful review that you've received on a previous paper, or some insightful feedback you've received from a lab member or collaborator, or whatever writing sample makes the most sense) and (2) your grading rubric.

  • proflikesubstance says:

    I will likely go over the whole class in a post towards the end of the semester, with what worked and what didn't. So far I think it has been very useful to the students, but we'll see what they give me for feedback.

  • Jim Thomerson says:

    I've had some training in writing across the curriculum. Some of my colleagues have taken it as a bunch of bull, but I have used it in several large section classes. I think it does more good than harm. The basic ideas are writing several drafts, with peer review and evaluation. One time I used one of Gould's anthologies and assigned each group a particular topic. If I got around to that particular topic in lecture, I would call on the group who knew about it. I got some really good input from the student 'experts".

Leave a Reply