The tutorials that I gave last week to first year biochemistry students have prompted me to think about assessment – and fairness.
The task for the tutorial, which is given on the Biological Chemistry course run by Prof Drickamer was pretty straight-forward. The students first had to attempt a four-part question taken from a previous exam that covers some core elements of amino acid chemistry and polypeptides structure. But the second part was more interesting: having tackled the question themselves, the students then had to mark the answers given to the same question by three former students. In the actual exam, staff had graded these three answers with scores of 45%, 65% and 88%.
This idea of course is to get students to think about the process of assessment.
In the groups that I had for my tutorials, most of the students marked their predecessors answers more severely than the staff had done, by 10% or more, although staff and students always agreed on the relative ranking of the three students’ answers. It was also clear that there was some variation between the marks awarded by the different tutorial students, despite the fact that they were all using the same marking criteria*.
One thing that cropped up in our discussions was that the staff markers seemed to have been a little over-generous in awarding 45% for the first student’s answer. In contrast, we agreed that the high marks awarded to the student who had scored 88% were justified, even though none of the tutorial students had dared to give such a high mark on their first assessment.
What all of this shows is that assessment is a difficult business. Even with defined criteria, it is tricky to weigh up precisely the worth of a single piece of work. There is inevitably some variation between markers. Within the department we mitigate this by ensuring that exams are first and second-marked by two different people but the subjective element is never eliminated completely.
That subjective variation might be a cause for concern among students but some reassurance can be found in thinking through the scale of the problem.
The question considered in the tutorial was worth about 14% of the Biological Chemistry exam, which counted for 75% of the marks of the course, which comprises 25% of the first year, which in turn accounts for 11% of the degree. So the question was worth 0.14 * 0.75 * 0.25 * 0.11 = 0.0029 = 0.29% of the total credit available for the degree. A large variation in marking the question of plus or minus 10% would therefore alter the final degree mark by 0.029%. That should provide some consolation against any perceived unfairness due to marker variation.
But we can provide additional consolation since over the course of the degree, each student is assessed on the basis of many pieces of coursework. On each occasion, they may benefit or lose out as a result of variation. The probabilities for the direction of variation are unknown but even if they are not 50:50 the overall effect will be to reduce the uncertainty in a given student’s marks.
The college formally recognises that it cannot assess students with ultimate precision because there are mechanisms in place to determine whether those students who end up just below degree class boundaries (by achieving 2.5% less than the mark required for a particular class of degree) should be awarded the higher degree.
Of course, none of the above considerations are intended to excuse clear errors in marking which students should feel free to flag up.
Finally, it may amuse you to know that universities and university departments are themselves increasingly subjected to the tyranny quantitative assessment, in the form of the Research Excellence Framework and various world rankings. If anyone would like to try to show me that the marking criteria used in these exercises are fit for purpose, please get in touch. I have my doubts.
*As I hope you all know the marking criteria we use for exams and all varieties of coursework are available on Blackboard – see Life Sciences (General Information).