Guidelines for Markers

Tim Kovacs. Last updated Dec. 2009.

This was originally written for markers on the Advanced Topics in Machine Learning unit who were marking either essays or reports on a data mining challenge. There are different kinds of assignments and feedback, and this discussion may be less relevant to other kinds.

Summative and Educative Assessment

There are two main objectives in marking coursework: The difference is emphasised by thinking of them as assessment of learning and assessment for learning respectively.

Students typically really appreciate good educative assessment. As a very rough rule a credit requires about 10 hours of work. An assignment worth 1/3 of a 10-credit unit might therefore take about 33 hours. Imagine you have spent 33 hours on an assignment. You would probably want to know why you got the mark you did and how you could do better in the future. This is particularly true if you did your best and could not easily think of ways to improve your work. In short, getting useful feedback is very important!

Educative assessment is particularly important for advanced work (such as final year projects) which requires many skills (e.g. an ability to write well, to judge what is relevant, critical evaluation), a high level of understanding, and ideally shows some creativity and a synthesis of material rather than just repetition of other sources. Note the use of the word 'skills'. Skills, unlike e.g. simple facts, are hard to teach and hard to learn. The only way to develop better skills is to use them, and to get feedback on them.

What to look for

Markers should of course read the assignment specification, and also the hints for written work which contains a list of things to watch out for. Here are some additions.

A report should:

A report should not:

Summative feedback

Provide a mark as a percentage using the online system, but do not publish it. For reports it is best to read the reports, make notes on them, and then rank them. Then you can map them to the UK's nonlinear scale. For MSc students a pass is 50% and marks of 70% or more are reserved for excellent work.

Educative feedback

You must provide individual comments on each report. You may also write group feedback in which you cover points which apply to many reports.

Educative feedback should indicate what is good and bad about a report and ways in which it could be improved. The amount of feedback should be in proportion to the value of the assignment. The standard approach is to upload a text file for each student. Here are some examples of good individual feedback for short reports in plain text. Another approach is to write comments on the report, scan it and upload it. Here is an example scanned report. Finally, you can also return the hardcopies to the students but we want to keep feedback on the online system for our records, so only do this if there is little important feedback which appears only in hardcopy.

Educative assessment is most useful when it is returned quickly, while the work is fresh in the students' minds. Since the students do a series of short reports it's important that they get feedback before doing the next in the series. Therefore we will set a deadline for each assignment to be marked. Please meet it or let us know as soon as possible if there is a problem.

Some final words for markers

If they disagree with your comments or marks students may reply to them. (Usually this is in the form of an email from the student which one of the lecturers forwards.) Often these students have worked hard and simply want to know whether their work really is correct or not - they want feedback. It can also be a matter of principle; when you're right you want the marker to acknowledge it. Often they are more interested in this than in changing their mark. (In fact it is the students who care more about whether they are right or wrong than about their marks who tend to be the best students.)

Remember that marking is an opportunity for the marker to develop some of the skills needed to produce reports, especially the ability to evaluate work critically.

If you have any questions on how to mark don't hesitate to ask one of the lecturers.

Some words for students

If you disagree with a marker's opinion you can debate it. Markers aren't always right, and don't always agree with each other.

Feedback on feedback is useful for markers. Some marking is done by PhD students who need to learn how to do it, and even very experienced lecturers have things to learn. So, if you feel you aren't getting the feedback you need, or if you find the feedback particularly useful, let us know.

More generally, if there are things you think are good or bad about a unit or degree, let us know. It's easier to calibrate the pace of a unit, or the way material is presented, or how useful the coursework is, with feedback from students.