Peer review ‐ how to set up

This page describes how peer reviews can be set up for exercises. Peer review is a useful alternative grading method to automatic or teacher grading.

What is a peer review

In the peer review process, students grade other students answers to the same question they have answered themselves. The grading is based on peer review questions defined by the teacher. Peer review is a useful alternative grading method to automatic or teacher grading.

How to set up a peer review exercise

1. Add a new exercise.

See Add new exercise wiki page for detailed instructions

2. Configure peer review

• Add peer review: toggle peer review on / off.
• Use course default peer review config:
  • On: the exercise uses the shared courses default peer review settings. This is useful if all peer reviews follow the same format, for example students always answer the same set of questions.
  • Off: the peer review settings only apply to this particular question. This is useful if the peer review questions only apply to this one question.
• Peer reviews to receive: This number determines how many peer reviews the student needs to receive for their answer before they can receive points for the exercise task.
• Peer reviews to give: This number determines how many submissions by other students the student needs to grade before they can receive points from the exercise task. (This number needs to be higher than the peer reviews to receive to make sure that every students receives enough peer reviews, taking into account that some students may drop out of the course and not submit peer reviews).
• Peer review accepting strategy:
  • A Automatically accept or reject by average:
    • PASS: If the average score of the likert scale questions that the student receives is above the peer review accepting threshold: student receives full points for the exercise.
    • FAIL: If the average score is below the threshold: student automatically receives 0 points.
  • B Automatically accept or manual review by average:
    • PASS: If the average score of the likert scale questions that the student receives is above the peer review accepting threshold: student receives full points for the exercise.
    • FAIL: The answer will be moved to the answer requiring attention section, where the course staff grades the answer. Student does NOT receive points automatically.
  • C Manual review everything: All answers are moved to the answer requiring attention section after they have received enough peer reviews.
• Peer review accepting threshold: Students either pass or fail the exercise based on the average of grades they receive from the Likert scale type peer review questions. If the average is above the threshold, the student completes the exercise successfully and get points (if accepting strategy is A or B). If the average is below the threshold: student does not receive points (accepting strategy A) or the answer will be moved to the answer requiring attention section, where the course staff grades the answer (accepting strategy B). NOTE: This setting does not apply to accepting strategy C.

Example: If student A receives these scores:

• Peer review question 1 (likert scale): 3 (from student C), 5 (from student Y): Average 4
• Peer review question 2 (likert scale): 2 (from student C), 4 (from student y): Average 3
• TOTAL AVERAGE: 3.5
• RESULT: 3.5 > 2.1 (Peer review accepting threshold set for the exercise task), which means that Student A's answer is passes the check and receives points.

3. Add peer review questions

• Add peer review question button: Adds a new peer review question
• Peer review question type dropdown: Select question type (likert scale, essay). Likert scale questions are required if you want to use automatic scoring for the exercise task.
• Peer review question: This text is shown to students when they peer review other students answers. Should include the criteria for grading the answer.
• Answer required: Determines whether an answer must be given or not.

Peer review flow - from a student point of view

1. Student A submits an answer to an exercise. The answer is usually a text of a certain length.
2. Student A starts the peer review process, they are randomly assigned another students submission to peer review. For example, students B and X.
3. Student A reviews student B's answer - usually by answering likert-scale questions.
4. Student A reviews student X's answer.
5. Student A waits until they have enough peer reviews from other students.
6. Student A receives points for the exercise based on the Peer review accepting strategy.