Skip to content

Authoring and Deploying Assessments

Jump to bottom
Armando Fox edited this page Dec 14, 2022 · 6 revisions

An assessment is either a homework/lab or an exam that students take via PrairieLearn. Essentially, you will create a JSON file that sets some assessment metadata (due date, points, grading, etc.) and then incorporates by reference one or more of the questions (in the questions subdirectory) of your course repo.

Assessment deployment policy: READ THIS

Before an assessment can be deployed to students, you must do the following, at least 7 days before the assessment is opened to students. Failure to do these steps may result in the assessment not working, not being open to students, etc., and you will have no one to blame but yourself.

  • Add an event on this shared Google calendar. If you need edit access, ask Prof. Fox.

  • Designate a course staff member who will be on call by email for the duration of the assessment in case something goes wrong and the ops team needs to notify course staff.

  • In the event name and description, include the course number, assessment name ("Midterm 1", "Lab 1", etc.), and the name and email of the assessment contact. (Ideally, the assessment contact should be the person creating the calendar entry.)

  • If the assessment is a synchronous exam, the calendar entry should match those times and include a timezone. If it is open-ended, just make a 1-hour entry and note the due date in the event description, i.e. for how long students will be accessing this assessment.

  • Post a note in the #berkeley-cbt-ops channel of the UIUC PL Slack stating that you've made the calendar entry.

Creating the assessment

  • The PL Assessment Configuration page provides most of the information needed to create and configure an assessment. See next items for useful and critical information
  • You must set the credit property in allowAccess otherwise it's assumed to be 0 and student scores will be 0% in an assessment even if they answer all questions correctly! The accessControl documentation of credit provides how the value of this property is used
  • You should avoid the "mode" : "Exam" property. At first look, this seems like something you want. But it's not! This is strictly for assessments run at UIUC's Computer Based Testing Facility and the assessment will only be available if it's access from there. The accessControl documentation of server modes explains this property. Basically, if you're not at UIUC, avoid it!

Deploying the assessment

If you're using bCourses and want to link the assessment from there, follow these instructions to set that up (search for "LTI"). Note that when setting things like due dates and when the assessment is available, these settings exist both in infoAssessment.json and bCourses, but the more restrictive settings will dominate. (If the student clicks thru bCourses but they are outside the time windows allowed in infoAssessment.json, PL will not allow them to do the assignment.)

As a corollary, if you need to put in extra-time accommodations or conflict times, you need to do that in infoAssessment.json whether or not you choose to do it in bCourses. Someday we hope to have this integration be better.

  1. Make sure you've debugged/dry-run everything and pushed to the master branch of the course's repo

  2. Log in to the PL server, click your course instance, then Sync (in the navbar), and Pull From GitHub Repo. This pulls the HEAD of master to be the deployed version.

TBD: instructions for "testing" an assessment using Docker, then deploying it on cbt-dev

Dealing with question bugs/issues

Issues with questions

A student can manually 'report an issue' with a question while doing an assessment, or if the question causes a runtime error, an issue is automatically filed. (So if 200 students hit the same issue on the same question, you get 200 issues to clean up.)

Important. For some simple question types, like mult-choice/select-all, the link to the question in the Issues view will link to the specific variant seen by the student. So don't be surprised if you fix the issue in the question and re-sync but then clicking the link in the Issue still causes the question error!

Issues appear most-recent-first and filtering is limited. Be careful of the "Close all issues" button - it really closes all issues, not just the ones returned by the current filter!

Access to variant given to student

If for some reason, you need to see the specific variant of a question that was given to a particular student, try the following (Posted by Jonatan on Slack#pl-help):

if you go to the assessment log under the assessment details, you will find the specific instances the student used (e.g., S-1#1). If you click on that link you'll find the student's version of that particular variant. If from there you go to the Instructor information box, you will see a link for the QID, that link feeds the appropriate variant seed to the instructor view.

After the assessment is closed

External docs

If you want students to be able to have read-only access to quizzes after the quiz closes (eg for review), make the following the first stanza of the allowAccess part of infoAssessment.json:

{
  "mode": "Public",
  "active": false
}

Public mode actually doesn't matter for us - it means any IP address can access the exam. (Exam mode should be avoided, as described above.) Non-active means that students cannot interact with the assessment in any way, only view it.

Clone this wiki locally