Exams by Us

In this class, we are going to try writing our own exams.

Why? Once you leave school, you won’t have instructors. But you won’t stop learning. So you’re going to need to take ownership of your own learning.

Since it’s easy to feel like you’ve learned something without actually learning it, some ways of assessing your learning are important. One way to do this is to try exercises, like you’ll be doing in your projects. Another way is to test yourself. So that’s what we’ll try now.

[GitHub Copilot adds, based on my outline:] Another benefit of writing your own exams is that you’ll get to practice learning in community. You’ll get to see how your classmates are learning, and you’ll get to help them learn. And you’ll get to see how you learn, and how you can help yourself learn.

Stage 1: Writing Questions

In this stage, each student will write questions. Expectations:

Aim for a range of questions about different topics. For example, when we first ran this, we got lots of questions about loss function selection early on. Review what’s already there before you suggest your own.

Multiple Choice

A multiple-choice question should have one clearly correct answer, and three relevant detractors. The question should be clear, and the distractors should be relevant. The distractors should not be so similar that they could be confused with the correct answer.

Short Answer

Short answer questions might start with words like “Explain”, “Compare”, “Describe”, “Define”, “Weigh the pros and cons of”, etc. There should be a single answer that is clearly correct.

Difficulty

You should try to vary the difficulty of your questions. As a rough guide, aim for each question to be at a different level of Bloom’s Taxonomy. Concentrate on the first 4 levels:

Stage 2: Reviewing Questions

Each student should contribute two revisions to questions written by other students. Revisions can be improvements to the clarity of the question (it should be clear what the question is asking), or improvements to the answer key (it should be clear what the correct answer is, and why it’s correct).

Stage 3: Taking the Exam

Individually: Each student will get a stratified random sample of questions. The sample should include one question from each unit, and one question from each type (multiple-choice, short-answer, etc.).

We will also collaboratively test ChatGPT. Each student will give ChatGPT a question, then reflect on whether the answer is correct.

We will also reflect on our experience with this first exam and how we can improve it for the next one.

Logistics

We’ll do this all in a GitHub repo: https://github.com/kcarnold/cs344-exam-23sp

How to take a random sample of questions?

We want a stratified random sample, with one question per unit. You could do this by hand, but here’s some code to do it. Run this in your JavaScript console on the GitHub repo page. (If this is the first time you’re pasting something into the Console, you may get a message saying that you need to type something like “allow pasting”. Do that, then paste again.)

The result will show up in the Console and in the page body.

(() => {
  let byUnit = new Map();
  document.querySelectorAll('[aria-label="Directory"]').forEach(x => {
    let name = x.closest('[role="row"]').querySelector('a').textContent;
    let unit = /^(u|uc)(\d+)-(.+)$/.exec(name)[2];
    let contents = byUnit.get(unit) || [];
    contents.push(name);
    byUnit.set(unit, contents);
    console.log(unit)
  })

  let result = [];
  for (let [unit, options] of byUnit) {
    console.log(options)
    result.push(options[Math.floor(Math.random() * options.length)]);
  }
  console.log(result.join(', ');
  document.body.innerHTML = `<h1>Your questions</h1>${result.join(', ')}`;
})()

References

← Forum Posts 22SP
Announcements 23SP →