Question grading

The Examplary platform supports two grading options for custom question types:

1. Deterministic response processing
   Best for question types that have a clearly defined correct answer, and can be graded automatically based on the submitted answer and question settings.
2. AI-assisted grading suggestions
   Best for open-ended questions where the answer needs to be interpreted against a rubric or teacher guidance.

Deterministic response processing

For simple question types that have a clearly defined correct answer, define responseProcessing in question-type.json. The condition is a JSONata expression that starts with = and evaluates to a boolean. If it returns true, the answer receives the question's maximum points; if it returns false, the answer receives 0 points.

The expression can read the submitted answer through answer and the question configuration through question. For example, answer.value is the user's submitted value, and question.settings contains the settings stored for that question.

{
  "responseProcessing": {
    "condition": "=answer.value in question.settings.possibleAnswers",
    "testCases": [
      {
        "answer": { "value": "42" },
        "question": { "settings": { "possibleAnswers": ["41", "42"] } },
        "expectedOutcome": true
      }
    ]
  }
}

Use testCases to validate the expression when you run the question type CLI checks (exp validate).

The legacy settings.correctAnswer field is still used by some built-in question types and import/export flows, but it does not automatically grade an answer on its own.

AI assisted grading suggestions

The grading field is an object that defines the settings for AI-assisted grading. You can enable or disable this feature, and provide any additional instructions for the AI.

If enabled, the platform can use AI to suggest points and feedback for the user's answer based on the provided instructions. This can be particularly useful for open-ended questions where there isn't a single correct answer. The teacher can review and adjust the AI's suggestions before finalizing the grade.

{
  "grading": {
    "enabled": true,
    "instructions": "Grade the answer based on the provided rubric."
  }
}