Question grading

The Examplary platform supports two grading options for custom question types:

Deterministic response processing
Best for question types that have a clearly defined correct answer, and can be graded automatically based on the submitted answer and question settings.
AI-assisted grading suggestions
Best for open-ended questions where the answer needs to be interpreted against a rubric or teacher guidance.

Deterministic response processing

For simple question types that have a clearly defined correct answer, define responseProcessing in question-type.json. The condition is a JSONata expression that starts with = and evaluates to a boolean. If it returns true, the answer receives the question's maximum points; if it returns false, the answer receives 0 points.

The expression can read the submitted answer through answer and the question configuration through question. For example, answer.value is the user's submitted value, and question.settings contains the settings stored for that question.

question-type.json (partial)

{
  "responseProcessing": {
    "condition": "=answer.value in question.settings.possibleAnswers",
    "testCases": [
      {
        "answer": { "value": "42" },
        "question": { "settings": { "possibleAnswers": ["41", "42"] } },
        "expectedOutcome": true
      }
    ]
  }
}

Use testCases to validate the expression when you run the question type CLI checks (exp validate).

Exact-value matching

When a teacher configures a question with the exact-values rubric type, the platform automatically grades submissions by comparing the student's answer against the expected values defined in the scoring criteria. No configuration in question-type.json is required for this to work.

Setting exactValuesCaseInsensitive: true in the question's scoring configuration lowercases both values and strips all whitespace (not just leading/trailing) before comparing — so "New York" and "NewYork" would be considered a match. (Leading/trailing whitespace is always trimmed, regardless of this setting.) This is useful for fill-in-the-blank or vocabulary questions where minor formatting differences shouldn't count against a student.

AI assisted grading suggestions

The grading field is an object that defines the settings for AI-assisted grading. You can enable or disable this feature, and provide any additional instructions for the AI.

If enabled, the platform can use AI to suggest points and feedback for the user's answer based on the provided instructions. This can be particularly useful for open-ended questions where there isn't a single correct answer. The teacher can review and adjust the AI's suggestions before finalizing the grade.

question-type.json (partial)

{
  "grading": {
    "enabled": true,
    "instructions": "Grade the answer based on the provided rubric."
  }
}

Grading answers that contain files

Some answers are files rather than text, like an uploaded document or a voice recording. Out of the box, the AI only sees the URL of such a file, not its contents. To let the AI read the file, add fileInputs to the grading object: a JSONata expression starting with = that evaluates to a file URL or an array of file URLs. Like responseProcessing.condition, it can read the submitted answer through answer and the question configuration through question.

question-type.json (partial)

{
  "grading": {
    "enabled": true,
    "instructions": "Grade the uploaded document against the rubric.",
    "fileInputs": "=answer.value"
  }
}

The files are downloaded and attached to the end of the grading prompt, so the AI can read documents, images and audio recordings directly. Only files hosted on the Examplary media CDN, such as those uploaded through the file upload API, are attached; any other URL is skipped.

exp validate checks that the expression is valid JSONata.