// File: index
# Examplary Developers
The Examplary platform provides a set of tools and APIs for developers to integrate and extend the platform's capabilities.
This documentation guides you through the available features, how to set up your development environment, and how to create custom question types and integrations.
**[Learn more about Examplary →](https://examplary.ai/)**
---
// File: embed-sessions
# Embed Sessions
Our platform makes it possible for other service providers to integrate parts of the Examplary experience within their products.
To make this possible, we offer the **Embed Sessions** API, which allows the service provider to get a unique link
to a part of the Examplary UI that can be embedded in their UI.
This makes it possible to integrate flows such as exam generation or question generation directly into your own application,
while still leveraging the full power of the Examplary platform.
## Getting started
To get started testing this flow, first create a [Examplary](https://app.examplary.ai/register) account.
You'll need to get your organisation approved for using embed sessions. Please contact us at [hi@examplary.ai](mailto:hi@examplary.ai) with your organisation details or use the chat widget in the app to request access.
Find your API key in the [Developer settings](https://app.examplary.ai/account/developer) page. Then, follow the instructions in the [exam generation flow](/developers/embed-sessions/flows/exam-generation) document to create your first embed session.
Or have a look at the [live code example](https://examplary.github.io/embed-sessions-demo/).
## Supported flows
- [Exam generation](/developers/embed-sessions/flows/exam-generation) (`generate-exam`)
- [Edit rubric](/developers/embed-sessions/flows/edit-rubric) (`edit-rubric`)
- [Question generation](/developers/embed-sessions/flows/generate-question) (`generate-question`)
- [Grade answer](/developers/embed-sessions/flows/grade-answer) (`grade-answer`)
- [Edit exam](/developers/embed-sessions/flows/edit-exam) (`edit-exam`)
- [Take exam (student)](/developers/embed-sessions/flows/take-exam) (`take-exam`)
- [Practice space (student)](/developers/embed-sessions/flows/practice-space-student) (`practice-space-student`)
## Actor field
Every `POST /embed-sessions` request requires an `actor` field: the user ID of the Examplary user who will interact with the embedded UI. Three constraints apply:
1. **Must be a user ID** — the value must start with `user_`.
2. **Must differ from the API key owner** — the actor cannot be the same user as the one whose API key you are using to authenticate the request. If they match, the API returns a `400` error with the message: _"Actor ID cannot be the same as the requesting user. Create a new user in the workspace and use its ID."_
3. **Must exist in the workspace** — the actor must already be a member of your workspace. If the user ID doesn't match any user in your workspace, the API returns a `400` error with the message: _"Actor with the given ID does not exist as a user within your workspace."_
The typical pattern is to call `POST /users` once per end-user and store the returned `user_…` ID. Use that stored ID as the `actor` for all future embed sessions for that user.
Every embed session also records a `createdBy` field: the `user_…` ID of the API key owner who created it (i.e. the requesting user from constraint 2 above, not the `actor`). Only that API key owner can `GET` or `DELETE` an embed session afterwards.
## Example flow
Example flow for exam generation using embed sessions:
```mermaid
sequenceDiagram
participant Dev as Service provider
participant API as Examplary API
participant UI as Examplary hosted UI
Note over Dev: User triggers test creation
Dev->>API: Call POST /users to create a user if not exists
API-->>Dev: Returns user ID
Dev->>API: Call POST /embed-sessions to create session
API-->>Dev: Returns session ID and URL
Dev->>UI: Open modal or iframe with session URL
Note over UI: User configures exam settings, tweaks exam outline, etc.
Note over UI: Exam is generated
UI-->>Dev: Redirect or postMessage() event with created exam ID
Dev->>API: Call POST /exams/{id}/export/qti3-zip
API-->>Dev: Returns created questions in QTI 3 package
```
---
// File: embed-sessions/theme-options
# Theme options
All embed session flows require a `theme` object in the `POST /embed-sessions` request body to customize the appearance of the embedded UI. Every key within it is optional, so pass `"theme": {}` if you don't want to customize anything.
| Key | Type | Description |
| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------- |
| `primaryColor` | string | A CSS hex color code for primary UI elements (buttons, highlights). Accepts `#RRGGBB` or `#RGB` format. |
| `backgroundColor` | string | A CSS hex color code for the background of the embed. Accepts `#RRGGBB` or `#RGB` format. |
| `locale` | string | A 2-character locale code (e.g. `en`, `nl`). Unsupported codes fall back to English. |
| `strings` | object | An object containing string overrides for the embed session. See [String overrides](#string-overrides) below. |
Example:
```json
{
"primaryColor": "#4f46e5",
"backgroundColor": "#f9fafb",
"locale": "en"
}
```
The "unsupported codes fall back to English" behavior above only covers codes the platform doesn't recognize. Omitting `locale` entirely is different for the `take-exam` and `practice-space-student` flows: instead of defaulting to English, the embed's UI automatically matches whatever language the exam or practice space itself is set to. Set `locale` explicitly if you need the UI language to stay independent of the content's language.
## String overrides
You might want to update labels in the UI. To do so, you can specify a `strings` object in the `theme` object. The keys in this object correspond to the labels in the UI, and the values are the strings you want to display.
Example:
```json
{
"theme": {
"strings": {
"account.attributes.student-level-name": "Candidate level",
"account.attributes.subject-name": "Assessment field"
}
}
}
```
To find the name of a particular string, you can use the Examplary dev tools. Reveal them by pressing `Shift + Control + Tilde`.
Then press the 'Language' icon, and switch to 'Translation keys'.

---
// File: embed-sessions/flows/exam-generation
# Exam generation flow
Using the Examplary exam generation flow, you can let users of your application use AI to generate questions from a set of source materials, optionally scoped to a specific subject.

## Presets
Presets allow you to pre-configure certain options for the embed session, so that when your users open the embed, they see a tailored experience. Every key within `presets` is optional, but the `presets` object itself is required — pass `"presets": {}` if you don't want to pre-configure anything.
| Key | Type | Description |
| -------------------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name` | string | The name of the exam. Plain text. |
| `subject` | string | The subject of the exam. Plain text. |
| `studentLevel` | string | The difficulty level of the exam. Expects a student level ID from `/student-levels`. |
| `intendedDuration` | string | The intended duration of the exam. One of `"15-30 minutes"`, `"30-45 minutes"`, `"45-60 minutes"`, `"60-90 minutes"`. |
| `taxonomyId` | string | The ID of the taxonomy to use for the exam, from `/taxonomies`. |
| `language` | string | The language code for the exam (e.g. "en" for English). Defaults to the user's locale. |
| `sourceMaterialIds` | string[] | An array of source material IDs to pre-select for the exam. |
| `allowedGenerationQuestionTypes` | string[] | An array of [question type IDs](/developers/guides/default-question-types) to limit the question generation options to. |
| `context` | string | Additional context or instructions for the exam generation. Plain text. |
| `hideSubject` | boolean | Whether to hide the subject field in the exam generation UI. Defaults to `false`. |
| `hideStudentLevel` | boolean | Whether to hide the student level field in the exam generation UI. Defaults to `false`. |
| `hideDuration` | boolean | Whether to hide the duration field in the exam generation UI. Defaults to `false`. |
| `hideTaxonomy` | boolean | Whether to hide the taxonomy field in the exam generation UI. Defaults to `false`. |
| `hideQuestionTypes` | boolean | Whether to hide the question types field in the exam generation UI. Defaults to `false`. |
| `showPreview` | boolean | Whether to show the generated questions for review before the session completes. Defaults to `false`. See [Preview step](#reference-preview-step). |
| `availableStudentLevels` | StudentLevel[] | An array of [custom student levels](#reference-custom-student-levels) to make available for selection in the exam generation UI. When specified, only these student levels will be shown. |
## Example
### 1. Create an Examplary user for your user
To make sure we can save personal preferences and source materials uploaded by the user to their specific account,
and can later allow them to reference these personal details, we require creating a user account in your workspace for each of your users.
```json title="POST /users"
{
"email": "my-user@example.com",
"name": "My User"
}
```
Store the returned user ID in your system to use with any future embed sessions for that user.
### 2. Create an embed session
Call the Examplary API to create a new embed session. You can configure presets for the exam, as well as theme options.
The `actor` field should contain the ID of the Examplary user account you created for this customer.
You can either specify a `returnUrl` or an `allowedOrigin`, based on how you want to be notified when generation is completed.
```json title="POST /embed-sessions"
{
"flow": "generate-exam",
"actor": "{userId}",
"presets": {
"subject": "Mathematics"
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"myUniqueIdentifier": "abc1234"
},
"returnUrl": "https://app.example.com",
"allowedOrigin": "https://app.example.com"
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 3. Lead the user to the embed URL
You can either redirect the user directly to the URL, or display it in an `iframe`.
The latter might be better for the user experience, especially when displayed as a modal. This also allows you to listen to status updates in real time,
through the `postMessage()` API.
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
window.addEventListener("message", (event) => {
// Make sure the message is coming from a trusted origin
if (event.origin !== new URL(embedUrl).origin) return;
// Handle the message
const { type, status, outputs } = event.data;
if (type === "examplary:embed-status-update") {
if (status === "started") {
// Fired as question generation progresses — outputs.questionGenerationStatus
// reflects the current generation step (e.g. "generating_outline",
// "generating_questions"). Useful for showing a progress indicator.
console.log("Generation in progress:", outputs.questionGenerationStatus);
} else if (status === "completed") {
const examId = outputs.examId;
console.log("Exam generation completed! Exam ID:", examId);
}
}
});
```
If you're using redirects, you can expect a redirect to your specified `returnUrl` on success, with the full `outputs` object and the embed session ID folded into the query string: `?status=completed&embedId=embed_session_55S8...&examId=exam_dj8948hf98hf43&questionGenerationStatus=completed`
If neither of these options work for you, you may also poll the Examplary API for status updates:
```jsonc title="GET /embed-sessions/{id}"
{
"id": "embed_session_55S843D7HfNfs9RY48PoTprXnRcz2...",
"flow": "generate-exam",
"status": "completed",
"createdBy": "user_9pQr82jXd0dJ...",
"outputs": {
"examId": "exam_dj8948hf98hf43",
"questionGenerationStatus": "completed",
},
// ...other embed session fields omitted
}
```
### 4. Get exam contents
Once the question generation has completed, you can use the API to retrieve your generated questions.
In Examplary's normal format:
```
GET /exams/{examId}
```
Or as a QTI package:
```
POST /exams/{examId}/export/qti3-zip
```
### 5. Cleanup
The embed session will expire automatically after 7 days, but because it gives some limited access to your account, you might want to remove it manually:
```
DELETE /embed-sessions/{embedSessionId}
```
You can also delete the generated exam once you don't need it anymore:
```
DELETE /exams/{examId}
```
---
## Reference: custom student levels
When you want to override student levels, you can supply the following preset option:
```json
{
"presets": {
"availableStudentLevels": [
{
"id": "beginner",
"name": "Beginner driver",
"instructions": "Use simple language, and try to avoid jargon. Keep explanations clear and concise."
},
{
"id": "advanced",
"name": "Advanced driver",
"instructions": "Use technical terminology. Be precise and detailed in your explanations, and expect depth of knowledge from the student."
}
]
}
}
```
Reference fields for a `StudentLevel` object include:
- `id`: A unique identifier for the student level.
- `name`: A human-readable name for the student level.
- `instructions`: Guidance on how to tailor explanations for students at this level. Include information about language use and the types of questions to include. Keep this to 2-3 sentences. This is included in every prompt, so very long descriptions make the generation process noticeably slower.
## Reference: preview step
With `showPreview` set to `true`, the embed shows a read-only list of the generated questions once generation has finished, instead of completing straight away. The user can go back to the settings to regenerate (this clears the generated questions), or accept the questions. The session only completes — and `examplary:embed-status-update` only fires with `status: "completed"` — after they accept.
---
// File: embed-sessions/flows/generate-question
# Question generation flow
Using the Examplary question generation flow, you can let users of your application use AI to generate a single question on a specific subject or a set of source materials.
## Presets
Presets allow you to pre-configure certain options for the embed session, so that when your users open the embed, they see a tailored experience. Every key within `presets` is optional, but the `presets` object itself is required — pass `"presets": {}` if you don't want to pre-configure anything.
| Key | Type | Description |
| ------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `subject` | string | The subject of the question. Plain text. |
| `studentLevel` | string | The difficulty level of the question. Expects a student level ID from `/student-levels`. |
| `language` | string | The language code for the question (e.g. "en" for English). Defaults to the user's locale. |
| `context` | string | Additional context or instructions for the question generation. Plain text. |
| `sourceMaterialIds` | string[] | An array of source material IDs to pre-select for the question. |
| `questionType` | string | Specify the [ID of a question type](/developers/guides/default-question-types) to generate. Will hide the question type selector in the UI. |
| `hideSubject` | boolean | Whether to hide the subject field in the question generation UI. Defaults to `false`. |
| `hideStudentLevel` | boolean | Whether to hide the student level field in the question generation UI. Defaults to `false`. |
| `outputFormat` | string | Specify the output format for the generated question. Can be either `examplary_json`, `qti_v2p1`, `qti_v3p0`. Defaults to `examplary_json`. When set to a QTI format, `outputs.question` will be an XML string instead of a JSON object. |
## Example
### 1. Create an Examplary user for your user
To make sure we can save personal preferences and source materials uploaded by the user to their specific account,
and can later allow them to reference these personal details, we require creating a user account in your workspace for each of your users.
```json title="POST /users"
{
"email": "my-user@example.com",
"name": "My User"
}
```
Store the returned user ID in your system to use with any future embed sessions for that user.
### 2. Create an embed session
Call the Examplary API to create a new embed session. You can configure presets for the question, as well as theme options.
The `actor` field should contain the ID of the Examplary user account you created for this customer.
You can either specify a `returnUrl` or an `allowedOrigin`, based on how you want to be notified when generation is completed.
```json title="POST /embed-sessions"
{
"flow": "generate-question",
"actor": "{userId}",
"presets": {
"subject": "Mathematics"
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"myUniqueIdentifier": "abc1234"
},
"returnUrl": "https://app.example.com",
"allowedOrigin": "https://app.example.com"
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 3. Lead the user to the embed URL
You can either redirect the user directly to the URL, or display it in an `iframe`.
The latter might be better for the user experience, especially when displayed as a modal. This also allows you to listen to status updates in real time,
through the `postMessage()` API.
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
window.addEventListener("message", (event) => {
// Make sure the message is coming from a trusted origin
if (event.origin !== new URL(embedUrl).origin) return;
// Handle the message
const { type, status, outputs } = event.data;
if (type === "examplary:embed-status-update") {
if (status === "completed") {
const question = outputs.question;
console.log("Question generation completed! Question:", question);
}
}
});
```
If you're using redirects, you can expect a redirect to your specified `returnUrl` on success, with the embed session ID folded into the query string: `?status=completed&embedId=embed_session_55S8...&question=...`
If neither of these options work for you, you may also poll the Examplary API for status updates:
```jsonc title="GET /embed-sessions/{id}"
{
"id": "embed_session_55S843D7HfNfs9RY48PoTprXnRcz2...",
"flow": "generate-question",
"status": "completed",
"createdBy": "user_9pQr82jXd0dJ...",
"outputs": {
"question": "..."
}
// ...other embed session fields omitted
}
```
`outputs.question` is a JSON object when using the default `examplary_json` format, or an XML string when using `qti_v2p1` or `qti_v3p0`.
### 4. Cleanup
The embed session will expire automatically after 7 days, but because it gives some limited access to your account, you might want to remove it manually:
```
DELETE /embed-sessions/{embedSessionId}
```
---
// File: embed-sessions/flows/edit-rubric
# Edit rubric flow
Examplary includes a rubric editor flow that allows users to create and edit rubrics for grading questions. You can embed this flow in your application to let users create their own rubrics, or edit existing ones.

## Presets
Presets allow you to pre-configure certain options for the embed session, so that when your users open the embed, they see a tailored experience. Every key within `presets` is optional, but the `presets` object itself is required — pass `"presets": {}` if you don't want to pre-configure anything.
| Key | Type | Description |
| --------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `rubric` | object | An object representing a rubric to pre-load in the editor. Should follow the same structure as the `rubric` field in the API response of the rubric editor flow. [JSON schema](https://schemas.examplary.ai/question-scoring.json). |
| `showDoneButton` | boolean | Whether to show a "Done" button in the UI that allows users to indicate they are done editing the rubric. Defaults to `true`. |
| `hideTemplateOptions` | boolean | Whether to hide the template selection options in the rubric editor. Defaults to `false`. |
| `hideGenerateOptions` | boolean | Whether to hide the AI generation options in the rubric editor. Defaults to `false`. |
| `hideExportOptions` | boolean | Whether to hide the export options in the rubric editor. Defaults to `false`. |
## Example
### 1. Create an Examplary user for your user
To make sure we can save personal preferences and rubric templates created by the user to their specific account,
and can later allow them to reference these personal details, we require creating a user account in your workspace for each of your users.
```json title="POST /users"
{
"email": "my-user@example.com",
"name": "My User"
}
```
Store the returned user ID in your system to use with any future embed sessions for that user.
### 2. Create an embed session
Call the Examplary API to create a new embed session. You can configure presets for the rubric editor, as well as theme options.
The `actor` field should contain the ID of the Examplary user account you created for this customer.
You can either specify a `returnUrl` or an `allowedOrigin`, based on how you want to be notified when generation is completed.
```json title="POST /embed-sessions"
{
"flow": "edit-rubric",
"actor": "{userId}",
"presets": {
"showDoneButton": true
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"myUniqueIdentifier": "abc1234"
},
"returnUrl": "https://app.example.com",
"allowedOrigin": "https://app.example.com"
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 3. Lead the user to the embed URL
You can either redirect the user directly to the URL, or display it in an `iframe`.
The latter might be better for the user experience, especially when displayed as a modal. This also allows you to listen to status updates in real time,
through the `postMessage()` API.
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
window.addEventListener("message", (event) => {
// Make sure the message is coming from a trusted origin
if (event.origin !== new URL(embedUrl).origin) return;
// Handle the message
const { type, status, outputs } = event.data;
if (type === "examplary:embed-status-update") {
console.log(status);
if (status === "completed") {
const rubric = outputs.rubric;
console.log(
"Rubric creation complete. You might want to hide the iframe now.",
rubric,
);
}
}
});
```
If you're using redirects, you can expect a redirect to your specified `returnUrl` on success, with the embed session ID and the rubric output folded into the query string:
- On success: `?status=completed&embedId=embed_session_55S8...&rubric={...}`
If neither of these options work for you, you may also poll the Examplary API for status updates:
```jsonc title="GET /embed-sessions/{id}"
{
"id": "embed_session_55S843D7HfNfs9RY48PoTprXnRcz2...",
"flow": "edit-rubric",
"status": "completed",
"createdBy": "user_9pQr82jXd0dJ...",
"outputs": {
"rubric": {
// ...
}
}
// ...other embed session fields omitted
}
```
You can find the JSON schema for the rubric output [here](https://schemas.examplary.ai/question-scoring.json).
### 4. Cleanup
The embed session will expire automatically after 7 days, but because it gives some limited access to your account, you might want to remove it manually:
```
DELETE /embed-sessions/{embedSessionId}
```
---
// File: embed-sessions/flows/grade-answer
# Grade answer flow
Using the Examplary grade answer flow, you can let users of your application use AI to grade a single answer on a specific subject or a set of source materials.

## Presets
| Key | Type | Description |
| --------------------- | ------- | --------------------------------------------------------------------------------------------------------- |
| `examId` | string | The ID of the exam that contains the question to be graded. Must start with `exam_`. Required. |
| `sessionId` | string | The ID of the assessment session that contains the answer to be graded. Must start with `ses_`. Required. |
| `questionId` | string | The Examplary ID of the question to be graded, or any external ID you assigned to it. Required. |
| `hideScoringCriteria` | boolean | Whether to hide the scoring criteria in the grading UI. Defaults to `false`. |
## Example
### 1. Create an Examplary user for your user
To make sure we can save personal preferences and source materials uploaded by the user to their specific account,
and can later allow them to reference these personal details, we require creating a user account in your workspace for each of your users.
```json title="POST /users"
{
"email": "my-user@example.com",
"name": "My User"
}
```
Store the returned user ID in your system to use with any future embed sessions for that user.
### 2. Create an exam
Before you can use the grade answer flow, you need to have an exam and assessment session that the user can reference when grading the answer.
Let's first create the exam ([API reference](/developers/rest-api/exams/post-exams)):
```json title="POST /exams"
{
"name": "Book review",
"subject": "English Literature",
"studentLevel": "us_undergraduate",
"questions": [
{
"id": "my_question_1",
"type": "single-line-text",
"title": "What is the meaning of life?",
"description": "Give a short answer to the ultimate question.",
"settings": {},
"scoring": {
"rubricType": "analytical",
"criteria": [
{
"id": "c1",
"title": "Correctness",
"levels": [
{
"id": "c1l1",
"title": "Incomplete",
"description": "Does not mention the number 42 or the Hitchhiker's Guide to the Galaxy.",
"points": 0
},
{
"id": "c1l2",
"title": "Partially correct",
"points": 0.5,
"description": "Mentions the number 42 but not the Hitchhiker's Guide to the Galaxy."
},
{
"id": "c1l3",
"title": "Correct",
"points": 1,
"description": "Mentions the number 42 and the Hitchhiker's Guide to the Galaxy."
}
]
}
]
}
}
],
"permissions": [
{
"actor": "{userId}",
"role": "editor"
}
],
"metadata": {
"my_service_internal_id": "abc1234"
}
}
```
The actor ID above should be the Examplary user ID that you created for this customer. This is important, as the user needs to have access to the exam to be able to grade it.
You can specify an `id` on each question and use that same value to reference the question in subsequent steps.
### 3. Add an assessment session
Next, you'll want to import answers for a specific student, so they can be reviewed ([API reference](/developers/rest-api/results/post-exams-id-sessions)):
```json title="POST /exams/{id}/sessions"
{
"studentName": "John Doe",
"answers": [
{
"question": "my_question_1",
"value": "42"
}
],
"autoGrade": true
}
```
### 4. Create an embed session
Call the Examplary API to create a new embed session. You can configure presets for the question, as well as theme options.
The `actor` field should contain the ID of the Examplary user account you created for this customer.
Specify an `allowedOrigin` to receive `postMessage` updates as the user interacts with the grading UI. This flow has no "completion" event — each time the grade changes, the session emits another `started` update — so `returnUrl` has no effect here; use `allowedOrigin`/`postMessage` or poll the exam session for the latest answer instead.
```json title="POST /embed-sessions"
{
"flow": "grade-answer",
"actor": "{userId}",
"presets": {
"examId": "{examId}",
"sessionId": "{sessionId}",
"questionId": "my_question_1"
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"my_service_internal_id": "abc1234"
},
"allowedOrigin": "https://app.example.com"
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 5. Lead the user to the embed URL
You can either redirect the user directly to the URL, or display it in an `iframe`.
The latter might be better for the user experience, especially when displayed as a modal. This also allows you to listen to status updates in real time, through the `postMessage()` API.
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
window.addEventListener("message", (event) => {
// Make sure the message is coming from a trusted origin
if (event.origin !== new URL(embedUrl).origin) return;
// Handle the message
const { type, status, outputs } = event.data;
if (type === "examplary:embed-status-update") {
console.log("Graded answer update:", outputs);
}
});
```
---
// File: embed-sessions/flows/edit-exam
# Edit exam flow
Using the Examplary edit exam flow, you can embed the full exam editor — including the questions list, the inline question editor, and the AI-powered "add question" composer — directly into your application. This is useful when you want users to refine an exam that was previously generated (for example via the [exam generation flow](/developers/embed-sessions/flows/exam-generation)) without sending them out to the Examplary dashboard.
Unlike the generation flows, the edit exam flow does not have a defined "completion" event: users keep editing for as long as the embed is open, and every change is persisted in real time to the underlying exam. When you no longer need the embed, simply close the iframe and (optionally) revoke the session.
## Presets
| Key | Type | Description |
| -------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------- |
| `examId` | string | The ID of the exam that should be loaded in the editor. Must start with `exam_`. Required. The actor user must have at least `editor` permissions on this exam. |
## Example
### 1. Create an Examplary user for your user
To make sure we can save personal preferences and changes to the user's specific account, and can later allow them to reference these personal details, we require creating a user account in your workspace for each of your users.
```json title="POST /users"
{
"email": "my-user@example.com",
"name": "My User"
}
```
Store the returned user ID in your system to use with any future embed sessions for that user.
### 2. Make sure an exam exists and the actor can edit it
The edit exam flow operates on an existing exam. If you don't have one yet, create it first ([API reference](/developers/rest-api#exams-create)) — and make sure to give the actor user `editor` (or higher) permissions on the exam, otherwise they won't be able to make changes from within the embed:
```json title="POST /exams"
{
"name": "Midterm Exam - Algebra",
"subject": "Mathematics",
"studentLevel": "us_high_school_lower",
"questions": [],
"permissions": [
{
"actor": "{userId}",
"role": "editor"
}
],
"metadata": {
"my_service_internal_id": "abc1234"
}
}
```
If you're feeding the embed an exam that was just produced by the exam generation flow, you can pass the exam ID returned in its `outputs` straight through.
### 3. Create an embed session
Call the Examplary API to create a new embed session for the `edit-exam` flow. The only required preset is `examId` — the ID of the exam to load in the editor.
The `actor` field should contain the ID of the Examplary user account you created for this customer.
```json title="POST /embed-sessions"
{
"flow": "edit-exam",
"actor": "{userId}",
"presets": {
"examId": "{examId}"
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"my_service_internal_id": "abc1234"
}
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 4. Lead the user to the embed URL
You can either redirect the user directly to the URL, or display it in an `iframe`. The latter is usually the better experience, especially when displayed as a modal or side panel.
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
```
---
// File: embed-sessions/flows/practice-space-student
# Practice space (student) flow
Using the Examplary practice space student flow, you can embed the full student-facing practice experience for an existing practice space directly into your application. From within the embed, your users get the practice space landing screen (with topic progress, teacher notes, and the "start practicing" call to action) and the question-by-question player that runs inside a practice session.
This flow is intended for ongoing, open-ended use: students return to the embed as often as they want and there is no specific "completion" event. Each answer is persisted in real time, and you can read back the student's progress from the API at any time.
## Presets
| Key | Type | Description |
| ----------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `practiceSpaceId` | string | The ID of the practice space that should be opened for the student. Must start with `space_`. Required. The actor user must have at least `participant` permissions on this practice space. |
## Example
### 1. Create an Examplary user for your student
Each student needs an Examplary user account in your workspace. This is how we attribute progress, sessions, and proficiency to the individual student.
```json title="POST /users"
{
"email": "my-student@example.com",
"name": "My Student",
"role": "student"
}
```
Set `role` to `"student"` — it defaults to `"teacher"` if omitted, which creates a billable seat and counts against your workspace's `max-seats` limit instead of being free like student accounts.
Store the returned user ID in your system — you'll use it as the `actor` for the student's embed sessions, and the same ID will also identify the student inside the practice space.
### 2. Make sure a practice space exists and the actor can participate
The practice space student flow operates on an existing practice space. If you don't have one yet, create it first ([API reference](/developers/rest-api#practice-spaces-create)). Practice spaces are typically created by a teacher / admin user in your workspace, not by the student themselves.
The student user needs at least the `participant` role on the space, otherwise they won't be able to load it inside the embed:
```json title="POST /practice-spaces"
{
"name": "Algebra practice",
"studentNotes": "Work through each topic at your own pace. Ask your teacher if you get stuck!",
"sourceMaterialIds": [
"material_abc123"
],
"masteryTopics": [
{
"id": "linear-equations",
"name": "Linear equations"
},
{
"id": "quadratic-equations",
"name": "Quadratic equations"
},
{
"id": "polynomials",
"name": "Polynomials"
}
],
"settings": {
"showTopicProgress": true,
"allowCustomPrompt": true
},
"permissions": [
{
"actor": "{userId}",
"role": "participant"
}
],
"metadata": {
"my_service_internal_id": "abc1234"
}
}
```
The `studentNotes` field is optional. When set, it appears as "teacher notes" on the practice space landing screen inside the embed — a good place to leave guidance or context for your students.
The `sourceMaterialIds` field specifies which source materials the practice space generates questions from. Without source materials, no questions will be generated for students to practice with.
The `masteryTopics` field is optional. It is an array of objects with `id` (any unique string you choose) and `name` (displayed to students) fields, defining the topics available for practice. If omitted, topics are automatically generated from the practice space name and attached source materials — this happens asynchronously and may take a moment after creation.
You can grant `participant` access to multiple students on the same practice space — each will get their own progress, sessions, and proficiency tracking when they open the embed under their own actor ID.
#### Available settings
| Setting | Type | Default | Description |
| --------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------ |
| `showHints` | boolean | `true` | Whether to offer AI-generated hints when a student is stuck on a question. |
| `showPoints` | boolean | `false` | Whether to show the question's point value to the student, in the question header and in the AI feedback view. |
| `allowCustomPrompt` | boolean | `true` | Whether students can enter a custom topic or question to practice with. |
| `showTopicProgress` | boolean | `true` | Whether to show the student's progress per topic on the landing screen. |
| `showProficiencyBadge` | boolean | `true` | Whether to award a proficiency badge once the student meets the proficiency thresholds. |
| `proficiencyPercentage` | number | `80` | Score percentage a student must achieve (across `proficiencyQuestionsCount` questions) to earn proficiency. |
| `proficiencyQuestionsCount` | number | `10` | Minimum number of questions a student must answer to be eligible for a proficiency badge — correctness is judged separately, via `proficiencyPercentage`. |
| `proficiencyDaysCount` | number | `2` | Minimum number of distinct days (not necessarily consecutive) the student must have answered questions on before they can earn the proficiency badge. |
### 3. Create an embed session
Call the Examplary API to create a new embed session for the `practice-space-student` flow. The only required preset is `practiceSpaceId` — the ID of the practice space to open.
The `actor` field should contain the ID of the Examplary user account you created for the student.
```json title="POST /embed-sessions"
{
"flow": "practice-space-student",
"actor": "{userId}",
"presets": {
"practiceSpaceId": "{practiceSpaceId}"
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"my_service_internal_id": "abc1234"
}
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 4. Lead the student to the embed URL
You can either redirect the student directly to the URL, or display it in an `iframe`. Embedding inside an iframe is usually preferable so the student stays inside your application's navigation.
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
```
### 5. Read back student progress
Retrieve a student's progress at any time using the practice space students endpoint ([API reference](/developers/rest-api#practice-spaces-students-get)):
```
GET /practice-spaces/{practiceSpaceId}/students/{userId}
```
The `userId` is the Examplary user ID of the student. The response includes their topic progress, session count, and total questions answered.
---
// File: embed-sessions/flows/take-exam
# Take exam (student) flow
Using the Examplary take exam flow, you can embed the full student-facing exam experience for an existing exam directly into your application. From within the embed, your students can take the exam and submit their answers.
This flow is intended for a single exam session: the student takes the exam once, answers all questions, and submits. Each answer is persisted in real time.
## Presets
| Key | Type | Description |
| -------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| `examId` | string | The ID of the exam that should be taken by the student. Must start with `exam_`. Required. The actor user must have access to take this exam. |
## Example
### 1. Create an Examplary user for your student
Each student needs an Examplary user account in your workspace. This is how we attribute answers and sessions to the individual student.
```json title="POST /users"
{
"email": "my-student@example.com",
"name": "My Student",
"role": "student"
}
```
Set `role` to `"student"` — it defaults to `"teacher"` if omitted, which creates a billable seat and counts against your workspace's `max-seats` limit instead of being free like student accounts.
Store the returned user ID in your system — you'll use it as the `actor` for the student's embed sessions.
### 2. Make sure an exam exists and the actor can take it
The take exam flow operates on an existing exam. If you don't have one yet, create it first ([API reference](/developers/rest-api#exams-create)).
The student user must be given the `participant` role on the exam — this is what allows them to take it and submit answers. This only applies while the exam is unpublished (its default state right after creation), or if your workspace has the "block public sharing" setting enabled: a published exam can otherwise be opened by anyone, participant permission or not, so keep new exams unpublished until every student's access is set up.
```json title="POST /exams"
{
"name": "Midterm Exam - Algebra",
"subject": "Mathematics",
"questions": [],
"permissions": [
{
"actor": "{userId}",
"role": "participant"
}
],
"metadata": {
"my_service_internal_id": "abc1234"
}
}
```
### 3. Create an embed session
Call the Examplary API to create a new embed session for the `take-exam` flow. The only required preset is `examId` — the ID of the exam to take.
The `actor` field should contain the ID of the Examplary user account you created for the student.
Specify a `returnUrl`, an `allowedOrigin`, or both, based on how you want to be notified when the student finishes.
```json title="POST /embed-sessions"
{
"flow": "take-exam",
"actor": "{userId}",
"presets": {
"examId": "{examId}"
},
"theme": {
"primaryColor": "#4f46e5",
"locale": "en"
},
"metadata": {
"my_service_internal_id": "abc1234"
},
"returnUrl": "https://app.example.com",
"allowedOrigin": "https://app.example.com"
}
```
The response is the new embed session. Two of its fields matter for the next steps: `embedUrl`, the page to show your user, which stays valid until the session's `expiresAt`, and `id`, to [look up the session](/developers/rest-api/embed-sessions/get-embed-sessions-id) later.
### 4. Lead the student to the embed URL
You can either redirect the student directly to the URL, or display it in an `iframe`. Embedding inside an iframe is usually preferable so the student stays inside your application.
If you specified an `allowedOrigin`, you can listen for status updates via the `postMessage` API:
```ts
const embedUrl = "https://app.examplary.ai/embeds/55S843D7HfNf...";
iframe.src = embedUrl;
window.addEventListener("message", (event) => {
// Make sure the message is coming from a trusted origin
if (event.origin !== new URL(embedUrl).origin) return;
const { type, status, outputs } = event.data;
if (type === "examplary:embed-status-update") {
if (status === "started") {
// Fired each time the student answers a question
console.log("Question answered");
} else if (status === "completed") {
if (outputs.status === "timed-out") {
// The exam time limit was reached
console.log("Time's up — exam auto-submitted");
} else if (outputs.status === "completed") {
// The student clicked "Hand in"
console.log("Exam submitted by student");
}
}
}
});
```
If you're using redirects instead, you can expect a redirect to your specified `returnUrl` on completion, with the embed session ID and the outcome folded into the query string as `status` — the same value as `outputs.status` above, not the top-level `completed` status: `?status=completed&embedId=embed_session_55S8...` or `?status=timed-out&embedId=embed_session_55S8...`
The following status values are emitted during a take-exam session:
| Status | Description |
| ----------- | --------------------------------------------------------------------- |
| `started` | Fired each time the student answers a question |
| `completed` | The student handed in the exam, or the session reached the time limit |
When `status` is `completed`, the `outputs.status` field indicates how the session ended:
| `outputs.status` | Description |
| ---------------- | -------------------------------------------------------- |
| `completed` | The student clicked the "Hand in" button |
| `timed-out` | The exam time limit was reached and the session auto-submitted |
### 5. Cleanup
The embed session will expire automatically after 7 days, but because it gives some limited access to your account, you might want to remove it manually:
```
DELETE /embed-sessions/{embedSessionId}
```
---
// File: grading
# Grading with Examplary
Use Examplary's AI grading in your own platform: send in student answers, get back scores and feedback per rubric criterion.
Examplary's grading API brings the AI grading teachers use in Examplary to your own platform. You send in your students' answers, and get back a score for each answer, broken down per rubric criterion, with the reasoning behind it and feedback for the student. Open questions and essays are graded by AI; questions with a single correct answer are graded automatically.
## How it works
```mermaid
flowchart LR
materials[Source materials] --> exam
questions[Questions and rubrics] --> exam
calibration[Calibration answers] --> exam
exam[Exam] --> grading
answers[Student answers] --> grading
grading[AI grading] --> results[Scores and feedback]:::highlight
```
Everything is organised around an **exam**, which holds what the AI needs to grade well:
- **[Questions and rubrics](/developers/grading/advanced/rubrics).** Every question has a rubric that describes what earns points. This is the one thing every exam needs.
- **[Source materials](/developers/grading/advanced/source-materials).** Optionally, the course materials the exam is based on, so answers are graded against what students were actually taught.
- **[Calibration answers](/developers/grading/advanced/calibration).** Optionally, answers your teachers have graded before, to show the AI how strictly to apply the rubric.
You then import your students' answers as **assessment sessions**, one per student. Examplary grades them in the background, and you collect the results by polling the API or through a [webhook](/developers/guides/webhooks).
The AI's grades come back as suggestions. You can use them as they are, have a teacher review them first, or let teachers do that in Examplary itself.
## Getting started
The [developer quickstart](/developers/grading/guide) walks you through the whole flow with requests you can run straight from the docs: creating an exam, adding questions, importing answers and getting the results.
If you'd rather let teachers review and adjust grades without building that interface yourself, the [grade answer flow](/developers/embed-sessions/flows/grade-answer) embeds Examplary's grading screen in your platform.
## Billing
Grading is billed per answer graded. Each graded answer uses 1 credit, plus 1 extra credit for every additional 250 words, or for every additional page of a scanned answer.
| Answer | Credits |
| -------------------------------------------------- | ------- |
| Minimum per graded answer | 1 |
| Every additional 250 words of text in the answer | 1 |
| Every additional scanned or PDF page of the answer | 1 |
---
// File: grading/guide
# Developer quickstart
This guide shows how to quickly get started with creating exams, adding questions, importing student answers,
and receiving auto-grading results in Examplary.
## Step 1: Create an exam
Exams are our container for all answers and grading. Create one in all cases, even when grading
just a single question. This allows you to specify [source materials](/developers/grading/advanced/source-materials) and other context that is needed to
accurately grade the answers.
Use the [Create exam](/developers/rest-api/exams/post-exams) endpoint to create the exam you want to grade in Examplary:
```json title="POST /exams"
{
"name": "Book review",
"subject": "English Literature",
"studentLevel": "us_undergraduate"
}
```
## Step 2: Add questions
You can either add questions in Examplary-format, or import QTI questions (either from XML or from a .zip IMS package).
```json title="POST /exams/{id}/questions"
[
{
"externalId": "my_question_1",
"type": "single-line-text",
"title": "What is the meaning of life?",
"description": "Give a short answer to the ultimate question.",
"settings": {},
"scoring": {
"rubricType": "analytical",
"criteria": [
{
"id": "c1",
"title": "Correctness",
"levels": [
{
"id": "c1l1",
"title": "Incomplete",
"description": "Does not mention the number 42 or the Hitchhiker's Guide to the Galaxy.",
"points": 0
},
{
"id": "c1l2",
"title": "Partially correct",
"points": 0.5,
"description": "Mentions the number 42 but not the Hitchhiker's Guide to the Galaxy."
},
{
"id": "c1l3",
"title": "Correct",
"points": 1,
"description": "Mentions the number 42 and the Hitchhiker's Guide to the Galaxy."
}
]
}
]
}
}
]
```
Give each question an `externalId` that you will use later to submit answers.
```json title="POST /exams/{id}/questions/import"
{
"type": "text/xml",
"content": "What is the meaning of life?"
}
```
The `identifier` of each `qti-item` will be saved as `externalId`, which you can later use to reference the question.
```json title="POST /exams/{id}/questions/import"
{
"type": "application/zip",
"url": "https://examplary.ai/downloads/example-qti.zip"
}
```
The `identifier` of each `qti-item` will be saved as `externalId`, which you can later use to reference the question.
Optionally, further [calibrate the AI grading](/developers/grading/advanced/calibration) by uploading previously scored answers.
## Step 3: Import answers
After adding the questions, we can start importing student answers. The API refers to these as 'assessment sessions'.
```json title="POST /exams/{id}/sessions"
{
"externalId": "my_student_1",
"studentName": "John Doe",
"answers": [
{
"question": "my_question_1",
"value": "42"
}
],
"autoGrade": true
}
```
This endpoint also accepts an array of assessment sessions, allowing you to import multiple students' answers in a single request.
You can reference questions both by their ID or by their `externalId`.
## Step 4: Receive auto-grading results
By setting `autoGrade` to `true` in the assessment session request, the system will automatically start grading the submitted answers.
To retrieve the results, you can either poll the assessment sessions endpoint or use a webhook to receive real-time updates.
```json title="POST /webhooks"
{
"url": "https://example.com/webhook",
"events": [
"exam.session.grading.completed"
],
"resources": [
"{examId}"
]
}
```
Make sure you register this webhook before you start importing assessment sessions, so you don't miss any grading updates.
Also note that this only fires when there are any AI-gradable questions in the session you've submitted. Machine-graded questions
(like multiple choice) will not trigger this webhook, as they are graded instantly.
It's recommended to [delete webhooks](/developers/rest-api/webhooks/delete-webhooks-id) after they are no longer needed.
Read the [webhooks guide](/developers/guides/webhooks) for the payload format, how to verify that requests come from Examplary, and how retries work.
We recommend polling every 30 seconds. A graded answer will have `"gradingStatus": "auto-graded"` (AI graded) or `"gradingStatus": "graded"` (deterministically graded).
```json title="GET /exams/{id}/sessions/{sessionId}"
```
We recommend polling every 30 seconds. A graded answer will have `"gradingStatus": "auto-graded"` (AI graded) or `"gradingStatus": "graded"` (deterministically graded).
```json title="GET /exams/{id}/sessions"
```
---
// File: grading/advanced/rubrics
# Rubrics & rubric types
How rubrics are structured, how each rubric type is scored, and how grading results come back from the API.
Every question is graded against a rubric, stored in the question's `scoring` object. The rubric describes what earns points, and Examplary's auto-grading AI applies it to each answer: it picks the criteria or levels that fit, awards points, and explains its reasoning per criterion.
There are four rubric types, set with `rubricType`:
| Rubric type | How it's scored | Maximum points | Graded by |
| -------------- | ------------------------------------------------------------- | ------------------------------------- | --------- |
| `simple` | Each criterion that applies adds its points. | The sum of all criteria | AI |
| `analytical` | One level is picked per criterion; the levels' points add up. | The sum of each criterion's top level | AI |
| `holistic` | One criterion is picked for the answer as a whole. | The highest criterion | AI |
| `exact-values` | The answer must match one of the accepted values exactly. | The highest criterion | Automatic |
The full structure is available as a [JSON Schema](https://schemas.examplary.ai/question-scoring.json), which you can use to validate rubrics or generate types in your own system. It's also published on npm as part of [`@examplary/schemas`](https://www.npmjs.com/package/@examplary/schemas).
## The scoring object
```json title="scoring"
{
"rubricType": "analytical",
"criteria": [ … ],
"guidance": "Accept answers that mention the number 42 without the book title.",
"modelAnswer": "42, the answer from The Hitchhiker's Guide to the Galaxy."
}
```
| Field | Type | Required | Description |
| ---------------------------- | ------- | -------- | ----------------------------------------------------------------------------- |
| `rubricType` | string | No | `simple` (default), `analytical`, `holistic` or `exact-values`. |
| `criteria` | array | **Yes** | The rubric's criteria. Their shape depends on the rubric type, see below. |
| `guidance` | string | No | Extra grading instructions. The AI takes these into account for every answer. |
| `modelAnswer` | string | No | An example of a full-marks answer. The AI uses it as a reference. |
| `exactValuesCaseInsensitive` | boolean | No | Only for `exact-values`: ignore capitalisation and whitespace when matching. |
| `excludeFromGrading` | boolean | No | Leaves the question out of the session's total points. It's still graded. |
| `templateId` | string | No | The workspace rubric template this rubric was created from, if any. |
Each criterion, and each level of an analytical criterion, has these fields:
| Field | Type | Required | Description |
| ------------- | ------ | ------------------------------------ | ---------------------------------------------------------------------------------------------- |
| `id` | string | **Yes** | A unique ID. You'll find it again in the grading results, so pick something you can recognise. |
| `title` | string | `analytical`, `holistic` | A short name. Used for analytical criteria and levels, and holistic criteria. |
| `description` | string | `simple`, `holistic`, `exact-values` | What an answer needs to earn the points. For `exact-values`, the accepted value itself. |
| `points` | number | **Yes** | The points awarded when the criterion or level applies. |
| `minPoints` | number | No | Optional lower bound, when the criterion or level allows a range of points. |
| `maxPoints` | number | No | Optional upper bound for that range. |
| `levels` | array | No | Only for `analytical` criteria: the levels to choose from, each with the fields above. |
Examplary tidies up rubrics when you save a question: missing `id`s are
generated, fields that don't apply to the rubric type are dropped (such as
`levels` outside analytical rubrics, or `title` on simple criteria), and
analytical levels and holistic criteria are sorted by points. Read the
question back after saving if you need the final shape.
## Simple
A checklist: each criterion describes something the answer should contain, and every criterion that applies adds its points. Use it for answers that need to cover several separate points.
```json title="scoring (simple)"
{
"rubricType": "simple",
"criteria": [
{
"id": "mentions-42",
"description": "Mentions the number 42.",
"points": 1
},
{
"id": "names-book",
"description": "Names The Hitchhiker's Guide to the Galaxy.",
"points": 1
},
{
"id": "explains-joke",
"description": "Explains why the answer is a joke.",
"points": 2
}
]
}
```
The maximum is the sum of all criteria, 4 points here. In the results, every criterion gets its own entry with the points the AI awarded for it:
```json title="Answer (simple)"
{
"gradingStatus": "auto-graded",
"gradedBy": "system-ai",
"pointsAwarded": 2,
"feedback": "
You found the answer, but didn't say where it comes from.
",
"scoringCriteria": [
{
"id": "mentions-42",
"pointsAwarded": 1,
"aiPointsAwarded": 1,
"confidenceScore": 95,
"reasoning": "States 42 directly."
},
{
"id": "names-book",
"pointsAwarded": 0,
"aiPointsAwarded": 0,
"confidenceScore": 90,
"reasoning": "No reference to the book."
},
{
"id": "explains-joke",
"pointsAwarded": 1,
"aiPointsAwarded": 1,
"confidenceScore": 60,
"reasoning": "Hints at the joke but doesn't explain it."
}
]
}
```
## Analytical
A grid: each criterion is a dimension of the answer, such as correctness or argumentation, with levels from weak to strong. The AI picks exactly one level per criterion, and the levels' points add up to the score. Analytical criteria only have a `title` and `levels`; the points live on the levels.
```json title="scoring (analytical)"
{
"rubricType": "analytical",
"criteria": [
{
"id": "correctness",
"title": "Correctness",
"levels": [
{
"id": "correctness-0",
"title": "Incorrect",
"description": "Doesn't mention 42.",
"points": 0
},
{
"id": "correctness-1",
"title": "Partially correct",
"description": "Mentions 42, but not the book.",
"points": 1
},
{
"id": "correctness-2",
"title": "Correct",
"description": "Mentions 42 and the book.",
"points": 2
}
]
},
{
"id": "clarity",
"title": "Clarity",
"levels": [
{ "id": "clarity-0", "title": "Unclear", "points": 0 },
{ "id": "clarity-1", "title": "Clear", "points": 1 }
]
}
]
}
```
The maximum is the sum of each criterion's highest level, 3 points here. In the results, each criterion's entry also says which level was picked, in `selectedLevelId`:
```json title="Answer (analytical)"
{
"gradingStatus": "auto-graded",
"gradedBy": "system-ai",
"pointsAwarded": 2,
"feedback": "
Correct number! Mention where it comes from for full marks.
",
"scoringCriteria": [
{
"id": "correctness",
"selectedLevelId": "correctness-1",
"aiSelectedLevel": "correctness-1",
"pointsAwarded": 1,
"aiPointsAwarded": 1,
"confidenceScore": 85,
"reasoning": "Mentions 42, but not the book."
},
{
"id": "clarity",
"selectedLevelId": "clarity-1",
"aiSelectedLevel": "clarity-1",
"pointsAwarded": 1,
"aiPointsAwarded": 1,
"confidenceScore": 90,
"reasoning": "A short, direct answer."
}
]
}
```
## Holistic
A single scale for the answer as a whole: each criterion describes one overall quality, and the AI picks the one that fits best. Use it when an answer is best judged in one go, like a short essay.
```json title="scoring (holistic)"
{
"rubricType": "holistic",
"criteria": [
{
"id": "insufficient",
"title": "Insufficient",
"description": "Misses the point of the question.",
"points": 0
},
{
"id": "adequate",
"title": "Adequate",
"description": "Gives the right answer without context.",
"points": 2
},
{
"id": "excellent",
"title": "Excellent",
"description": "Gives the answer and explains its origin.",
"points": 4
}
]
}
```
The maximum is the highest criterion, 4 points here. In the results, every criterion has an entry, but only the picked one has points; the others are `0`:
```json title="Answer (holistic)"
{
"gradingStatus": "auto-graded",
"gradedBy": "system-ai",
"pointsAwarded": 2,
"scoringCriteria": [
{
"id": "insufficient",
"pointsAwarded": 0,
"aiPointsAwarded": 0,
"confidenceScore": 90,
"reasoning": "…"
},
{
"id": "adequate",
"pointsAwarded": 2,
"aiPointsAwarded": 2,
"confidenceScore": 80,
"reasoning": "Right answer, no context."
},
{
"id": "excellent",
"pointsAwarded": 0,
"aiPointsAwarded": 0,
"confidenceScore": 85,
"reasoning": "…"
}
]
}
```
To find the picked criterion, look for the entry with points. If a teacher changes the grade in Examplary, the results instead hold just the picked criterion, with `selectedLevelId` set to its own `id`.
## Exact values
For answers with a known correct value, like fill-in-the-blank questions. Each criterion's `description` is an accepted value, with the points it's worth. There's no AI involved: the answer is graded straight away when it's saved.
```json title="scoring (exact-values)"
{
"rubricType": "exact-values",
"exactValuesCaseInsensitive": true,
"criteria": [
{ "id": "full", "description": "42", "points": 2 },
{ "id": "written-out", "description": "forty-two", "points": 1 }
]
}
```
The first matching criterion wins, and the maximum is the highest criterion, 2 points here. Answers are compared after trimming surrounding whitespace. With `exactValuesCaseInsensitive`, capitalisation and all whitespace are ignored too, so `"New York"` matches `"newyork"`. See [exact-value matching](/developers/apps/question-types/grading#exact-value-matching) for the details.
The results are final straight away, and hold the matching criterion only:
```json title="Answer (exact-values)"
{
"gradingStatus": "graded",
"gradedBy": "system-exact-values",
"pointsAwarded": 2,
"scoringCriteria": [
{ "id": "full", "selectedLevelId": "full", "pointsAwarded": 2 }
]
}
```
An answer that matches nothing gets `"pointsAwarded": 0` and an empty `scoringCriteria`.
## Changing a rubric later
When you [update a question](/developers/rest-api/exam-questions/patch-exams-id-questions-questionid) and its `scoring` changes, Examplary re-grades the answers that were graded automatically, in the background. Grades a teacher has set or given feedback on are left alone, and so are answers that were never graded. Re-graded answers go back to pending, so if you use the `exam.session.grading.completed` webhook, expect it to fire again once the new results are in. Only toggling `excludeFromGrading` doesn't trigger a re-grade.
## Reading the results
Grading results are part of each answer in the assessment session, which you get from the [sessions endpoints](/developers/rest-api/results/get-exams-id-sessions-sessionid) or the `exam.session.grading.completed` webhook. See [step 4 of the developer guide](/developers/grading/guide#step-4-receive-auto-grading-results) for how to fetch them.
| Field | Description |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `gradingStatus` | `auto-grading-pending`, `auto-graded` (an AI suggestion, not yet reviewed), `auto-grading-failed`, `graded` (final), or `not-graded`. |
| `gradedBy` | `system-ai`, `system-exact-values`, `system-response-processing` (graded by the question type), or the ID of the teacher who graded it. |
| `pointsAwarded` | The answer's points, rounded to one decimal. |
| `pointsToEarn` | The question's maximum points. |
| `feedback` | Feedback for the student, as HTML (see below). Not always present. |
| `scoringCriteria` | The score per criterion, matched to your rubric by `id`. See below. |
| `gradedAt` | When the answer was graded. |
Each entry in `scoringCriteria` has:
| Field | Description |
| ----------------- | ----------------------------------------------------------------------------------------------------- |
| `id` | The criterion's `id` from the rubric. |
| `pointsAwarded` | The points for this criterion. |
| `selectedLevelId` | The picked level, for analytical criteria (and the matched value, for exact values). |
| `reasoning` | The AI's explanation of the score, as HTML (see below). Useful to show to teachers next to the grade. |
| `confidenceScore` | How sure the AI is, from 0 to 100. |
| `aiPointsAwarded` | What the AI originally awarded. Stays the same when a teacher changes `pointsAwarded`. |
| `aiSelectedLevel` | The level the AI originally picked, for analytical criteria. |
Both `feedback` and `reasoning` can contain HTML formatting, plus two custom tags: `` for formulas, and `` to quote part of the student's answer. Its `quote` attribute holds the exact text it refers to:
```html
The student mentioned "42" but
did not name the book.
```
When showing these in your own interface, you can highlight the quoted text in the answer, or just render the tag's content as plain text.
The AI's grades are suggestions, so they come back as `auto-graded` and count towards the session's `points` but not its final `grade`. A teacher can review them in Examplary, or you can finalise them yourself: [accept a suggestion](/developers/rest-api/results/post-exams-id-sessions-sessionid-suggestions-questionid-accept) as it is, [accept all of a session's suggestions](/developers/rest-api/results/post-exams-id-sessions-sessionid-suggestions-accept-all), or [save your own grade](/developers/rest-api/results/post-exams-id-sessions-sessionid-feedback). Either way the answer becomes `graded`, and a late AI result never overwrites it.
Clear, specific criterion and level descriptions make the biggest difference.
Add `guidance` for anything that applies across criteria, and a `modelAnswer`
to show what full marks looks like. To teach the AI your own grading style,
[add previously graded answers](/developers/grading/advanced/calibration) as
examples.
---
// File: grading/advanced/source-materials
# Grading with source materials
Attach course materials to an exam, so the AI grades answers against what your students were actually taught.
Without source materials, the AI grades answers with its own general knowledge and the rubric. When you attach the materials a course is based on, such as a textbook chapter, lecture slides as PDF or a recorded lecture, Examplary looks up what they say about each question and takes that into account when grading. Answers are then judged against what students were taught, including definitions, terminology and examples that are specific to your course.
Adding source materials takes two steps: create a source material from a file, and attach it to the exam.
## Step 1: Create a source material
Source materials are created from a file's URL. Any public HTTPS URL works, and Examplary keeps its own copy of the file, so the original link doesn't have to stay online. If your file isn't online, [upload it to Examplary](/developers/guides/uploading-files) first.
```json title="POST /source-materials"
{
"url": "https://example.com/course-101/chapter-3.pdf",
"contentType": "application/pdf",
"externalId": "course-101-chapter-3",
"tags": [
"course-101"
]
}
```
These file types are supported:
- **Documents:** PDF, DOCX, TXT and HTML
- **Images:** PNG and JPEG
- **Audio:** such as MP3, WAV, M4A, AAC, OGG and FLAC
- **Video:** such as MP4, MOV, MPEG, AVI and WebM
Examplary uses `contentType` to tell what kind of file it is, or the file extension in the URL if you leave it out. We recommend keeping files under 50 MB. Web pages work when their content is in the page's HTML: set `contentType` to `text/html`. Pages that only load their content with JavaScript, and video sites such as YouTube, aren't supported.
| Field | Type | Description |
| ------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------- |
| `url` | string | A public HTTPS URL of the file, such as one you [uploaded](/developers/guides/uploading-files). **Required.** |
| `contentType` | string | The file's content type, such as `application/pdf`. |
| `externalId` | string | Your own ID for the material. You can [list materials](/developers/rest-api/source-materials/get-source-materials) by it later. |
| `tags` | array | Up to 20 tags, to organise materials in Examplary. |
The response includes the material's `id`, which starts with `material_`.
## Step 2: Attach it to the exam
Link materials to an exam with its `sourceMaterialIds`. You can do this straight after creating them: there's no need to wait for them to be processed. Pass them when [creating the exam](/developers/rest-api/exams/post-exams), or set them later:
```json title="PATCH /exams/{id}"
{
"sourceMaterialIds": [
"{materialId}"
]
}
```
`sourceMaterialIds` replaces the exam's list of materials, so include all of them when you add one. The same materials are used for every question in the exam.
Examplary prepares what it needs from the materials when an exam is published
or its first answers are graded. Materials you attach after that may not be
used for the exam's questions, and answers that were already graded aren't
graded again. Attach all materials first, then import answers.
---
// File: grading/advanced/calibration
# Calibrating the AI
Teach the auto-grading AI how strictly to apply a rubric, using answers your teachers have already graded.
A rubric tells the AI what earns points, but not how strict to be. Is "42" without the book title worth half the points, or none?
Calibration answers settle questions like that: previously graded answers to the same question, which the AI uses as examples of
how your teachers apply the rubric.
The AI still grades every answer on its own merits. It uses the examples to judge how strictly to apply the rubric
and when to give partial credit.
## Adding calibration answers
Add calibration answers per question, after you've [created the exam and its questions](/developers/grading/guide)
and before you import the answers you want graded. Each one is an answer with the points a teacher gave it:
```json title="POST /exams/{id}/questions/{questionId}/calibration-answers"
[
{
"value": "42, from The Hitchhiker's Guide to the Galaxy.",
"pointsAwarded": 1,
"reasoning": "Names both the number and the book."
},
{
"value": "The answer is 42.",
"pointsAwarded": 0.5,
"reasoning": "Correct number, but doesn't say where it comes from."
},
{
"value": "To be happy.",
"pointsAwarded": 0,
"reasoning": "A personal answer, not the one from the book."
}
]
```
The endpoint takes an array of up to 1,000 answers per request, and returns `201 Created` without a body.
| Field | Type | Description |
| ----------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `value` | string or array | The answer, in the same format as `value` when [importing sessions](/developers/rest-api/results/post-exams-id-sessions). |
| `pointsAwarded` | number | The points the teacher gave. **Required.** |
| `pointsToEarn` | number | The maximum points the answer was graded out of. Only needed when that differs from the question's current maximum, see below. |
| `reasoning` | string | Why the answer got these points. Optional, but the most useful thing you can add. |
| `scoringCriteria` | array | The points per rubric criterion, in the same format as [grading results](/developers/grading/advanced/rubrics#reading-the-results). |
For example, an answer to the [analytical rubric](/developers/grading/advanced/rubrics#analytical) from the developer guide can say which level the teacher picked:
```json title="Calibration answer with scoringCriteria"
{
"value": "The answer is 42.",
"pointsAwarded": 0.5,
"scoringCriteria": [
{ "id": "c1", "selectedLevelId": "c1l2", "pointsAwarded": 0.5 }
],
"reasoning": "Correct number, but doesn't say where it comes from."
}
```
## Choosing good examples
A handful of well-chosen examples does more than a large pile of similar ones:
- **Cover the range.** Include weak, middling and strong answers. Answers in the middle are the most informative, since that's where partial credit gets decided.
- **Explain the score.** A short `reasoning` shows the AI what made the difference, which is more useful than the score alone.
- **Keep them representative.** Pick answers that look like the ones your students actually write, rather than unusually long or unusual ones.
Equally, however, our system will automatically choose the most representative examples for the grading operation from the
set you provide, so feel free to upload as many examples as you can.
## Different point scales
Calibration answers are usually graded out of the question's maximum points. If yours were graded on a different scale, for
example out of 10 in your own system, set `pointsToEarn` to that maximum. Examplary then rescales the score to the question's
current maximum, so `{ "pointsAwarded": 7, "pointsToEarn": 10 }` counts as 70% of the points. This also keeps calibration
answers useful if you change the question's points later.
Calibration answers can't be listed, changed or removed through the API, so
check them before you send them. Grades that teachers give in Examplary aren't
added as calibration answers automatically: only answers you add with this
endpoint are used.
---
// File: apps
# Building apps
Apps allow developers to extend Examplary with new features and integrations.
They connect Examplary to other platforms and extend it with new question types.
Workspace owners and admins can install apps from the _Apps & Integrations_
section in their workspace settings.
Our own integrations, like Google Classroom, Woots, Schoolyear and Safe Exam
Browser, are built as apps, using the same tools described here.
## The anatomy of an app
An app is a package that defines a set of **Capabilities**. These capabilities
determine what the app can do within Examplary, such as adding new question
types, integrating with external services, or providing custom dashboard pages.
An app is a folder with an `app.ts` file that describes it, and other files that
contain the actual logic and UI components of the app.
## Getting started
Create a new app with the Examplary CLI:
```shell
npx @examplary/cli init acme-gradebook --id acme.gradebook --name "Acme Gradebook"
```
This creates a folder with an `app.ts`, and a `package.json` with the Examplary
packages and scripts to run and publish the app.
Make sure that your `app.ts` has a `defineApp` call that defines your app and
exports it as the default export.
```ts title="app.ts"
import { defineApp, InstallationScope } from "@examplary/apps";
import exportOption from "./capabilities/export";
const app = defineApp({
id: "acme.gradebook",
name: { en: "Acme Gradebook" },
description: { en: "Send tests to your Acme classes" },
icon: "./icon.svg",
public: true,
});
app.addSetting({
key: "apiToken",
label: { en: "Acme API token" },
help: { en: "You'll find it in your Acme account settings." },
type: "password",
secret: true,
});
app.addCapability(exportOption);
export default app;
```
The `id` identifies your app on the platform and can't be changed later.
Use a reverse domain-like name, in lowercase: `.`.
To try it out in your workspace while you build it, run this:
```shell
npm run dev
```
To publish it to your organization, or submit it for public use, follow the
steps in the [publishing guide](/developers/apps/publishing).
## App properties
The `defineApp` function is used to define your app and its properties:
| Property | Description |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Unique ID, like `acme.gradebook`. Required. |
| `name` | Display name. Required. |
| `description` | One-line description, shown on the app's card. Required. |
| `icon` | Path to a square icon (SVG or PNG). |
| `readme` | Path to a Markdown file with a longer description. |
| `previewImage` | Path to a preview image. |
| `website` | Where to read more about the app. |
| `version` | Your own version number, like `1.2.0`. |
| `scope` | Who fills in the settings by default: the workspace (`org`) or each teacher (`user`). See [settings](/developers/apps/settings). |
| `public` | Whether every Examplary workspace can find and install the app. See [publishing](/developers/apps/publishing). |
| `deprecated` | Existing installations keep working, but the app can no longer be installed. |
| `requiredPlans` | The plans a workspace needs to install the app or change its settings: any of `free`, `pro` or `enterprise`. All plans when left out. |
Texts like `name` and `description` are translatable: give an object with a
value per language (`{ en: "…", nl: "…" }`). Paths are relative to the app's
root folder; the CLI uploads the files they point at.
## Directory structure
Apart from `app.ts`, you're free to organize your app however you like.
A typical app might look like this:
```
acme-gradebook/
├── app.ts
├── package.json
├── icon.svg
├── README.md
├── capabilities/
│ └── export.ts
├── components/
│ └── connect-button.tsx
└── lib/
└── acme-api.ts
```
`app.ts` can import from other files and npm packages, as long as they don't
rely on Node.js APIs. See [hooks](/developers/apps/building-blocks/hooks) for what's available.
---
// File: apps/question-types
# Question types
Examplary comes with a set of question types that can be used to create interactive tests. An [app](/developers/apps) can add its own question types, to extend the platform with new kinds of questions.
A question type is part of an app: once a workspace installs the app, its question types can be used in that workspace's tests.
## Process
Building a question type involves the following steps:
1. Write its [definition](/developers/apps/question-types/metadata) and add it to your app
2. Implement [components](/developers/apps/question-types/components) for various views (assessment, print, etc.)
3. Optionally configure AI [generation](/developers/apps/question-types/generation) and [grading](/developers/apps/question-types/grading)
4. [Publish](/developers/apps/publishing) the app to the Examplary platform
## Adding a question type to your app
Each question type is a `QuestionType` [capability](/developers/apps), usually defined in its own folder:
```ts title="short-answer/definition.ts"
import { QuestionType } from "@examplary/apps";
export default new QuestionType({
id: "com.acme.quiz.short-answer",
name: { en: "Short answer" },
description: { en: "A question that requires a short text response." },
icon: "./short-answer/icon.svg",
components: {
assessment: "./short-answer/component-assessment.tsx",
print: "./short-answer/component-print.tsx",
},
});
```
Your app then adds it in `app.ts`:
```ts title="app.ts"
import { defineApp } from "@examplary/apps";
import ShortAnswer from "./short-answer/definition";
const app = defineApp({
id: "com.acme.quiz",
name: { en: "Acme Quiz" },
description: { en: "Extra question types for your tests" },
});
app.addCapability(ShortAnswer);
export default app;
```
An app can add as many question types as you like. When you publish a new version of the app, its question types are updated with it.
## Directory structure
We recommend a folder per question type, with its definition, components, an `icon.svg`, and optionally a `README.md` and preview image:
```
acme-quiz/
├── app.ts
├── package.json
└── short-answer/
├── definition.ts
├── icon.svg
├── README.md
├── preview.png
├── component-assessment.tsx
└── component-print.tsx
```
All paths in a definition are relative to `app.ts`, like `./short-answer/icon.svg`.
## Examples
You can find the source code of the default question types in the [`default-questions-pack` repository](https://github.com/examplary/default-questions-pack). It's an app like any other, which comes pre-installed with Examplary.
It serves as a great reference for how to structure your own question types and what components you might need to implement, and is a good starting point for your own question types.
---
// File: apps/settings
# App settings
Settings are what gets filled in when an app is installed, like an API token
or the URL of a school's environment. Examplary shows them as a form on the
app's card, validates them, and passes them to every
[hook](/developers/apps/building-blocks/hooks) of the app.
```ts title="app.ts"
app.addSetting({
key: "environment",
label: { en: "Environment" },
type: "select",
options: [
{ value: "eu", label: { en: "Europe" } },
{ value: "us", label: { en: "United States" } },
],
default: "eu",
});
```
## Workspace and user settings
By default, apps are installed with workspace-level settings (`org` scope).
However, some apps might make more sense as an app an individual teacher
connects, using user-level settings. For such apps, you can define
`scope: "user"` in the `defineApp` call.
This will allow teachers to enable the app for their account after it has been
installed for the workspace.
Once you've set the app-level scope to "user", you can then define individual
settings with `scope: "user"` to make them user-specific.
The default scope for a setting is "org" - so leave it like that for things
like a school's API key, and use `scope: "user"` for things that are specific
to each teacher, like their own course ID.
## Field types
| Type | Shown as |
| ----------- | ---------------------------------------------------------- |
| `text` | A text input |
| `password` | A text input that hides what's typed |
| `url` | A URL input |
| `boolean` | A switch |
| `select` | A dropdown with the given `options` |
| `component` | Your own React component, see [below](#component-settings) |
Other properties of a setting:
| Property | Description |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `key` | The key its value is stored under, like `apiToken`. Required. |
| `label` | The field's label. A setting without a label isn't shown: use these for values your hooks set. |
| `help` | Help text shown under the field. |
| `placeholder` | Placeholder text. |
| `default` | The initial value, a string or (for `boolean`) a boolean. |
| `schema` | A [JSON schema](https://json-schema.org) the value has to match, for anything its `type` doesn't check, like `{ type: "string", pattern: "^acme_" }`. A [zod](https://zod.dev) schema works too. |
| `optional` | Whether the field may be left empty. Fields are required by default. |
| `secret` | Store the value, but never send it back to the browser. Use this for passwords and tokens. |
| `scope` | `org` or `user`, see above. |
## Checking and preparing settings
Before saving, Examplary checks each value against its setting's `type` (a
non-empty string, a valid URL, a boolean, or one of a select's options) and
its `schema`, if it has one. To check more, like whether an API token actually
works, add a validate step. It gets the settings and has to return `true`;
throw an error to show its message to the person filling in the form:
```ts title="app.ts"
app.addSettingsValidateStep(async (settings) => {
const res = await fetch("https://api.acme.com/me", {
headers: { Authorization: `Bearer ${settings.apiToken}` },
});
if (!res.ok) {
throw new Error("Acme didn't accept this API token.");
}
return true;
});
```
A prepare step runs before that, and can turn what was filled in into what
you'd like to store. It returns the settings to save, which are then checked
as usual. A common use is swapping a one-time sign-in code for a long-lived
token, which you then store in a `secret` setting without a label:
```ts title="app.ts"
app.addSetting({
key: "refreshToken",
type: "text",
secret: true,
});
app.addSettingsPrepareStep(async (settings) => ({
refreshToken: await exchangeCode(settings.authorizationCode),
}));
```
To clean up when a workspace removes your app, add an uninstall step. It gets
the workspace's settings; if it throws, the app stays installed and the admin
sees your message:
```ts title="app.ts"
app.addUninstallStep(async (settings) => {
await disconnectWorkspace(settings.apiToken);
});
```
## Component settings
When a form field isn't enough, like for a "Sign in with…" button that opens a
popup, a setting can render your own [component](/developers/apps/building-blocks/components):
```ts title="app.ts"
app.addSetting({
key: "authorizationCode",
label: { en: "Acme account" },
type: "component",
component: {
path: "./components/connect-button.tsx",
translations: {
connect: { en: "Connect your Acme account" },
},
},
optional: true,
});
```
The component gets `setValue(key, value)` to set the value of any of the app's
settings, and `submit()` to save them. A card with a component setting has no
save button of its own, so the component has to call `submit()`:
```tsx title="components/connect-button.tsx"
import { Button, FrontendSettingsFieldComponent } from "@examplary/ui";
const ConnectButton: FrontendSettingsFieldComponent = ({
t,
setValue,
submit,
}) => (
);
export default ConnectButton;
```
---
// File: apps/publishing
# Publishing apps
Apps are published with the [Examplary CLI](https://www.npmjs.com/package/@examplary/cli). It builds your app, uploads its files, and publishes the new version, which workspaces then use right away.
## Acquire an API key
You'll need the API key of an admin of your workspace. You'll find it on the settings page of your Examplary account. You can also provide it through the `EXAMPLARY_API_KEY` environment variable, instead of passing `--key` every time.
## Trying out your app
While you work on your app, run this in its folder:
```shell
npx @examplary/cli dev
```
This publishes your own private copy of the app, installs it in your workspace, and opens Examplary in your browser. Whenever you save a file, the copy is updated. While it's installed, it takes the place of the app itself in your workspace, and its hooks get the app's [secrets](/developers/apps/building-blocks/hooks#secrets).
The developer toolbar in the browser shows whether it's still connected to `dev`, with a button to reload once something changed. Its **Dev app** panel shows your app's capabilities, settings and hooks, and the latest calls of its hooks, with their logs.
Press Ctrl + C to stop: the copy is then removed from your workspace.
## Validating your app
To check your app without publishing it, run this in its folder:
```shell
npx @examplary/cli validate
```
This loads `app.ts`, checks the app's definition, and bundles its hooks and components to make sure they compile. For [question types](/developers/apps/question-types), it also runs the `responseProcessing.testCases` you defined, and warns when a component is larger than 100 kB. It's a good check to run on pull requests: inside GitHub Actions, warnings show up as annotations on the files.
To see exactly what would be uploaded, run `npx @examplary/cli package`. It writes the bundles, the other files, and the app's definition (`app.json`) to a `.bundled` folder.
## Publishing
```shell
npx @examplary/cli publish
```
Publishing the app again creates a new version, unless nothing changed since the last one. Workspaces that installed it keep their settings and get the new version straight away, or, for a [public app](#public-apps), once it's been reviewed. That includes its question types.
To publish an app in a CI environment, make sure the `EXAMPLARY_API_KEY` environment variable is set with an admin API key, then run `npx @examplary/cli publish`.
To set a [secret](/developers/apps/building-blocks/hooks#secrets), publish the app first, then run:
```shell
npx @examplary/cli set-secret ACME_CLIENT_SECRET "..."
```
## Public apps
By default, an app is only available in your own workspace. Set `public: true` to let every Examplary workspace find and install it:
```ts
const app = defineApp({
id: "acme.gradebook",
// ...
public: true,
});
```
Public apps run code in other workspaces, and their question types show up in the public [question type library](/question-types), so their new versions are reviewed by the Examplary team before they're used, in your own workspace too. You'll get an email when a version enters review, and another once it's approved or rejected. Until then, every workspace keeps using the last approved version. Once an app is public it can't be made private again: publishing a version with `public: false` is refused. To try out a new version before submitting it, use [`dev`](#trying-out-your-app).
## Retiring an app
To stop new workspaces from installing your app, publish a new version with `deprecated: true`. Workspaces that already installed it keep using it.
You can only delete an app (`DELETE /apps/{id}`) once no workspace has it installed, and never while it's preinstalled; until then the API answers with a 409. Deprecate it instead.
## Retiring a question type
To retire one of your app's question types, set `deprecated: true` on its definition, or leave it out of the app, and publish a new version. Existing questions of that type keep rendering, grading and regenerating everywhere, but no new questions of that type can be created.
Deprecate rather than hide: a public app can't be made private again, because that would stop other workspaces from resolving the definitions of questions they already have, so those questions stop rendering.
## Publishing on push to main
You can publish your app from CI, for example with a GitHub Actions workflow:
```yaml title=".github/workflows/publish.yml"
name: Publish to Examplary
on:
push:
branches:
- main
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24
- run: npm ci
- run: npx @examplary/cli publish
env:
EXAMPLARY_API_KEY: ${{ secrets.EXAMPLARY_API_KEY }}
```
---
// File: apps/question-types/metadata
# Question type definition
A question type is defined in TypeScript as a `new QuestionType()` from `@examplary/apps`, usually in a `definition.ts` in the question type's own folder. Here's an example as a starting point:
```ts title="short-answer/definition.ts"
import { QuestionType } from "@examplary/apps";
export default new QuestionType({
id: "com.acme.quiz.short-answer",
name: {
en: "Short answer",
nl: "Kort antwoord",
},
description: {
en: "A question that requires a short text response.",
nl: "Een vraag die een kort tekstantwoord vereist.",
},
icon: "./short-answer/icon.svg",
components: {
assessment: "./short-answer/component-assessment.tsx",
print: "./short-answer/component-print.tsx",
},
readme: { en: "./short-answer/README.md", nl: "./short-answer/README.nl.md" },
previewImage: "./short-answer/preview.png",
timeEstimateMinutes: 2,
generation: {
enabled: true,
},
translations: {
placeholder: {
en: "Type your answer...",
nl: "Typ je antwoord...",
},
},
});
```
Your app then adds it in `app.ts`:
```ts title="app.ts"
import ShortAnswer from "./short-answer/definition";
app.addCapability(ShortAnswer);
```
The files a definition refers to (`icon`, `components`, `readme` and `previewImage`) are paths relative to your `app.ts`, not to the definition itself, like `./short-answer/icon.svg`.
## Question type ID
The `id` field uniquely identifies your question type across Examplary. It must follow this format:
- **Starts with your app's ID**: the app `com.acme.quiz` can add `com.acme.quiz.short-answer`
- **Dot-separated namespacing**: must contain at least one dot (e.g. `com.acme.quiz.multiple-choice`)
- **Lowercase only**: each segment may only contain lowercase letters (`a`–`z`), digits (`0`–`9`), or hyphens (`-`)
- **Letter-first**: each segment must start with a letter, not a digit or hyphen
As app IDs use **reverse domain notation**, so do the IDs of their question types, which keeps them from clashing with other question types.
| Valid (in the app `com.acme.quiz`) | Invalid |
| ---------------------------------- | --------------------------------------------------- |
| `com.acme.quiz.multiple-choice` | `com.acme.quiz.MultipleChoice` (uppercase) |
| `com.acme.quiz.math.equation` | `com.acme.other.equation` (not the app's ID) |
| `com.acme.quiz.type-1` | `com.acme.quiz.1type` (segment starts with a digit) |
## Translatables
Most of the fields that are displayed in the UI can be translated. The `name` and `description` fields are good examples of this. You are encouraged to provide translations for all supported languages.
If you don't want to translate the field yet, you can also use a simple string field:
```ts
new QuestionType({
name: "Short answer",
});
```
## Name, description and icon
The `name` and `description` fields are used to display the question type in the UI. The `icon` field points to the SVG icon that represents the question type.
This SVG will be displayed in the question adder, and possibly in other places in the UI. Specifications:
- Displayed as a 24×24‑pixel icon
- Should either use `black` or `currentColor` as the only fill color
Most of the icons in the default question types are based on [Lucide icons](https://lucide.dev/). You can use these icons as a starting point for your own question type icons. We recommend a stroke width of 2.25px when exporting the SVG from the Lucide website.

## Components
The `components` field is an object that defines the React components used for different views of the question type. The keys are the view names, and the values are the paths to the component files.
Component files can be written in TypeScript or JavaScript, and should have a default export that is a React component. The component will receive the question data as props, which you can use to render the question in the specific view.
```ts
new QuestionType({
components: {
assessment: "./short-answer/component-assessment.tsx",
print: "./short-answer/component-print.tsx",
},
});
```
More information about the components can be found in the [components documentation](/developers/apps/question-types/components).
Questions without any components are allowed, but are only useful in specific cases (the default question types include 'paragraph', 'intro' and 'outro' which do not have any components). If you don't provide any components, the student can't interact with the question in the assessment view, but the title and description will still be displayed.
## Settings
Question types can have settings that are displayed in the question editor:

These settings are defined in the `settings` of the definition. Default fields 'question type', 'tags', and 'scoring criteria'/'points' will always be present, but in this example the following were added:
```ts
new QuestionType({
settings: [
{
id: "minWords",
name: {
en: "Minimum word count",
nl: "Minimaal aantal woorden",
},
type: "number",
placeholder: "0",
step: 50,
},
{
id: "maxWords",
name: {
en: "Maximum word count",
nl: "Maximaal aantal woorden",
},
type: "number",
placeholder: "0",
step: 50,
},
],
});
```
You can also add settings one at a time with `addSetting()`, after those in the definition:
```ts title="essay/definition.ts"
const essay = new QuestionType({
// ...
});
essay.addSetting({
id: "minWords",
name: { en: "Minimum word count", nl: "Minimaal aantal woorden" },
type: "number",
placeholder: "0",
step: 50,
});
export default essay;
```
Each settings object can have the following fields:
| Field | Type | Description |
| ------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | string | A unique identifier for the setting, used to store the value in the question data. This should be a valid JavaScript identifier. |
| `name` | [Translatable](#translatables) | The name of the setting, displayed in the UI. |
| `type` | string | The type of the setting. See below. |
| `helpText` | [Translatable](#translatables) | Optional information for the user, shown as a tooltip on hover. |
| `description` | string | An optional description for the setting, used exclusively for AI generation. Useful to specify the format of the values expected. |
| `default` | any | The default value for the setting. Must match the setting type. |
| `hidden` | boolean | If true, the setting will not be shown in the question editor UI. Useful for settings that are managed programmatically or set via import, rather than by the teacher. |
| `required` | boolean | If true, marks the setting as required when the AI generates a question of this type — the AI must populate it. This is not enforced when a question is created or edited manually in the question editor. |
| `index` | number | Controls the display order of this setting relative to the built-in ones (which use negative indices) and other custom settings. Lower values are shown first. Optional — settings without an explicit `index` are shown in array order, after the built-ins. |
| `options` | array | An array of options for `enum` type settings. Each option should be an object with `value` and `label` fields, where `value` is the actual value stored in the question data, and `label` (Translatable) is the text displayed in the UI. |
| `schema` | object | A JSON Schema describing the value of a `custom` setting. The AI uses it to generate a value of the right shape. Defaults to a string when omitted. |
| `placeholder` | [Translatable](#translatables) | A placeholder text for the input field, displayed when the field is empty. |
| `step` | number | The step size for number inputs. Optional. |
| `min` | number | The minimum value for number inputs. Optional. |
| `max` | number | The maximum value for number inputs. Optional. |
The following settings types are supported:
- `enum` - A dropdown with a list of options. The `options` field is required.
- `boolean` - A checkbox/switch. The value will be `true` or `false`.
- `number` - A number input field. The `placeholder`, `step`, `min` and `max` fields are optional.
- `string` - A single-line text input field. The `placeholder` field is optional.
- `string-array` - A multi-value input where users can add and remove string entries. Stored as an array of strings.
- `custom` - A value of any shape, such as an object or a list of objects, that has no input in the question editor. Your own components can read and write it. Describe its shape in `schema` so AI-generated questions fill it in correctly.
## Optional fields
These other fields are available in the definition:
| Field | Type | Description |
| --------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `readme` | string or object | The path of a Markdown file describing the question type, or one per language code. Shown on its page in the [question type library](/question-types). |
| `previewImage` | string or object | The path of a preview image, or one per language code, shown in the question type library. |
| `generation.enabled` | boolean | If true, the question type can be automatically generated by the platform's AI exam generator. |
| `generation.instructions` | string or string[] | Optional additional AI instructions for generating questions of this type. |
| `grading.enabled` | boolean | If true, the question type can be automatically graded by the platform's AI grading system. |
| `grading.instructions` | string or string[] | Optional additional AI instructions for grading questions of this type. |
| `scanning.instructions` | string or string[] | Optional additional AI instructions for converting scanned student answers for this question type. |
| `capabilities["answer-scanning"]` | boolean | Whether this question type supports scanning student answers using their phone camera. Only supported in practice space flows. |
| `responseProcessing` | object | Defines deterministic auto-grading logic using a JSONata expression. See [Question grading](/developers/apps/question-types/grading). |
| `export.qti3` | object | Configures native QTI 3.0 export for this question type instead of the default PCI wrapper. See [QTI interchange](/developers/apps/question-types/qti3). |
| `import.qti3` | object | Maps incoming QTI 3.0 interaction data to this question type's settings on import. See [QTI interchange](/developers/apps/question-types/qti3). |
| `timeEstimateMinutes` | number | An estimate of the time required to answer the question, in minutes. |
| `translations` | object | An object containing [translatable strings](/developers/apps/question-types/translations) for the question type, such as placeholders. |
| `isAi` | boolean | Specifies that this question type uses AI functionality within the question. |
| `backgroundColor` | string | A CSS color value for the background color of the question type in the UI. |
| `enforcePosition` | "start" or "end" | Specifies whether the question type should always be added at the start or end of the question list in an exam. Also limits an exam to a single question of this type — once one exists, teachers can't add another. |
| `enforceTitle` | [Translatable](#translatables) | Force questions of this type to always have a specific title. |
| `hideSettings` | boolean | If true, question settings will not be displayed in the UI when editing a question of this type. |
| `isPreviewRefreshable` | boolean | If true, the question preview shows a refresh button. |
| `hasSimpleScoring` | boolean | Removes the 'scoring criteria' setting, and instead lets the teacher pick points to award for this question. |
| `titlePlaceholder` | [Translatable](#translatables) | A placeholder for the title input field in the question editor. |
| `descriptionPlaceholder` | [Translatable](#translatables) | A placeholder for the description input field in the question editor. |
| `untitledPlaceholder` | [Translatable](#translatables) | A placeholder for untitled questions in the question editor. |
| `metadata` | object | Optional metadata for the question type, with any JSON values, like `preview`: example question and answer data per language. Not used by the Examplary platform — available to external applications. |
---
// File: apps/building-blocks/hooks
# Hooks
Hooks are the functions in your app that run on the server: the [settings](/developers/apps/settings) steps, and the functions you give to a capability, like `startExport`. You don't call them yourself; Examplary does, when it needs them.
Every hook gets the settings of the installation as its first argument: the workspace's settings, merged with the teacher's own when the app has [user settings](/developers/apps/settings#workspace-and-user-settings).
## Where hooks run
When you publish your app, the CLI bundles `app.ts`, with everything it imports, into a single script. Examplary runs that script in a sandbox, a fresh JavaScript environment for every call. That keeps apps apart from each other and from the platform, but it also means hooks can't use everything Node.js offers.
Available in hooks:
- Standard JavaScript, including `async`/`await`, `JSON`, `Date`, `Math` and `RegExp`
- `fetch()`, with `Headers`, `Response`, `URL` and `URLSearchParams`
- `TextEncoder`
- `crypto.randomUUID()`, and `crypto.subtle.digest()` with `SHA-1`, `SHA-256`, `SHA-384` or `SHA-512`
- `console` for logging
Not available:
- Node.js modules, like `fs`, `http`, `crypto` or `Buffer`, and npm packages that need them
- Timers like `setTimeout`
- `AbortController`, streaming response bodies, `Blob` and `FormData`
Requests made with `fetch()` have to use HTTPS and can only reach public addresses. A hook can run for up to 25 seconds, with 256 MB of memory. For work that takes longer, like an export that a platform processes in the background, start it in your hook and let Examplary check back later; [export options](/developers/apps/capabilities/export-options) work that way.
Nothing is kept between two calls of a hook. Store what you need to remember in a setting, or return it from your hook when the capability allows it, like the `data` of an export.
## Errors
When a hook throws an error, Examplary shows its message to the person who caused the hook to run: the admin filling in the settings, the teacher exporting a test, or the student opening it. Write those messages for them:
```ts
if (res.status === 401) {
throw new Error(
"Your Acme API token has expired. Update it in the app's settings.",
);
}
```
## Secrets
Some values belong to your app rather than to a workspace, like the client secret of your OAuth app. Store those as secrets, which only your app's hooks can read:
```shell
npx @examplary/cli set-secret ACME_CLIENT_SECRET "..." --key YOUR_API_KEY
```
Then read them from `env` inside a hook. Secret names are written in `UPPER_SNAKE_CASE`, and a secret that isn't set is `undefined`:
```ts
import { env } from "@examplary/apps";
const exchangeCode = async (code: string) => {
const res = await fetch("https://api.acme.com/oauth/token", {
method: "POST",
body: new URLSearchParams({
code,
client_id: "examplary",
client_secret: env.ACME_CLIENT_SECRET!,
}),
});
return (await res.json()).refresh_token;
};
```
Secrets are only available while a hook runs. Read `env` inside your functions, not at the top of a file, where it would still be empty.
## Who a hook runs for
`context` tells you the workspace a hook runs for, and the user who started it, if any. These are Examplary IDs, which you can use to tell installations apart on your side:
```ts
import { context } from "@examplary/apps";
const workspaceId = context.org; // "org_…"
const userId = context.user; // "user_…", or undefined
```
---
// File: apps/question-types/components
# Question type components
Examplary renders React components for each question type in various contexts.
You can implement components for the following views:
- `assessment`: The main view where users interact with the question, either by the student during the assessment, or by the teacher in the question editor preview window.
- `print`: A simplified view for printing the question, which is used in the print preview and when printing the assessment.
- `results`: Displaying the results of the question, both in the teacher's review of the assessment and when a student reviews their own results — so don't show anything here that students shouldn't see.
- `settings-area`: Displaying custom settings for the question type in the question editor.
Only use the `settings-area` component if you need to display custom settings for your question type that can't be covered using the standard settings described in the [metadata](/developers/apps/question-types/metadata) section.
## Building and testing components
Components should be implemented as React components in JavaScript or TypeScript. You can use any React features, including hooks, context, and state management.
Here's a simple example of the `assessment` component in the `single-line-text` question type, written in TypeScript:
```tsx title="component-assessment.tsx"
import { FrontendAssessmentComponent, RichTextField } from "@examplary/ui";
const AssessmentComponent: FrontendAssessmentComponent = ({
answer,
saveAnswer,
t,
}) => {
return (
{
const valid = value.trim().length > 0;
saveAnswer({ value, completed: valid });
}}
/>
);
};
export default AssessmentComponent;
```
## Object types
### Question object
This represents the question itself, which is passed to the components as a prop. It contains metadata about the question type, such as its name, description, and settings.
It looks like this:
```json
{
"id": "q_qQjXQasmUfpWRV5KO5mzkxoy",
"title": "Napoleon's role in the French Revolution",
"type": "single-line-text",
"description": "When Napoleon Bonaparte rose to power, he was seen as a hero of the French Revolution. However, his actions and policies during his rule have been debated by historians. What was Napoleon's role in the French Revolution?",
"settings": {
"maxLength": 200
},
"scoring": {
"rubricType": "analytical",
"criteria": [
{
"id": "c1",
"title": "Correctness",
"levels": [{ "id": "c1l1", "title": "Correct", "points": 1 }]
}
]
},
"externalId": "my_question_1",
"metadata": { "my_service_internal_id": "abc1234" }
}
```
The `settings` field contains your question type's own settings, as defined in its definition. During an actual exam session, any setting literally named `correctAnswer` is always stripped from this object before it reaches the `assessment` component, regardless of question type — avoid that name if you need the value available while a student is taking the exam.
The `scoring` field holds the rubric configuration set by the teacher. It is available to the `results` component so you can display how the question was scored.
The `metadata` field is a key-value store for external system data — values must be strings, numbers, booleans, or `null`; nested objects and arrays are rejected. It is passed through unchanged to `print`, `results`, and `settings-area` components, and to the `assessment` component in the question editor preview. During an actual exam session, the `assessment` component instead receives a reduced object: `id`, `type`, `title`, `description`, `index` (the question's position in the array, zero-based), `number` (its display number, which skips `paragraph` questions), and `settings` (with `correctAnswer` removed) are included, plus `pointsToEarn` and `canProvideFeedback` when the exam is configured to show them.
### Answer object
The answer object represents the student's answer to the question. It contains the following properties:
```json
{
"value": "He was important because he was a general in the French army and later became the Emperor of France.",
"completed": true,
"context": {}
}
```
The `value` property contains the student's answer, which can be a string or an array of strings — array items can also be `null`, which is used to represent an empty slot in a partially-filled ordering or matching answer. The `completed` property indicates whether the answer is complete, and the `context` property can hold additional information relevant to the answer. The question type can define what this context should contain, but it is not required.
## Component props
### Assessment component props
| Prop | Type | Description |
| ------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `question` | [Question](#question-object) | The question object, which contains metadata about the question type, such as its title and settings. |
| `answer` | [Answer](#answer-object) | The current answer object, which may contain the student's answer and its completion status. |
| `saveAnswer` | function | A function to save the answer. It takes an object with `value`, `completed` and `context` properties. |
| `isPreview` | boolean | Indicates whether the component is being rendered in a preview context (e.g., in the question editor). |
| `isLoading` | boolean | Indicates whether the question is currently loading (e.g., during AI content generation). |
| `reviewMode` | boolean (optional) | When `true`, the component is rendered in review mode where the student can see but not modify their answer. |
| `environment` | `"exam"` \| `"practice-space"` | The context in which the component is rendered. |
| `api` | [API object](/developers/apps/question-types/api-access) | An API client instance for making requests to the Examplary REST API, including file uploads and AI generation. |
| `t` | [Translation function](/developers/apps/question-types/translations) | A function to access translations defined in the question type's definition. |
| `i18n` | object | The i18next instance, providing full access to the i18n API (use `t` for simple translations). |
### Print component props
| Prop | Type | Description |
| ------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `question` | [Question](#question-object) | The question object, which contains metadata about the question type, such as its title and settings. |
| `answerBoxes` | boolean | Whether to show boxes for the student to write their answers down. |
| `environment` | `"exam"` \| `"practice-space"` | The context in which the component is rendered. |
| `api` | [API object](/developers/apps/question-types/api-access) | An API client instance for making requests to the Examplary REST API, including file uploads. |
| `t` | [Translation function](/developers/apps/question-types/translations) | A function to access translations defined in the question type's definition. |
| `i18n` | object | The i18next instance, providing full access to the i18n API (use `t` for simple translations). |
### Results component props
| Prop | Type | Description |
| ------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `question` | [Question](#question-object) | The question object, which contains metadata about the question type, such as its title and settings. |
| `answer` | [Answer](#answer-object) | The current answer object, which may contain the student's answer and its completion status. |
| `environment` | `"exam"` \| `"practice-space"` | The context in which the component is rendered. |
| `api` | [API object](/developers/apps/question-types/api-access) | An API client instance for making requests to the Examplary REST API, including file uploads. |
| `t` | [Translation function](/developers/apps/question-types/translations) | A function to access translations defined in the question type's definition. |
| `i18n` | object | The i18next instance, providing full access to the i18n API (use `t` for simple translations). |
### Settings area component props
| Prop | Type | Description |
| --------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `question` | [Question](#question-object) | The question object, which contains metadata about the question type, such as its title and settings. |
| `settings` | object | The current settings for the question type, which can be modified by the user. |
| `setSetting` | function | A function to update a specific setting. It takes the setting ID and the new value as arguments. |
| `setMultipleSettings` | function | A function to update multiple settings at once. It takes an object with setting IDs as keys and their new values as values. |
| `environment` | `"exam"` \| `"practice-space"` | The context in which the component is rendered. |
| `api` | [API object](/developers/apps/question-types/api-access) | An API client instance for making requests to the Examplary REST API and uploading files. |
| `t` | [Translation function](/developers/apps/question-types/translations) | A function to access translations defined in the question type's definition. |
| `i18n` | object | The i18next instance, providing full access to the i18n API (use `t` for simple translations). |
---
// File: apps/building-blocks/components
# Components
Components are React components your app shows inside Examplary: a [component setting](/developers/apps/settings#component-settings) in the app's settings, or the screens of an [assessment prerequisite](/developers/apps/capabilities/assessment-prerequisites). They work like the components of [custom question types](/developers/apps/question-types/components).
You point at a component by its path, optionally with translations:
```ts
playerComponent: {
path: "./components/player.tsx",
translations: {
title: { en: "Open the Acme browser", nl: "Open de Acme-browser" },
},
},
```
Each component file default-exports a React component. When you publish the app, the CLI bundles each one for the browser, with everything it imports.
## Props
Every component gets:
| Prop | Description |
| ------ | ---------------------------------------------------------------------------------------------------------- |
| `t` | Translates a key from the component's `translations` into the current user's language. |
| `i18n` | The [i18next](https://www.i18next.com) instance, e.g. for `i18n.language`. |
| `api` | An authenticated API client, as in [question type components](/developers/apps/question-types/api-access). |
On top of these, each kind of component gets props of its own, described on the page of the setting or capability it belongs to. `@examplary/ui` has a type for each, like `FrontendSettingsFieldComponent`, so TypeScript knows what's there:
```tsx title="components/player.tsx"
import { Button, FrontendPrerequisitePlayerComponent } from "@examplary/ui";
const Player: FrontendPrerequisitePlayerComponent = ({ t, exam }) => (
);
export default Player;
```
## Styling and UI
Use the components of [`@examplary/ui`](https://www.npmjs.com/package/@examplary/ui), like `Button`, `Input` and `Switch`, so your app looks like the rest of Examplary. React and `@examplary/ui` are provided by the platform: they aren't included in your bundle, and you only need to install them for their types.
You can style your own elements with [Tailwind CSS](https://tailwindcss.com) classes. When you publish the app, the CLI finds the classes used in your app's folder and bundles them into a stylesheet, which is loaded along with your components. See [styling your components](/developers/apps/building-blocks/styling) for the most-used components of the UI kit.
Components run in the browser of teachers and students, so other npm packages you import end up in the bundle they download. Do the heavy lifting in [hooks](/developers/apps/building-blocks/hooks) instead, and call those from your component where the capability allows it, like an assessment prerequisite's `callFunction()`.
---
// File: apps/question-types/generation
# Question generation
The Examplary platform supports AI-assisted question generation for custom question types.
You can enable this feature by configuring the `generation` option in your question type's metadata.
The `generation` field is an object that defines the settings for AI-assisted question generation.
Specify an `enabled` boolean to turn this feature on or off, and provide any additional instructions for the AI.
```ts title="definition.ts (partial)"
new QuestionType({
generation: {
enabled: true,
instructions:
"Requires an `options` setting with an array of strings that will be used as multiple choice options. Also set `possibleAnswers` to an array containing the accepted option values for response processing.",
},
});
```
## Validating generated questions
Question types can check the questions the AI generates before they are added to a test. Write a function that receives `{ question }` and returns `{ valid: boolean, error?: string }`, and add it to your question type with `addValidator()`. It can be async.
```ts title="multiple-choice/validate-question.ts"
import type { Question } from "@examplary/schemas";
export default ({ question }: { question: Question }) => {
const options = question.settings?.options ?? [];
if (options.length < 2) {
return { valid: false, error: "Provide at least two options." };
}
return { valid: true };
};
```
```ts title="multiple-choice/definition.ts"
import { QuestionType } from "@examplary/apps";
import validateQuestion from "./validate-question";
const multipleChoice = new QuestionType({
// ...
});
multipleChoice.addValidator(validateQuestion);
export default multipleChoice;
```
When a question is invalid, it isn't added and your `error` message goes back to the AI, so make it specific. The validator runs on our servers as one of your app's [hooks](/developers/apps/building-blocks/hooks), in the same sandbox. If it fails or returns anything else, the question is accepted as is.
---
// File: apps/question-types/translations
# Translations management
Examplary natively supports translations, which allows you to easily manage and extend the platform's language support. This documentation will guide you through the process of adding and managing translations for custom question types.
## Adding translations
In addition to providing support for multiple languages for the main fields of a question type's definition, you can also add translations for the components of your custom question types. This is done by including a `translations` object in its definition.
```ts
new QuestionType({
translations: {
placeholder: {
en: "Enter your answer here",
nl: "Typ je antwoord hier",
},
},
});
```
## Using translations in components
Each component gets passed in the `t` function as a prop, which you can use to access the translations defined in the question type's definition. For example, if you have a translation for a placeholder text, you can use it like this:
```jsx
import React from "react";
export default function MyComponent({ t }) {
return ;
}
```
## Interpolation and pluralization
Examplary supports basic interpolation and pluralization features. You can use these features to create dynamic translations that adapt based on the context.
For example, if you want to display a message that includes a count, you can define your translation like this:
```ts
new QuestionType({
translations: {
itemCount_one: {
en: "{{count}} item",
nl: "{{count}} item",
},
itemCount_other: {
en: "{{count}} items",
nl: "{{count}} items",
},
welcomeMessage: {
en: "Welcome, {{name}}!",
nl: "Welkom, {{name}}!",
},
},
});
```
```jsx
import React from "react";
export default function MyComponent({ t }) {
const itemCount = 5;
return (
{t("itemCount", { count: itemCount })}
{t("welcomeMessage", { name: "John" })}
);
}
```
---
// File: apps/building-blocks/styling
# Styling your components
There are a few tools to style your components in Examplary. The platform uses [Tailwind CSS](https://tailwindcss.com)
for utility-first styling, and you can also use the [Examplary UI kit](https://www.npmjs.com/package/@examplary/ui)
for pre-built components and styles.
## Using Tailwind CSS
When you use Tailwind CSS utility classes, these will automatically be bundled in with your code.
```jsx
import React from "react";
export default function MyComponent() {
return
Hello, Tailwind!
;
}
```
Read more about [Tailwind CSS](https://tailwindcss.com/docs/styling-with-utility-classes) to learn how to
use its utility classes effectively.
## Examplary UI kit
The package `@examplary/ui` provides a set of pre-built components and styles that you can use in your
components. It is provided by the platform at runtime, for every app, so it isn't included in your bundle; you only need to install it for its types.
You can use the components from the UI kit like this:
```jsx
import React from "react";
import { RichTextField } from "@examplary/ui";
export default function MyComponent() {
return ;
}
```
Common components from the UI kit include:
### ``
Styled HTML input field.
```jsx
import { Input } from "@examplary/ui";
export default function MyComponent() {
return ;
}
```
### ``
Styled HTML select dropdown.
```jsx
import {
Select,
SelectContent,
SelectGroup,
SelectItem,
SelectLabel,
SelectTrigger,
SelectValue,
} from "@examplary/ui";
export default function MyComponent() {
return (
);
}
```
### ``
Component that displays print answer boxes, for the print component of a [question type](/developers/apps/question-types).
```jsx
import { AnswerBox, FrontendPrintComponent } from "@examplary/ui";
const PrintComponent: FrontendPrintComponent = ({ answerBoxes }) => {
if (!answerBoxes) return null;
return ;
};
export default PrintComponent;
```
### ``
Component that displays a rich text field for user input. Allows HTML formatting and
is useful for text-based questions.
Props:
| Prop | Type | Default | Description |
| -------------------- | -------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------- |
| `value` | string | `""` | The current value of the rich text field (HTML string). |
| `defaultValue` | string | — | Initial value for uncontrolled usage. Changes to this prop reset the editor content. |
| `onChange` | `(value: string) => void` | — | Called with the new HTML string whenever the content changes. |
| `onBlur` | `(value: string) => void` | — | Called with the current HTML string when the editor loses focus. |
| `placeholder` | string | `""` | Placeholder text shown when the field is empty. |
| `className` | string | `""` | Additional CSS classes for the editor element. |
| `singleLine` | boolean | `false` | Restricts the editor to a single line, preventing newlines. |
| `autoFocus` | boolean | `false` | Automatically focuses the field when it mounts. |
| `showFormattingMenu` | boolean | `false` | Shows a floating formatting toolbar when text is selected. |
| `id` | string | — | Gives the editor a stable ID, accessible via `getEditorById(id)` from `@examplary/ui`. |
| `slotBefore` | `(editor: Editor) => React.ReactNode` | — | Render callback called with the Tiptap `Editor` instance; return a React node to render before the editor. |
| `uploadFile` | `(file: File) => Promise<{ url: string; name: string }>` | — | Enables image pasting and drag-and-drop. Called with the dropped/pasted file; return the hosted URL. |
Example:
```jsx
import { useState } from "react";
import { RichTextField, RichTextToolbar } from "@examplary/ui";
export default function MyComponent() {
const [value, setValue] = useState("");
return (
setValue(value)}
placeholder="Type your answer here..."
className="border p-2 rounded"
slotBefore={(editor) => }
/>
);
}
```
### ``
Component that displays rich text content. It is useful for showing formatted HTML in a read-only format,
with the appropriate styles, similar to how it would appear in a rich text editor.
Props:
- `children`: The rich text content to display, which should be a string containing HTML.
- `as`: The HTML element to render (default is `div`).
```jsx
import { RichTextDisplay } from "@examplary/ui";
export default function MyComponent() {
const content = "
This is rich text content.
";
return {content};
}
```
---
// File: apps/capabilities/dashboard-pages
# Dashboard pages
A dashboard page adds a page of your own to Examplary, with a menu item in the sidebar. The page renders one of your app's [components](/developers/apps/building-blocks/components), which can call your app's server-side code through **actions**.
```mermaid
flowchart LR
menu[Menu item] --> page[Your component]
page -- "action(name, input)" --> hook[Your action, in the sandbox]:::highlight
hook -- result --> page
```
## Defining a page
```ts title="capabilities/avatar-page.ts"
import { DashboardPage } from "@examplary/apps";
export default new DashboardPage({
id: "avatar",
menuTitle: { en: "My avatar", nl: "Mijn avatar" },
location: "sidebar",
component: {
path: "./components/avatar-page.tsx",
translations: {
title: { en: "Pick an avatar", nl: "Kies een avatar" },
},
},
actions: {
shuffle: async (settings, { seed }) => {
const response = await fetch(`https://avatars.example.com/${seed}`, {
headers: { Authorization: `Bearer ${settings.apiToken}` },
});
return response.json();
},
},
});
```
| Property | Description |
| ----------- | ---------------------------------------------------------------------------------- |
| `id` | Unique within your app. |
| `menuTitle` | The label of the page's menu item. |
| `location` | Where the menu item goes, see [below](#locations). The main sidebar when left out. |
| `component` | The component that renders the page. |
| `actions` | Hooks your component can call with `action(name, input)`, see [below](#actions). |
## Locations
| Location | Menu item | Who sees it |
| ------------------- | -------------------------------------------------------- | ------------------- |
| `sidebar` | In the main sidebar, below Library. | Teachers and admins |
| `account.personal` | In the account settings, under the user's own settings. | Teachers and admins |
| `account.workspace` | In the account settings, under the workspace's settings. | Admins |
| `account.advanced` | In the account settings, under the advanced settings. | Admins |
The page shows up in every workspace that installed your app.
## The component
Besides the [usual props](/developers/apps/building-blocks/components#props), the page's component gets `action(name, input)`, which runs one of your actions and returns its result. Use the `FrontendPageComponent` type from `@examplary/ui`:
```tsx title="components/avatar-page.tsx"
import { useState } from "react";
import { Button, FrontendPageComponent } from "@examplary/ui";
const AvatarPage: FrontendPageComponent = ({ t, action }) => {
const [avatar, setAvatar] = useState();
const shuffle = async () => {
const { url } = await action("shuffle", { seed: crypto.randomUUID() });
setAvatar(url);
};
return (
<>
{t("title")}
{avatar && }
>
);
};
export default AvatarPage;
```
When all you need is Examplary's own API, you don't need an action: call it from the component with `api`, like `api.patch("/me", { ... })`.
## Actions
Actions are [hooks](/developers/apps/building-blocks/hooks): they run on our servers, in a sandbox, so they can use your app's [secrets](/developers/apps/building-blocks/hooks#secrets) and talk to your own platform without exposing credentials to the browser.
Each action gets:
- `settings`: the workspace's settings for your app, merged with the user's own.
- `input`: whatever the component passed to `action()`.
Return anything that can be sent as JSON. [`context.user`](/developers/apps/building-blocks/hooks#who-a-hook-runs-for) tells you who called it. If you throw an error, its message is shown to the user, and `action()` rejects.
## Calling the Examplary API
In an action, `context.examplary` is an [Examplary SDK](https://www.npmjs.com/package/@examplary/sdk) client that acts as the user who called the action, with exactly their permissions in this workspace:
```ts
import { context, DashboardPage } from "@examplary/apps";
export default new DashboardPage({
id: "sync",
menuTitle: { en: "Sync to Acme" },
component: "./components/sync-page.tsx",
actions: {
syncTests: async (settings) => {
const tests = await context.examplary.exams.list();
await fetch("https://api.acme.com/tests", {
method: "POST",
headers: { Authorization: `Bearer ${settings.apiToken}` },
body: JSON.stringify(tests.map(({ id, name }) => ({ id, name }))),
});
return { synced: tests.length };
},
},
});
```
Its methods take a single object, as described in the SDK's README. Your code never sees the user's credentials: the client only works while the action runs, and only for this workspace. Account security settings, like the user's email address and two-factor authentication, can't be changed through it.
Actions of pages in `account.workspace` and `account.advanced` can only be called by admins, just like the pages themselves.
Every call of an action is recorded in the workspace's audit log.
---
// File: apps/question-types/api-access
# Calling the API and uploading files
In your components, you might want to call the [Examplary REST API](https://examplary.ai/developers/rest-api).
## API prop
All of your components will receive an `api` prop, which is an authenticated instance of [Axios](https://axios-http.com/docs/intro). You can use this prop to make API calls directly from your components.
In `settings-area`, `results`, and `print` components, this is authenticated against the currently signed-in user (teacher). In `assessment` components, it is authenticated against the student who is currently taking the assessment. Students usually only have access to public endpoints.
When your question type is exported as a [Portable Custom Interaction (PCI)](/developers/apps/question-types/qti3) and rendered inside an external LMS, the `api` prop will be `null`. Any code that calls `api.get(...)`, `api.uploadFile(...)`, or any other method on `api` will throw an error in that context.
If your `assessment` component relies on API access, guard against this with a null check:
```jsx
if (api) {
const data = await api.get("/some-endpoint");
}
```
### Basic usage
```jsx
export default function MyQuestionSettingsArea({ api }) {
const fetchData = async () => {
try {
const response = await api.get("/question-types");
console.log(response.data);
} catch (error) {
console.error("Error fetching data:", error);
}
};
useEffect(() => {
fetchData();
}, []);
return
Check the console for data.
;
}
```
### Using the AI API
Public question components (i.e. `assessment` components) have the `api` object extended with an `ai.generate()` method that allows you to generate content using AI. It takes a list of messages in the standard chat format and returns the AI's response as a string.
```jsx
import { useState, useEffect } from "react";
export default function MyQuestionAssessmentComponent({ question, api }) {
const [story, setStory] = useState("");
const generateStory = async () => {
const response = await api.ai.generate({
messages: [
{
role: "system",
content:
"You are a teacher telling the student a story about a subject.",
},
{
role: "user",
content: `Tell me a story about ${question.description}.`,
},
],
});
// response is a string when no schema is provided
setStory(response);
};
useEffect(() => {
generateStory();
}, []);
return
{story}
;
}
```
You can also pass a `schema` key with a [JSON Schema](https://json-schema.org/) object to receive a structured data response instead of a plain string:
```jsx
const response = await api.ai.generate({
messages: [
{
role: "user",
content: `Generate 3 quiz options for: ${question.description}`,
},
],
schema: {
type: "object",
properties: {
options: {
type: "array",
items: { type: "string" },
},
},
required: ["options"],
},
});
// response is a structured object matching the schema
console.log(response.options); // ["Option A", "Option B", "Option C"]
```
## File uploads
It's also possible to prompt the user to upload files, which will be stored in the Examplary cloud storage.
### `api.uploadFile(accept?)`
Opens a file picker dialog and uploads the selected file. Available in all component types.
The optional `accept` parameter is a string that specifies the file types to accept (e.g., `"image/*"` for all image types).
```jsx
import { useState } from "react";
export default function MyQuestionSettingsArea({ api }) {
const [fileUrl, setFileUrl] = useState("");
const handleUpload = async () => {
const { url } = await api.uploadFile("image/*");
setFileUrl(url);
};
return (
{fileUrl && }
);
}
```
### `api.uploadPickedFile(file)`
Uploads a `File` object that you have already obtained (e.g., from a drag-and-drop handler or a custom ``). Only available in `settings-area`, `results`, and `print` components.
```jsx
import { useState } from "react";
export default function MyQuestionSettingsArea({ api }) {
const [fileUrl, setFileUrl] = useState("");
const handleDrop = async (event) => {
event.preventDefault();
const file = event.dataTransfer.files[0];
if (file) {
const { url } = await api.uploadPickedFile(file);
setFileUrl(url);
}
};
return (
e.preventDefault()}
className="border-2 border-dashed p-8"
>
Drop a file here
{fileUrl && (
)}
);
}
```
---
// File: apps/question-types/grading
# Question grading
The Examplary platform supports two grading options for custom question types:
1. **Deterministic response processing** Best for question types that have a clearly defined correct answer, and can be graded automatically based on the submitted answer and question settings.
2. **AI-assisted grading suggestions** Best for open-ended questions where the answer needs to be interpreted against a rubric or teacher guidance.
## Deterministic response processing
For simple question types that have a clearly defined correct answer, define `responseProcessing` in its definition. The `condition` is a JSONata expression that starts with `=` and evaluates to a boolean. If it returns `true`, the answer receives the question's maximum points; if it returns `false`, the answer receives `0` points.
The expression can read the submitted answer through `answer` and the question configuration through `question`. For example, `answer.value` is the user's submitted value, and `question.settings` contains the settings stored for that question.
```ts title="definition.ts (partial)"
new QuestionType({
responseProcessing: {
condition: "=answer.value in question.settings.possibleAnswers",
testCases: [
{
answer: {
value: "42",
},
question: {
settings: {
possibleAnswers: ["41", "42"],
},
},
expectedOutcome: true,
},
],
},
});
```
Use `testCases` to validate the expression when you validate or publish your app (`npx @examplary/cli validate`). Grading requires the expression to evaluate to a strict boolean — any other result (a string, a number, `undefined`, and so on) fails the answer with an error instead of awarding points. The CLI coerces the outcome to a boolean before comparing it against `expectedOutcome`, so a condition that only returns a truthy/falsy non-boolean can still pass its test cases while failing for real students — make sure your condition itself resolves to `true`/`false`, not just something that looks like it.
## Exact-value matching
When a teacher configures a question with the `exact-values` rubric type, the platform automatically grades submissions by comparing the student's answer against the expected values defined in the scoring criteria. No configuration in the definition is required for this to work.
Setting `exactValuesCaseInsensitive: true` in the question's scoring configuration lowercases both values and strips **all** whitespace (not just leading/trailing) before comparing — so `"New York"` and `"NewYork"` would be considered a match. (Leading/trailing whitespace is always trimmed, regardless of this setting.) This is useful for fill-in-the-blank or vocabulary questions where minor formatting differences shouldn't count against a student.
## AI assisted grading suggestions
The `grading` field is an object that defines the settings for AI-assisted grading. You can enable or disable this feature, and provide any additional instructions for the AI.
If enabled, the platform can use AI to suggest points and feedback for the user's answer based on the provided instructions. This can be particularly useful for open-ended questions where there isn't a single correct answer. The teacher can review and adjust the AI's suggestions before finalizing the grade.
```ts title="definition.ts (partial)"
new QuestionType({
grading: {
enabled: true,
instructions: "Grade the answer based on the provided rubric.",
},
});
```
### Grading answers that contain files
Some answers are files rather than text, like an uploaded document or a voice recording. Out of the box, the AI only sees the URL of such a file, not its contents. To let the AI read the file, add `fileInputs` to the `grading` object: a JSONata expression starting with `=` that evaluates to a file URL or an array of file URLs. Like `responseProcessing.condition`, it can read the submitted answer through `answer` and the question configuration through `question`.
```ts title="definition.ts (partial)"
new QuestionType({
grading: {
enabled: true,
instructions: "Grade the uploaded document against the rubric.",
fileInputs: "=answer.value",
},
});
```
The files are downloaded and attached to the end of the grading prompt, so the AI can read documents, images and audio recordings directly. Only files hosted on the Examplary media CDN, such as those uploaded through the [file upload API](/developers/apps/question-types/api-access), are attached; any other URL is skipped.
`npx @examplary/cli validate` checks that the expression is valid JSONata.
---
// File: apps/capabilities/export-options
# Export options
An export option adds your platform to the **Send to…** menu of a test, so
teachers can send or export a test to it in a few clicks.
```mermaid
flowchart LR
menu[Send to…] --> options[getExportOptions]
options --> dialog[Teacher picks options]
dialog --> start[startExport]:::highlight
start --> status[getExportStatus]
status -- in progress --> status
status --> done[View the result]
```
## Defining an export option
```ts title="capabilities/export.ts"
import {
ExportFormat,
ExportOption,
ExportResourceType,
ExportStatus,
} from "@examplary/apps";
export default new ExportOption({
id: "acme",
name: { en: "Acme Gradebook" },
icon: "./icon.svg",
resourceTypes: [ExportResourceType.Exam],
formats: [ExportFormat.Qti21],
// Called to fetch any options the teacher can choose from
// before starting the export.
getExportOptions: async (settings) => {
const classes = await listClasses(settings.apiToken);
return [
{
key: "classId",
label: { en: "Class" },
type: "select",
options: classes.map((c) => ({ value: c.id, label: c.name })),
},
];
},
// Called to start the export with the chosen options.
startExport: async (settings, test) => {
const job = await startImport(settings.apiToken, {
classId: settings.classId,
packageUrl: test.files.qti21!.url,
});
return { status: ExportStatus.inProgress, data: { jobId: job.id } };
},
// Called to check the status of an export that's still in progress.
getExportStatus: async (settings, { data }) => {
const job = await getImport(settings.apiToken, data.jobId);
if (job.state === "running") {
return { status: ExportStatus.inProgress, data };
}
if (job.state === "failed") {
return { status: ExportStatus.failed, message: job.error, data };
}
return { status: ExportStatus.completed, resultUrl: job.testUrl, data };
},
});
```
Then add it to your app with `app.addCapability()`, as shown in the [introduction](/developers/apps).
| Property | Description |
| --------------------- | ------------------------------------------------------------------------- |
| `id` | Unique within your app. |
| `name` | Shown in the **Send to…** menu. |
| `icon` | Path to the icon shown in the menu. |
| `resourceTypes` | What can be exported. For now, only `ExportResourceType.Exam` (tests). |
| `formats` | Files Examplary should prepare for `startExport`, see [below](#the-test). |
| `statusCheckInterval` | How often to call `getExportStatus`, in milliseconds. Defaults to `1000`. |
| `getExportOptions` | Returns the fields the teacher fills in for this export. Optional. |
| `startExport` | Starts the export. Required. |
| `getExportStatus` | Checks on an export that's still in progress. Optional. |
## Export options
`getExportOptions` returns the fields to show the teacher before the export
starts, like the class to send the test to. They're written like
[settings](/developers/apps/settings), and checked the same way. The values the
teacher picks are passed to `startExport` along with the app's settings,
so `settings.classId` above is the chosen class.
When a teacher hasn't filled in their own
[user settings](/developers/apps/settings#workspace-and-user-settings) yet,
like connecting their account, Examplary asks for those first.
## The test
`startExport` gets the test to export:
| Property | Description |
| ------------- | ------------------------------------------------------------------------ |
| `id` | The test's ID. |
| `name` | The test's name. |
| `publicUrl` | The link students use to open the test. |
| `totalPoints` | The points to earn, or `null` when the test doesn't show points. |
| `files` | A download `url` and `filename` for each of the `formats` you asked for. |
The only format for now is `ExportFormat.Qti21`, a
[QTI 2.1](https://www.1edtech.org/standards/qti) package that most assessment
platforms can import. The download URLs are only valid for a short while, so
pass the file on (or let your platform download it) right away.
## Export status
`startExport` and `getExportStatus` both return the export's status:
| Property | Description |
| ----------- | ---------------------------------------------------------------------------------------------------- |
| `status` | `ExportStatus.inProgress`, `completed` or `failed`. |
| `resultUrl` | Where the teacher can see the result, like the imported test. Shown as a button once it's completed. |
| `message` | What went wrong, or anything else the teacher should know, like questions that couldn't be imported. |
| `data` | Anything you need to check on the export later, like the ID of an import job. |
When an export can finish right away, return `completed` from `startExport`.
Otherwise, return `inProgress`, and Examplary calls `getExportStatus` with
the last status (including its `data`) every `statusCheckInterval` milliseconds,
until the export is completed or failed. An export that's still in progress
after 10 minutes counts as failed.
---
// File: apps/capabilities/assessment-prerequisites
# Assessment prerequisites
An assessment prerequisite is something students have to meet before they can start a test. Secure browsers, like Schoolyear, Muute and Safe Exam Browser, are built this way: once a teacher turns one on for a test, students can only take it from inside that browser.
A prerequisite adds a switch to the test's settings. When a teacher turns it on, Examplary shows its settings and its settings component. When students open the test, its player component explains how to meet it, like how to open the test in the secure browser, until it's met.
```mermaid
flowchart LR
teacher[Teacher turns it on] --> updated[onExamUpdated]
student[Student opens the test] --> met{isMet}
met -- no --> player[Player component]
player --> met
met -- yes --> session[Student takes the test]:::highlight
session --> validate[validateRequest, on every request]
```
## Defining a prerequisite
```ts title="capabilities/prerequisite.ts"
import { AssessmentPrerequisite } from "@examplary/apps";
export default new AssessmentPrerequisite({
id: "acme-browser",
name: { en: "Require the Acme browser" },
description: { en: "Students need to take this test in the Acme browser." },
requestHeaders: ["x-acme-signature"],
exitButton: true,
examSettings: [
{
key: "allowCalculator",
label: { en: "Allow the calculator" },
type: "boolean",
default: false,
},
],
playerComponent: {
path: "./components/player.tsx",
translations: {
title: { en: "Open this test in the Acme browser" },
},
},
isMet: (settings, { request }) => !!request.headers["x-acme-signature"],
validateRequest: async (settings, { request }) => {
const valid = await verifySignature(
settings.apiToken,
request.headers["x-acme-signature"],
);
if (!valid) {
throw new Error("Open this test in the Acme browser.");
}
},
});
```
| Property | Description |
| ------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `id` | Unique within your app. |
| `name` | The label of the switch in the test's settings. |
| `description` | Shown below the switch once it's turned on. |
| `examSettings` | Settings per test, shown once it's turned on. Written like [settings](/developers/apps/settings), without components. |
| `requiredPlans` | The Examplary plans the switch is available on, like `["pro", "enterprise"]`. All plans when left out. |
| `requestHeaders` | The request headers your hooks get to see, like a signature your browser adds. |
| `exitButton` | Shows a button to close the secure browser once the student has handed in the test. |
| `settingsComponent` | A component shown in the test's settings, once it's turned on. |
| `playerComponent` | A component shown to students while the prerequisite isn't met. |
| `isMet` | Whether a student may start the test. Met when left out. |
| `validateRequest` | Checks every request a student makes while taking the test. |
| `onExamUpdated` | Runs when the test is saved, e.g. to set it up on your side. |
| `functions` | Hooks your components can call. |
| `files` | Files served at a public URL, like a configuration file for your browser. |
## Checking students
Every check gets the app's settings and:
| Property | Description |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `exam` | The test: its `id`, `name`, whether it's `published`, `startDateTime`, `endDateTime`, `spellCheck`, `publicUrl` and `playerUrl`, and the workspace's `orgUrl` and `orgDomain`. |
| `examSettings` | The values of your `examSettings` for this test. |
| `data` | What `onExamUpdated` returned for this test, see [below](#setting-up-the-test-on-your-side). |
| `request` | The student's request: its `method`, `path`, `host`, and the `headers` you listed in `requestHeaders`. |
`isMet` decides whether the student can start the test. It runs when the student opens the test and when they start it. It also gets `input`, a value your player component passed with `setValue()`, which is useful when your browser tells the page something that the server can check, like a configuration key.
`validateRequest` runs on every request a student makes while taking the test, like saving an answer. Throw an error to refuse the request; its message is shown to the student. Since it runs often, keep it quick.
Your hooks only see the request headers you list in `requestHeaders`, never the student's cookies or authorization headers.
## Setting up the test on your side
When the test is saved while your prerequisite is on, Examplary calls `onExamUpdated` with the same `exam`, `examSettings` and `data`, plus `enabled`. That's the moment to create or update the test in your own platform. Whatever you return is stored for this test, and passed to every later hook as `data`:
```ts
onExamUpdated: async (settings, { exam, data, enabled }) => {
if (enabled && !data.acmeId) {
// first time enabled
return { acmeId: await createProctor(settings.apiToken, exam) };
}
if (data.acmeId && !enabled) {
// previously enabled, now disabled
await removeProctor(settings.apiToken, data.acmeId);
}
// update with the latest exam details
await updateProctor(settings.apiToken, data.acmeId, {
name: exam.name,
});
},
```
It also runs after the teacher turns the prerequisite off, as long as there's `data`, so you can archive the test on your side. When a test is duplicated, the copy starts without `data`. An error thrown here is logged, but doesn't stop the teacher from saving the test.
## Components
Both components get `settings` (the test's values of your `examSettings`), `icon` (the app's icon), and `callFunction(name, input)`, besides the [usual props](/developers/apps/building-blocks/components#props).
The **settings component** also gets the `exam`: its `id`, whether it's `published`, and its `startDateTime` and `endDateTime`. Use it for things like a link to the test's dashboard on your platform.
The **player component** gets:
| Prop | Description |
| ----------------- | --------------------------------------------------------------------------------------- |
| `exam` | The test's `id`, and `privateUrl`, the link signed-in students use to take it. |
| `user` | Whether the student is signed in (`isAuthenticated`), and their `role`. |
| `setValue(value)` | Lets the student continue to start the test, where `isMet` checks the value as `input`. |
| `fileUrl(name)` | The public URL of one of your `files`. |
## Functions
Functions are hooks your components can call with `callFunction(name, input)`, which returns the function's result. Each one says who may call it:
- `editor`: teachers who can edit the test, from the settings component.
- `student`: signed-in students, from the player component. These also get the `user`: their `id`, `name`, `firstName`, `lastName`, `email` and `signInUrl`, a link that signs them in and opens the test, valid for an hour.
```ts
functions: {
dashboardUrl: {
for: "editor",
run: async (settings, { data }) => getDashboardLink(settings.apiToken, data.acmeId),
},
openInBrowser: {
for: "student",
run: async (settings, { data, user }) =>
createSession(settings.apiToken, data.acmeId, user!.signInUrl),
},
},
```
## Files
Some secure browsers start from a configuration file. A file is a hook that returns its `contentType`, an optional `filename` and its `body`, served at a public URL that the player component gets with `fileUrl(name)`:
```ts
files: {
config: (settings, { exam, examSettings }) => ({
contentType: "application/xml",
filename: `${exam.id}.acme`,
body: buildConfig(exam, examSettings),
}),
},
```
Anyone with the URL can download the file, so don't put anything secret in it.
---
// File: apps/capabilities/settings-buttons
# Settings buttons
A `SettingsButton` adds a button to the app's card, for admins. Its `run` hook
gets the workspace's [settings](/developers/apps/settings), and the message of
an error it throws is shown to the admin:
```ts title="app.ts"
import { SettingsButton } from "@examplary/apps";
app.addCapability(
new SettingsButton({
id: "sync",
label: { en: "Sync now" },
run: async (settings) => {
await startSync(settings.apiToken);
},
}),
);
```
---
// File: apps/ai
# Building apps with AI
AI coding assistants, like Claude Code, Cursor or Codex, can build a working app
from a short description. They do best with the documentation at hand and a few
ground rules.
## Give it the documentation
[`https://examplary.ai/developers/llms.txt`](https://examplary.ai/developers/llms.txt)
holds all of the developer documentation as plain text. Point your assistant at
it, or paste it into the conversation if your assistant can't open web pages.
## Set up the app with the CLI
Start from a folder made by the
[Examplary CLI](https://www.npmjs.com/package/@examplary/cli), rather than
letting the assistant set up the project:
```shell
npx @examplary/cli init acme-gradebook --id acme.gradebook --name "Acme Gradebook"
```
This gives the assistant an `app.ts` and a `package.json` with the right
packages to build on. The CLI is also how the assistant checks its own work:
`npx @examplary/cli validate` loads the app and compiles its hooks and
components, without publishing anything.
Trying the app out with `npx @examplary/cli dev` and publishing it are best left
to you: both use your API key, and `dev` installs the app in your workspace. See
[publishing](/developers/apps/publishing).
## A starting prompt
Open the app's folder in your assistant and start with this prompt,
followed by what you want to build.
```md
You are building an app for Examplary, an online assessment and practice
platform for schools. Apps add capabilities to Examplary, like question types,
export options, dashboard pages and assessment prerequisites.
The developer documentation is here: https://examplary.ai/developers/llms.txt
Read it before you start, and follow it closely. Don't guess APIs: if something
isn't documented, don't use it.
This folder was created with the Examplary CLI (`@examplary/cli`).
Some guidelines to keep in mind:
- `app.ts` describes the app with `defineApp()` and default-exports it. Add
capabilities with `app.addCapability()` and settings with `app.addSetting()`.
Paths in the app's definition are relative to `app.ts`.
- Put each capability in its own file or folder.
- Add an `icon.svg` and reference it in `defineApp()`. A Lucide (lucide.dev)
icon is a good starting point.
- Calls to other services, and anything that needs a secret, go in hooks. Hooks
run on Examplary's servers in a sandbox: they can use `fetch()`, but no
Node.js modules.
- Keep React components small and simple. Build them with the components of
`@examplary/ui` and Tailwind CSS classes, and use Lucide React for icons.
Don't assume `@examplary/ui` has components that aren't documented: write
those yourself, in a separate file.
- Match Examplary's look: clean and minimal, with soft rounded corners and
subtle, light-colored borders.
- Texts that teachers see are translatable objects, like `{ en: "…" }`.
- Use modern TypeScript, with JSX for components. Use comments only where
they're needed.
- When you're done, run `npx @examplary/cli validate` and fix everything it
reports. Don't run `dev` or `publish`; I'll do that myself.
- If you're building question types:
- Put each question type in its own folder, with a `definition.ts` that
default-exports a `new QuestionType({...})` with all of the required
parameters.
- In most cases, you want `assessment` and `print` components, and possibly a
`results` component.
- If the question type's settings are more than a dropdown, text input or
number, add a `settings-area` component.
```
---
// File: apps/question-types/qti3
# QTI interchange
Examplary supports importing and exporting questions in [QTI 3.0](https://www.1edtech.org/standards/qti) format.
By default, custom question types are exported as [Portable Custom Interactions (PCI)](https://www.imsglobal.org/spec/qti/v3p0/impl#h.vvkq9hmm3wl3), which bundles your React component inside a QTI-compatible container. This works without any extra configuration.
If your question type maps cleanly to one of the standard QTI interaction types, you can define a native mapping via the `export.qti3` field in your question type's definition. This generates standard QTI XML instead of a PCI wrapper, and is more interoperable with other LMS systems that support native QTI interaction types.
## Basic structure
The QTI 3.0 configuration lives under the `export.qti3` key:
```ts title="definition.ts (partial)"
new QuestionType({
export: {
qti3: {
interaction: {
type: "ChoiceInteraction",
options: { ... },
},
},
},
});
```
| Field | Required | Description |
| --------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `interaction.type` | Yes | The QTI interaction type to generate. Must be one of the [supported types](#supported-interaction-types) (PascalCase). |
| `interaction.options` | No | A record of [JSONata](https://jsonata.org/) expressions whose evaluated results are passed as properties to the QTI interaction constructor. |
When no `export.qti3` config is present, the question type is exported as a Portable Custom Interaction (PCI) instead. PCI is fully supported by Examplary's own QTI packages and many third-party LMS systems.
When rendered as a PCI in an external LMS, only a subset of the [assessment component's props](/developers/apps/question-types/components#assessment-component-props) is passed through: `question`, `answer`, `saveAnswer`, `api` (always `null`, see [API calls and file uploads](/developers/apps/question-types/api-access)), `t`, and `i18n`. `isPreview` is always `false`. `isLoading`, `reviewMode`, and `environment` are not passed at all, so they'll be `undefined` — don't rely on them to decide what to render in that context.
## JSONata expressions
Values inside `interaction.options` can be either literals or [JSONata](https://jsonata.org/) expressions:
- **Literal value** — any string, number, or boolean is passed through as-is: `"shuffle": true`
- **JSONata expression** — a string starting with `=` is evaluated as a JSONata expression: `"shuffle": "=$.question.settings.shuffleOptions"`
Expressions are evaluated at export time against a root context object (`$`) containing:
- `$.question` — the full question object (id, title, description, settings, scoring, etc.)
- `$.questionType` — your question type's definition
- `$.exam` — the parent exam object
```json
// Example root context (the $.question part):
{
"id": "q_abc123",
"title": "What is the capital of France?",
"description": "Select the correct answer.",
"settings": {
"options": [
{ "text": "London", "correct": false },
{ "text": "Paris", "correct": true },
{ "text": "Berlin", "correct": false }
],
"shuffleOptions": true,
"maxSelections": 1
}
}
```
Access properties using `$.question.settings.shuffleOptions`, `$.question.description`, etc.
### Common JSONata patterns
The examples below show expression bodies (the part after the `=` prefix). When placing these inside strings in your definition, use single quotes for string literals to avoid having to escape double quotes.
```javascript
// Simple property access
$.question.settings.shuffleOptions // → true
// Conditional (ternary) — use single quotes for string literals inside JSON
$.question.settings.maxSelections = 1 ? 'single' : 'multiple' // → "single" or "multiple"
// String concatenation — single quotes avoid JSON escaping
'choice_' & $string($idx) // → "choice_0", "choice_1", etc.
// Map array to new structure
$map($.question.settings.options, function($opt, $idx) {
{ 'identifier': 'choice_' & $string($idx), 'content': $opt.text }
})
// Filter array by condition
$.question.settings.options[correct = true] // → items where correct is true
// Get identifiers for items matching a condition (position binding)
$.question.settings.options#$i[correct = true].('choice_' & $string($i)) // → ["choice_1"]
```
For comprehensive JSONata documentation, visit [jsonata.org](https://jsonata.org/). You can also use the [JSONata Exerciser](https://try.jsonata.org/) to test your expressions interactively.
## Export configuration
The `interaction.options` record defines the properties passed to the QTI interaction element. The property names and their expected values depend on the specific interaction type — refer to the [QTI 3.0 specification](https://www.1edtech.org/standards/qti) for the full list of attributes each interaction supports.
### Example: multiple choice
```ts title="definition.ts (partial)"
new QuestionType({
export: {
qti3: {
interaction: {
type: "ChoiceInteraction",
options: {
shuffle: "=$.question.settings.shuffleOptions",
maxChoices: "=$.question.settings.maxSelections",
prompt: "=$.question.description",
choices:
"=$map($.question.settings.options, function($opt, $idx) { { 'identifier': 'choice_' & $string($idx), 'content': $opt.text } })",
correctResponse:
"=$.question.settings.options#$i[correct = true].('choice_' & $string($i))",
},
},
},
},
});
```
### Example: essay / extended text
```ts title="definition.ts (partial)"
new QuestionType({
export: {
qti3: {
interaction: {
type: "ExtendedTextInteraction",
options: {
prompt: "=$.question.description",
expectedLength: "=$.question.settings.maxWords * 6",
},
},
},
},
});
```
### Example: ordering
This example assumes each item has a `correctPosition` field (1-based) that indicates its position in the correct order. `choices` uses source-order indices as identifiers; `correctResponse` sorts by `correctPosition` and maps back to those identifiers.
```ts title="definition.ts (partial)"
new QuestionType({
export: {
qti3: {
interaction: {
type: "OrderInteraction",
options: {
shuffle: "=$.question.settings.shuffleItems",
prompt: "=$.question.description",
choices:
"=$map($.question.settings.items, function($item, $idx) { { 'identifier': 'item_' & $string($idx), 'content': $item.text } })",
correctResponse:
"=$sort($.question.settings.items#$i, function($a, $b) { $a.correctPosition < $b.correctPosition }).('item_' & $string($i))",
},
},
},
},
});
```
## Text with gaps
For a question type built around a text with blanks (cloze), use `export.qti3.gaps` instead of `interaction`. The text is exported as one item with a `TextEntryInteraction` for each gap, and each gap is scored separately.
```ts title="definition.ts (partial)"
new QuestionType({
export: {
qti3: {
gaps: {
text: "=$.question.settings.text",
answers: "=$.question.settings.answers",
caseInsensitive: "=$.question.settings.caseInsensitive",
},
},
},
});
```
| Field | Required | Description |
| ----------------- | -------- | ---------------------------------------------------------------------------------------------------------------------- |
| `text` | Yes | A JSONata expression returning the text with the gaps, as HTML. |
| `marker` | No | The string that marks a gap in the text. Defaults to `___`. |
| `answers` | Yes | A JSONata expression returning the correct answer for each gap, in order. |
| `caseInsensitive` | No | A JSONata expression returning whether answers are accepted regardless of letter case. |
| `anyOrder` | No | A JSONata expression returning whether the answers may be given in any order, which accepts every answer in every gap. |
`interaction` and `gaps` are alternatives; a question type with `gaps` is not exported as a PCI.
## Supported interaction types
The following QTI 3.0 interaction types are supported. The `type` value must match exactly (PascalCase):
| Interaction type | Description | Typical use |
| ----------------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `ChoiceInteraction` | Single or multiple choice | Multiple choice questions |
| `ExtendedTextInteraction` | Long text response | Essays, open questions |
| `TextEntryInteraction` | Short text response | Fill-in-the-blank, short answer |
| `OrderInteraction` | Arrange items in order | Sequencing, ranking |
| `MatchInteraction` | Match pairs of items | Matching exercises |
| `InlineChoiceInteraction` | Dropdown within text | Cloze tests |
| `HotspotInteraction` | Click on image regions | Image-based questions |
| `GapMatchInteraction` | Drag items into gaps | Drag-and-drop fill-in |
| `SliderInteraction` | Numeric slider | Numeric estimation |
| `UploadInteraction` | File upload | Document submissions |
| `DrawingInteraction` | Freeform drawing | Diagrams, sketches |
| `MediaInteraction` | Audio/video response | Recording responses |
| `AssociateInteraction` | Associate pairs | Association exercises |
| `GraphicAssociateInteraction` | Associate pairs on image | Image-based association |
| `GraphicGapMatchInteraction` | Gap match on image | Image-based drag-and-drop |
| `GraphicOrderInteraction` | Order items on image | Image-based sequencing |
| `HottextInteraction` | Select text regions | Hottext selection |
| `PortableCustomInteraction` | Portable Custom Interaction | Encapsulates a custom question type as a self-contained portable bundle; use in `import.qti3` to match PCIs exported from other systems |
| `PositionObjectInteraction` | Position object on image | Image positioning |
| `SelectPointInteraction` | Select point on image | Point selection on image |
| `EndAttemptInteraction` | End attempt button | Manual submission |
## Import configuration
When a QTI file is imported into Examplary, questions with standard interaction types are matched to your question types using the `import.qti3` field in your question type's definition. This tells Examplary how to reconstruct your question type's settings from the incoming QTI data.
The import configuration lives under the `import.qti3` key:
```ts title="definition.ts"
new QuestionType({
import: {
qti3: {
interaction: {
type: "ChoiceInteraction",
},
condition: "=$.interaction.maxChoices = 1",
settings: {
shuffleOptions: "=$.interaction.shuffle",
options:
"=$map($.interaction.choices, function($c) { { 'text': $c.content, 'correct': $c.identifier in $.correctResponse } })",
maxSelections: "=$.interaction.maxChoices",
},
},
},
});
```
| Field | Required | Description |
| ------------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `interaction.type` | Yes | The QTI interaction type this import mapping applies to. Must be one of the [supported types](#supported-interaction-types) (PascalCase). |
| `condition` | No | A JSONata expression (starting with `=`) that must evaluate to `true` for this mapping to be selected. Use this to distinguish between multiple question types that share the same interaction type (e.g. single vs. multiple choice). |
| `settings` | No | A record of [JSONata](https://jsonata.org/) expressions that map QTI data to your question type's settings fields. |
### Import context
Expressions in `condition` and `settings` are evaluated against a root context object (`$`) containing:
| Variable | Type | Description |
| ------------------- | ---------- | ------------------------------------------------------------------------------------------- |
| `$.interaction` | object | The raw QTI interaction element as a plain JavaScript object. |
| `$.response` | object | The interaction's response declaration, for example with an `areaMapping` for select point. |
| `$.correctResponse` | `string[]` | Array of correct response identifiers from the QTI `correctResponse` element. |
| `$.points` | `number` | The point value assigned to the question in the source QTI file. |
### Importing text with gaps
To import QTI items with gaps in a text (several text entries, dropdowns, word bank gaps, or one text entry inside a line of text), set `import.qti3.gaps` to `true` instead of `interaction`. The first question type that is not deprecated and has `gaps` set handles all such items.
```ts title="definition.ts (partial)"
new QuestionType({
import: {
qti3: {
gaps: true,
settings: {
text: "=$.text",
answers: "=$.answers",
caseInsensitive: "=$.caseInsensitive",
anyOrder: "=$.anyOrder",
},
},
},
});
```
The `settings` expressions are evaluated against `{ text, answers, choices, caseInsensitive, anyOrder, points }`, where each gap in `text` is marked as `___` and `answers` holds the correct answer per gap, in order. `choices` lists the options per gap when any gap offers them (dropdowns and word banks), and is an empty list for plain text entries. `anyOrder` is `true` when every gap accepts every answer.
### Importing selectable text
To import QTI items with selectable text (hottext), set `import.qti3.hottext` to `true` instead of `interaction`. The `settings` expressions are evaluated against `{ text, answers, selectableHidden, points }`:
| Variable | Description |
| ------------------ | ------------------------------------------------------------------------------------------ |
| `text` | The text, with each selectable part marked as `[part]`. |
| `answers` | The correct parts, as they appear between the brackets in `text`. |
| `selectableHidden` | `true` when the item hides which parts are selectable (the `qti-unselected-hidden` class). |
### Example: single vs multiple choice
If you have two question types that both use `ChoiceInteraction` — one for single-choice and one for multiple-choice — use `condition` to tell them apart:
```ts title="definition.ts (single-choice)"
new QuestionType({
import: {
qti3: {
interaction: {
type: "ChoiceInteraction",
},
condition: "=$.interaction.maxChoices = 1",
settings: {
options:
"=$map($.interaction.choices, function($c) { { 'text': $c.content, 'correct': $c.identifier in $.correctResponse } })",
shuffleOptions: "=$.interaction.shuffle",
},
},
},
});
```
```ts title="definition.ts (multiple-choice)"
new QuestionType({
import: {
qti3: {
interaction: {
type: "ChoiceInteraction",
},
condition: "=$.interaction.maxChoices > 1",
settings: {
options:
"=$map($.interaction.choices, function($c) { { 'text': $c.content, 'correct': $c.identifier in $.correctResponse } })",
maxSelections: "=$.interaction.maxChoices",
shuffleOptions: "=$.interaction.shuffle",
},
},
},
});
```
When multiple question types define a `condition` for the same interaction type, Examplary evaluates them in order and selects the first match. If no `condition` is defined, the mapping always matches that interaction type.
Deprecated question types are excluded from this generic interaction-type matching step: their `import.qti3` mapping is skipped, even if its `interaction.type`/`condition` would otherwise match. Matching continues to the next non-deprecated type whose mapping matches, or falls back to a built-in type if none does.
## Testing your configuration
### Testing export
1. Create a test exam with your custom question type
2. Export the exam to QTI format from the Examplary UI
3. Verify the generated XML matches your expectations
Use the [JSONata Exerciser](https://try.jsonata.org/) to debug export expressions. Set up your context as `{ "question": {...}, "questionType": {...}, "exam": {...} }` and test each expression individually.
### Testing import
1. Obtain a QTI file containing the interaction type your mapping targets
2. Import the QTI file into an exam from the Examplary UI
3. Verify the question settings were reconstructed correctly from the imported data
To debug import expressions, use the [JSONata Exerciser](https://try.jsonata.org/) with a context of `{ "interaction": {...}, "response": {...}, "correctResponse": [...], "points": 1 }` that matches the structure of the QTI interaction you're importing.
## Troubleshooting
### Expression returns undefined
Make sure the path exists in your data. Use the conditional operator to provide defaults:
```javascript
$.question.settings.shuffleOptions != null
? $.question.settings.shuffleOptions
: true;
```
### Literal strings in expressions
Inside a `=`-prefixed expression, use single quotes for string literals — this avoids having to escape double quotes within the string:
```ts
{
cardinality: "='single'";
}
```
Single quotes inside an expression also work inline within larger expressions:
```ts
{
cardinality: "=$.question.settings.maxSelections = 1 ? 'single' : 'multiple'";
}
```
### Array indexing issues
Remember that JSONata arrays are zero-indexed. Use the `#$i` position binding to get the index of items in an array:
```javascript
// Get all items matching a condition with their original indices
$.question.settings.options#$i[correct = true].('choice_' & $string($i))
// → ["choice_1"] for the example above where Paris (index 1) is correct
```
## Learn more
- [QTI 3.0 Specification](https://www.1edtech.org/standards/qti)
- [JSONata Documentation](https://docs.jsonata.org/)
- [JSONata Exerciser](https://try.jsonata.org/)
- [Importing QTI files](/developers/guides/importing/qti)
---
// File: guides/question-object
# The Question object
Questions are the core building block of the Examplary platform. The same Question object shape is used everywhere a question appears: in the REST API (creating exams, managing exam questions, and the question bank), in [embed session flows](/developers/embed-sessions), and as the `question` prop passed to [custom question type components](/developers/apps/question-types/components).
This guide describes the structure of that object and what each field is for. A machine-readable [JSON Schema of the Question object](https://schemas.examplary.ai/question.json) is also available, which you can use to validate questions or generate types in your own system.
## Example
```json
{
"id": "q_qQjXQasmUfpWRV5KO5mzkxoy",
"v": 3,
"type": "single-line-text",
"title": "Napoleon's role in the French Revolution",
"description": "When Napoleon Bonaparte rose to power, he was seen as a hero of the French Revolution. However, his actions and policies during his rule have been debated by historians. What was Napoleon's role in the French Revolution?",
"settings": {
"maxLength": 200
},
"scoring": {
"rubricType": "analytical",
"criteria": [
{
"id": "c1",
"title": "Correctness",
"levels": [
{ "id": "c1l1", "title": "Correct", "points": 1 },
{ "id": "c1l2", "title": "Partially correct", "points": 0.5 },
{ "id": "c1l3", "title": "Incorrect", "points": 0 }
]
}
],
"modelAnswer": "Napoleon rose to prominence as a general during the Revolution and later seized power, ending the revolutionary period while consolidating some of its reforms."
},
"tags": ["history", "french-revolution"],
"externalId": "my_question_1",
"metadata": { "my_service_internal_id": "abc1234" }
}
```
## Fields
| Field | Type | Description |
| -------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | string | The question's ID, assigned by Examplary. |
| `v` | number | Version number, incremented each time the question is edited. |
| `type` | string | ID of the question type, e.g. `single-line-text` or `examplary.expansion.matching`. **Required.** |
| `title` | string | The question stem shown to the student. Can contain HTML — see [Rich text and math notation](#rich-text-and-math-notation). |
| `description` | string | Optional supporting text shown below the title, such as context or instructions. Can contain HTML — see [Rich text and math notation](#rich-text-and-math-notation). |
| `settings` | object | Settings specific to the question type. **Required** (may be an empty object). |
| `scoring` | object | The rubric configuration set by the teacher. See [Scoring](#scoring) below. |
| `printing` | object | Print-layout overrides for the question, e.g. `answerBoxHeight` (the height of the answer box in print mode, in pixels). |
| `tags` | string[] | Tags associated with the question, used for categorisation and search. |
| `externalId` | string | Optional ID you provide when creating the question via the API, used to correlate questions back to your own system. |
| `metadata` | object | An arbitrary key-value store for external system data (values may be strings, numbers, booleans, or `null`). Passed through unchanged. |
| `questionBankItemId` | string | The ID of the [question bank](/developers/rest-api) item this question was added from, if any. |
| `traceIds` | string[] | Trace IDs used internally to track AI generation processes. You can ignore this field. |
### Rich text and math notation
`title` and `description` accept a safe subset of HTML for basic formatting (e.g. ``, ``, `