Examplary's AI to generate test questions doesn't work the exact same way as asking ChatGPT or Copilot to generate some questions.
Whilst the underlying technology is the same, Examplary provides the AI model with additional tools and information to generate high-quality content, and we're always working on improving this system and evaluating the quality of its output.
Always based on your source materials
When you first create a test or an individual question, you'll be asked to provide source materials. These can be Word documents, PDFs, audio and video.
The more information you upload, the better. If you have a past test that is about similar topics, upload it. The teacher's manual of your text book? Add it. A recording of a lecture? Definitely useful.
Examplary's AI is trained to follow your source materials as closely as possible, to avoid hallucinations.
A thorough flow
Examplary starts by extracting facts from the source materials, mapping out topics, and looking at our pedagogical principles to create an outline of a test.
It always based this outline on a taxonomy, like Bloom's Taxonomy of Learning (but you can also select others or create your own). This creates structure: the taxonomy defines the balance between recall-questions and questions that make students combine and apply the principles they learned.
It also uses specific instructions per student level and subject you can select to tailor the difficulty and language usage of the questions, along with the other settings and instructions you supplied.
A set of tools
We also provide the AI model with additional tools to be able to generate graphs and mathematical equations, check calculations, make references, and more. These improve the richness of the questions, and ensure a certain level of quality.
A fun feature we support here is called 'page clippings', which allows the model to copy over relevant illustrations or diagrams from your source materials to use in questions, further pinning the questions to the supplied sources.
Measured and constantly improved
We constantly run what are called 'evals' on our question generation flow. These are a suite of tests which evaluate the quality of the outputs generated by the AI, and whether they adhere to our quality standards and known best-practices for assessment item design. This results in a percentage score.
On a weekly basis our team works to increase that score by making small tweaks to our prompts, tools and other inputs the model receives. In addition, we also keep upgrading the models themselves when our AI providers release better iterations of the underlying LLM models we use.