# Randomization Framework

The current version of our randomization framework classifies randomization into six levels ranging from 0 to 5.
Levels in this framework are **not** necessarily on a quality hierarchy from worst to best.
In fact, a question is allowed to belong to multiple levels. Questions of relatively higher complexity, such as multi-part questions with many question parameters, may have the potential for a mix of level 1-4 randomization. Further note that level 5 randomization (highlighted in italics) mainly applies to assessments since it requires entirely independent variants.

As an example, this can be achieved by implementing a large question bank from which the assessment questions for **each student** are randomly chosen.

| Level | Label                 | Description                                                                      | Purpose                                                                                                            | Example                                                                                                                                              |
|-------|-----------------------|----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| 0     | Unrandomized          | Every student receives precisely the same question.                              | Classify questions that are not randomized at all.                                                                 | Abdallah uses a perfect binary tree to implement a small database system with 63 nodes. How many leaves will this tree have?                         |
| 1     | Surface Features      | Surface level features (e.g., names, colours, phrases) change for each variant.  | Increase cognitive load for students when they are pattern-matching.                                               | \{\{ Abdallah \}\} uses a perfect binary tree with 63 nodes to implement a \{\{ small database system \}\}. How many leaves will this tree have?     |
| 2     | Conditions            | Within a single problem scenario, conditions and values change for each variant. | Offer opportunities for repeated retrieval practice and more productive group work.                                | Abdallah uses a perfect binary tree to implement a small database system with \{\{ 63 \}\} nodes. How many leaves will this tree have?               |
| 3     | Scenarios             | Problem scenarios change for each variant, assessing a single concept.           | Enable students to develop strategies, algorithms, and procedures to solve multiple problem scenarios.             | Abdallah uses an \{\{ unbalanced \}\} binary tree to implement a small database system with 63 nodes. What \{\{ max height \}\} will this tree have? |
| 4     | Concepts              | Randomized variations lead to assessment of different concepts.                  | Diversify question style so students need to synthesize knowledge across concepts and cannot easily pattern-match. | Abdallah uses an \{\{ M-ary tree with $m=4$ \}\} to implement a small database system with 63 nodes. How many leaves will this tree have?            |
| *5*   | *Different Questions* | *Each question variant is entirely independent.*                                 | *Provide students with authentic re-assessment opportunities to demonstrate proficiency.*                          | *\{\{ Abdallah uses a perfect binary tree to implement a small database system with 63 nodes. How many leaves will this tree have? \}\}*             |