# QuML Overview
Assessment has always played an important role in education. Most, if not all, types of formal education use some sort of assessment, typically including a final exam to earn a grade, a degree, a license, or some other form of qualification.
Today, assessment is no longer restricted to grading at the end of an instruction (summative assessment), but it has been recognized that assessment is also useful for continuous monitoring & feedback (formative assessment) and guiding of the learning progress (means to learn), without being necessarily used for grading purposes.
## Questions for continuous feedback
Formative assessment, including self-assessment, can play a vital role in motivating students since it provides them with a way to judge their own competency level and allows them to track their progress. It also enables students to identify areas where more work is required, and to thereby remain motivated to improve further. Of course, this requires that students receive feedback as quickly as possible.
Formative assessment also provides timely feedback for teachers, both with respect to the effectiveness of the teaching and the performance of the students; it thus helps to identify points that might need clarification.
For both groups, teachers and students, frequent testing is preferable. Infrequent testing makes each exam a “major event”, with students investing much effort into preparation and they may even stop attending class to prepare for the exam. With infrequent tests, students may be unable to determine whether they are studying the right material and with sufficient depth.
> ### Though it may be more time consuming for teachers, frequent testing reduces the importance of each individual exam and helps students to better gauge their progress.
Questions from a question bank can be used to create different types of formative assessments. Presenting questions from a question bank has many advantages. Firstly, it can ensure that questions are always fresh, do not become stale and repetitive and the questions can evolve in their precision of measuring the student’s proficiency.
```
Figure: Summative and Formative Assessments
```
## Questions as a means to learn
Answering questions and solving problems is an effective way to learn. Unlike traditional tests where the questions are used to measure the proficiency of a student, if questions are presented as learning tools, they will encourage students to do a full and meaningful enquiry about the related concepts. In fact, one of the main objectives of questions should be “achieving defined goals”. A student should be asked questions that will require him or her to use the skills that he or she is trying to learn. Such questions should be more about provoking a process of learning than about finding an answer.
If the questions focus on micro-concept level assessment of student’s proficiency, it is possible to identify strengths, areas that need focus & improvement, and recommend the relevant content to the student that specifically address the individual learning needs.
Questions that are tagged with appropriate pedagogic metadata and associated with relevant concepts will enable “questioning” to be used as an effective means to encourage learning.
```
Figure: Questions in a question bank with proper tagging and associations will enable multiple use-cases
of questions
```
## Need for a common standard
Assessment is always a time-consuming activity for teachers, especially if large numbers of students are to be assessed or, if assessment is frequent. This has motivated the development of technical devices to support assessment, starting with relatively simple mechanical devices and evolving to today’s computer-aided assessment (CAA) or e-assessment.
E-assessment is one of the fundamental elements of e-learning. E-assessments have a number of practical advantages; in particular, scoring can be automated (to most extent). This makes them especially attractive in e-learning settings, as it allows to make assessment available “anyplace, anytime”.
Creating high-quality e-assessments, however, is challenging, especially if they are to assess higher-order cognitive levels, such as application, analysis, synthesis, and evaluation in the traditional taxonomy of Bloom.
> ### While e-assessments are indeed attractive, they are extremely expensive to construct: question creation is a highly refined and time-consuming art, especially if one expects to develop good questions that are relatively unambiguous.
Ensuring the reusability, longevity, and platform independence of tests can mitigate the high costs of creation and can help preserve investments and intellectual assets when hardware and software change, thus ensuring sustainability. This requires a standard, platform-neutral, vendor-independent interchange file format for e-assessments.
```
Figure: A common standard will enable distributed assessments authoring and delivery
```
The common standard should enable the following capabilities for e-learning systems:
#### ➢ Provide a standard content format for storing and exchanging questions independent of the authoring tool used to create them.
#### ➢ Support the deployment of question banks across a wide range of learning and assessment delivery systems.
#### ➢ Provide a standard content format for storing and exchanging tests independent of the test construction tool used to create them.
#### ➢ Support the deployment of questions, question banks and tests from diverse sources in a single learning or assessment delivery system.
#### ➢ Provide systems with the ability to report test results in a consistent manner.
A number of standards aiming to promote interoperability and sustainability of e-learning content and e-assessment have emerged over the last couple of decades. IMS Question and Test Interoperability (IMS QTI) is the best-known standard for tests. However, IMS QTI is not used as the standard here primarily because of the following two reasons:
1. QTI is a very large specification with many optional parts. It is very difficult to
ensure interoperability between systems that do not implement all parts of the
standard.
2. Although we see familiar elements like and in the specification,
everything is in the QTI namespace and not in the XHTML namespace. The standard does not prescribe that you
have to render it with a browser, which leaves a lot of room for interpretation.
So, consistent rendering is a major problem.
This document specifies a model (for questions and tests) which addresses the above concerns with IMS QTI specification. The proposed model is derived from the IMS QTI specification, with changes made to meet the following requirements.
### Formal Definition
A standard or a specification intended as such should rely as far as possible on existing and proven standards. This allows keeping the specification of the interchange format concise and enables implementers to make use of available and proven tools. The proposed model is primarily based on HTML and Javascript.
> ### HTML and javascript, being well-defined and globally acknowledged standards, makes it simple for implementations to interpret and transfer questions and tests in the form intended by the original author.
### Composability
There are different scenarios for the interchange of tests. In some cases, tests may be reused in their entirety, whereas in other cases only individual questions from a test or a test collection (question bank) may be integrated into another test. It is therefore essential to clearly separate the different aspects of tests, especially individual questions and sets in a test.
### Size and Scope
A model that is able to describe all of the functionality of all systems may seem desirable to enable the interchange of complete tests with all their properties. Looking closer, one can see that this requirement is illusionary: The facilities of test systems are too diverse and too varied and no system supports all test and scoring types.
A model should therefore restrict itself to a relatively small “core set” of test and question types. Further types can be added later on once it has become clear which types are actually required in practice.
### Longevity
Finally, a specification should ensure longevity, i.e., questions and tests described using the format should remain processable for as long as possible. For correcting errors and extending the format, new revisions will become necessary from time to time, but it must be avoided that new revisions interfere with the data interchange. This means that different revisions should be compatible with each other as far as possible. Gratuitous incompatibilities must be avoided at all costs; sometimes, however, incompatible changes may be necessary.
To mitigate the potential negative impact of such changes, rigorous revision management is essential, starting with the distinction between major and minor revisions and corresponding numbering schemes. Incompatible changes must then only be introduced in major revisions, after having been announced before. Inside a major revision, say, 1 .x, all minor revisions (e.g., 1. 1 , 1. 2 , etc.) are all compatible with each other. Revision 2. 0 may introduce incompatible changes, but not 2. 1. This ensures that implementers and users can easily and reliably decide whether a specific question or test can be processed or not.
## QuML Specification
Question Markup Language (QuML in short) is a specification for storage, rendering and distribution of Questions and Tests. QuML allows assessment materials to be authored and delivered on multiple systems interchangeably. It is designed to facilitate interoperability between systems.
> ### QuML defines a standard format for representation of questions, tests and their results, supporting the exchange of this material between authoring and delivery systems, repositories and other e-learning systems.
This specification enables aggregating questions from a wide variety of existing sources (Question papers, PDFs, documents, teachers, individuals, etc) into one repository and stored in a common format. With all questions available in a repository and in the same format, the reach and applicability of these questions gets amplified multi-fold. When these questions are available with QuML as the interface language, multiple applications (which understand & render QuML) can get created to serve a wide variety of use cases and users.
In a nutshell, a microservices architecture centered around “questions”, with different platforms & applications as constituent services, is possible. It allows anyone to offer a scalable, entirely new application that uses QuML questions, effortlessly.
```
Figure: Services centered around “QuML Questions”
```
A lot of questions currently exist in PDFs, word documents, as exam papers (soft and hard copies), in the minds of teachers & other creators, in existing question repositories and in many other sources. And on the demand side, teachers, students & parents have access to some of those resources only - mainly because existing systems have access to questions from at most one source. And some of the questions are completely not accessible as there are no systems or services built for them.
> ### QuML powered question repositories create more connections between existing supply of questions and the demand for learning (via questions).
If questions from multiple sources are imported into a QuML compliant question repository, multiple systems get access to all the questions. It is possible because systems need to adhere to only one specification (QuML), unlike before, where each question is stored and represented in a different way.
# Question Markup Language (QuML) Specification - Draft
Question Markup Language (QuML in short) is a specification for storage, rendering and distribution of Questions and Tests. QuML allows assessment materials to be authored and delivered on multiple systems interchangeably. It is designed to facilitate interoperability between systems.
This specification is based on [IMS QTI Specification 2.2](http://www.imsglobal.org/question/index.html#version2.2) with necessary extensions. This specification describes the information model for questions and tests used in assessments. Formally these are known as Question and Question Set. QuML specification describes a model for the representation of questions and question sets data and their corresponding results reports. Therefore, the specification enables the exchange of the question, question set and results data between authoring tools, question banks, test constructional tools, learning systems and assessment delivery systems.
## Documentation
* [**Overview**](/): Importance of questions in learning and the need for a specification like QuML.
* [**Common Elements**](/v1/common): The reference guide to the data model of common elements used in this specification.
* [**QuML for Questions**](/v1/question): The reference guide to the data model for questions. The document provides detailed information about the model and specifies the requirements of delivery engines and authoring systems.
* [**Question Schema**](https://github.com/sunbird-specs/QuML/tree/c4d260d5d6ac8a461d9f131e3a4031b2b542bb6b/v1/question-schema.json): JSON schema for QuML questions.
* [**QuML for Tests**](/v1/questionset): The reference guide to the data model for question sets, which are used to represent tests.
* [**Test Schema**](https://github.com/sunbird-specs/QuML/tree/c4d260d5d6ac8a461d9f131e3a4031b2b542bb6b/v1/question-set-schema.json): JSON schema for QuML tests.
* [**Library Methods**](/v1/methods): The reference guide to methods to be implemented by QuML implementations.
* [**Telemetry**](/v1/telemetry): The reference guide to track usage, and report results.
* [**Appendix**](/v1/appendix): Additional information and references.
* [**Samples**](/v1/samples): This folder has sample questions and question sets represented using QuML.
# v1
V1 of the QuML specification
This is a container for V1 of the QuML specification
# Telemetry
The word ‘Telemetry’ is derived from its Greek etymological roots, **tele** - remote and **metron** - measure. In today’s world, *Telemetry* is a term used for technologies that automatically record and measure statistical data from real-world use and forward it to systems for further analysis and study.
Usage is captured in form of events. ‘Events’ are broad, human-readable actions. Events are used to categorize telemetry data. They are the basic unit for analytics and help identify user navigation or flow.
> ## The concept of telemetry events is to identify: Who did what, on what, and where, using what, in relation to what?
QuML implementations should capture all user interactions (explicit and implicit), generate telemetry events and send them to the QuML repository. Analysis of telemetry offers insights into behaviour and usage patterns, and thereby drive decisions, recommendations, and outcomes.
```
Figure: Telemetry generation and processing
```
QuML recommends usage of Sunbird platform’s [telemetry specification](http://docs.sunbird.org/latest/developer-docs/telemetry/overview/) as the standard for telemetry events. QuML implementations should generate the telemetry events defined in the sunbird telemetry specification during the usage of questions and question sets.
For example, during a question session, the student goes through a series of steps. Each step produces one or more telemetry events. The journey starts with the student attempting a question. Student provides a response to the question, response is evaluated and student sees the result. Other alternate paths during the flow include when the student's response is incorrect and student sees the solution. Similarly the student may abandon the journey at any point (end the question session). Telemetry generated during this journey and the metrics computed using those events are indicated in figure below.
```
Figure: Sample telemetry events and metrics in a question attempt
```
An individual event in isolation is of little value. However, when we see across a series of events that represent a user journey, the insights become more powerful. Processing of the telemetry events over a journey provides insights and equip systems using the QuML questions to provide a wide range of other services in addition to question storage and delivery:
### ➢ Track hard spots for a student and enable systems to provide personalised learning
### ➢ Provide feedback to question authors on how to improve or change the question
### ➢ Identify concepts that need focus at an individual, class or a school level
### ➢ Set usage metadata for questions and question sets
### ➢ Update other metadata of questions and question sets based on usage
# Common Elements
All of the common elements used within this specification are described in this Document. Common elements include the data types, and their properties used in this specificaiton.
## Cardinality
All the variables in the specification will have a cardinality. The cardinality attribute defines the number of values allowed for the variable and whether the variables are ordered.
Supported types of cardinality are:
* *single*: only one value is allowed for the variable
* *multiple*: one or more values are allowed for the variable
* *ordered*: ordered list of values are allowed for the variable
## Data Types
### string
String data types should be used to represent string literal values.
### integer
Integer data types should be used to represent numbers without a fractional part.
### float
Float data types should be used to represent numbers with a fractional part.
### boolean
Boolean data types should be used to represent values with two states: true or false.
### uri
URI data types should be used to represent Uniform Resource Identifier (URI) reference values.
### map
Map data type should be used for variables whose values are key-value pairs. Map is a non-primitive data type with the following attributes.
| Attribute | Schema | Description |
| --------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| key | dataType: *string*, required: *true* | a map cannot contain duplicate keys and each key can map at most one value |
| value | dataType: *any*, required: *false*, defaultValue: *NULL* | value can be of any other data type defined in QuML. value is optional and is by default set to NULL, if not provided |
### coordinate
Coordinate data type should be used to represent coordinates (x and y) of a single point in a canvas.
| Attribute | Schema | Description |
| --------- | ------------------------------------- | ------------------------------- |
| x | dataType: *float*, required: *true* | x-coordinate value of the point |
| y | dataType: *float*, required: *true* | y-coordinate value of the point |
### points
Points data type should be used for variables whose values represent an area on a canvas. Points are made up of a shape and the coordinates that define the boundary of the area.
| Attribute | Schema | Description |
| ----------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| shape | dataType: *string*, required: *true*, range: *“point”, “circle”, “ellipse”, “poly”, “rect”* | represents the shape of points. it must be equal one of the predefined values defined in the range |
| coordinates | dataType: *list of coordinate objects*, required: *true* | Coordinates should have at least one coordinate object. The number of coordinates depend on the shape. |
### media
Media data type should be used for representing a single media file. Media are re-usable to create assets which are used in questions and/or tests.
| Attribute | Schema | Description |
| --------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| id | dataType: *string*, required: *true* | identifier of the media object |
| mimeType | dataType: *string*, required: *true* | technical mime type of the asset used by QuML players to understand the format of the asset. Supported types are image/png, audio/mp3, video/mp4, and video/webm |
| mediaType | dataType: *string*, required: *true* | type of the asset. this should be auto-derived from the mimeType value, current supported types are image, audio and video |
| src | dataType: *string*, required: *true* | path of the media file. QuML players will use this value to load all the media used in a question/test |
| baseUrl | dataType: *string*, required: *false* | baseUrl of the server where the media is stored. Base URL can be used by QuML players to load media in a browser without CORS issues. this is achieved by QuML server proxying path to the provided base url. |
# Library Methods
QuML players should provide implementation for the library methods which can be used in response, template and outcome processing of questions and question sets.
## Question Library Methods
### getResponseVariable
Returns the value of a response variable. If no response variable is found or if the value is not set, NULL value will be returned.
*Parameters:*
* name of the response variable
*Returns:*
* value of the response variable
### getOutcomeVariable
Returns the value of a outcome variable. If no outcome variable is found or if the value is not set, NULL value will be returned.
*Parameters:*
* name of the outcome variable
*Returns:*
* value of the outcome variable
### setOutcomeVariable
Sets the specified value for specified outcome variable.
*Parameters:*
* name of the outcome variable
* value to be set for the specified outcome variable
*Returns:*
* true, if the outcome variable is updated successfully
* false, if the outcome variable is not updated, e.g.: if variable is not declared or if the data type of the value does not match with the declaration
### getTemplateVariable
Returns the value of a template variable. If no template variable is found or if the value is not set, NULL value will be returned.
*Parameters:*
* name of the template variable
*Returns:*
* value of the template variable
### setTemplateVariable
Sets the specified value for specified template variable.
*Parameters:*
* name of the template variable
* value to be set for the specified template variable
*Returns:*
* true, if the template variable is updated successfully
* false, if the template variable is not updated, e.g.: if variable is not declared or if the data type of the value does not match with the declaration
### matchCorrect
Compares the value of response variables with the correct response value configured in the response declaration. If values of all variables match with the correct response, the outcome variable SCORE is set to 1.0, else if the values do not match or any of the variables do not have correct response definition, SCORE is set to 0.0. If no response variables are defined, SCORE is set to NULL.
*Returns:*
* the value of outcome variable SCORE - either 1.0, 0.0 or NULL
### mapResponse
Sets the value of outcome variable SCORE using mapping and areaMapping configuration of response variables provided in the response declaration. If mapping data is not present for any of the response variable, the response variable is not used for computing the score. Optionally if a mapping config is provided, other outcome variable values are also set using the mapping config.
*Parameters:*
* mappingConfig: optional parameter to set values of outcome variables other than SCORE
### matchTemplate
Sets the value of outcome variable SCORE using matchTemplate configuration passed as input to the method. Optionally if a mapping config is provided, other outcome variable values are also set using the mapping config.
*Parameters:*
* matchTemplateConfig: required config to set the value of outcome variable SCORE based on the values of one or more template variables
* mappingConfig: optional parameter to set values of outcome variables other than SCORE
## Question Set Library Methods
### getQuestionSets
Returns the identifiers of question sets that are members of the question set.
*Returns:*
* list of question set identifiers, NULL if the question set do not have any member sets
### getQuestions
Returns the identifiers of questions that are members of the question set.
*Returns:*
* list of question identifiers, NULL if the question set do not have any member questions
### getQuestionSetOutcomeVariables
Returns a map of outcome variables and their values for the specified question set identifier. The question set should be a member of the current question set.
*Parameters:*
* identifier of the question set which is a member of the current question set
*Returns:*
* map of outcome variables and values of the specified question set, NULL if the question set is not a member of the current question set
### getQuestionSetDuration
Returns the time spent (in milliseconds) for the specified question set identifier. The question set should be a member of the current question set.
*Parameters:*
* identifier of the question set which is a member of the current question set
*Returns:*
* duration of the specified set in the current session, NULL if the question set is not a member of the current question set or if the question set is not attempted by the user in the current session
### getQuestionOutcomeVariables
Returns a map of outcome variables and their values for the specified question identifier. The question should be a member of the current question set.
*Parameters:*
* identifier of the question which is a member of the current question set
*Returns:*
* map of outcome variables and values of the specified question, NULL if the question is not a member of the current question set
### getQuestionResponseVariables
Returns a map of response variables (including built-in variables numAttempts & duration) and their values for the specified question identifier. The question should be a member of the current question set.
*Parameters:*
* identifier of the question which is a member of the current question set
*Returns:*
* map of response variables and values of the specified question, NULL if the question is not a member of the current question set
### getOutcomeVariable
Returns the value of a outcome variable. If no outcome variable is found or if the value is not set, NULL value will be returned.
*Parameters:*
* name of the outcome variable
*Returns:*
* value of the outcome variable
### setOutcomeVariable
Sets the specified value for specified outcome variable.
*Parameters:*
* name of the outcome variable
* value to be set for the specified outcome variable
*Returns:*
* true, if the outcome variable is updated successfully
* false, if the outcome variable is not updated, e.g.: if variable is not declared or if the data type of the value does not match with the declaration
# List of Appendices
## Appendix 1: Connected Question Banks
A potential possibility with QuML is **connected question banks**. Imagine a user has access to a question bank with questions in one language and the user has a need to get the same question in a different language (that exists in another question bank): it is possible if a connection (virtual) can be established between questions in different question banks.
With QuML, all questions have the same metadata categories (e.g. difficulty level, topics, subject, medium, etc) and data structure. If values of all metadata categories and data structure are the same for two questions, then an equivalence relation can be established between them. And if only medium and text data is different for two questions, both the questions can be connected as translations of each other.
```
Figure - Connected question banks
```
> ## Connected Question Banks: Get or Give Questions & Tests - Anywhere, Anyhow, Anytime & Anything.
A global question index can be created where an index of questions from multiple QuML compliant question banks is maintained. Users can gain access to questions in remote question banks via the global question index. This is a powerful idea which can be very helpful in countries where there are diverse education boards, publishers and governing bodies that provide learning facilities. Students, Teachers and Parents across the country can gain access to good quality questions created by a different education board, publisher or governing body in another part of the country via the global question index.
## Appendix 2: Offline access
## Appendix 3: Printable format
# QuML for Questions
Digital assessments become easier with a standard format for the exchange of questions and structures, scoring information, the ability to analyze results, and ways to accommodate students’ personal needs and preferences. Digital assessments are especially important at a time when most institutions are looking to eliminate or greatly reduce the use of paper in classrooms within the next few years. QuML specification for questions defines a standard format that enables interoperability of questions and digital assessments.
This document has the following sections:
1. [Anatomy of a Question](/v1/question#anatomy-of-a-question) - the structure and composition of a QuML question
2. [Conceptual Model](/v1/question#conceptual-model) - a conceptual model of the actors and components related to QuML quetions
3. [Question Data Model](/v1/question#question-data-model) - the core that defines the actual question data exchange (including the content itself)
4. [Question Metadata](/v1/question#question-metadata) - QuML specific metadata
## Anatomy of a Question
Traditionally, questions are monolithic in nature and are used like black boxes. Such questions are primarily used for only one purpose - getting a score when a user interacts with the question. There is no (or very little) knowledge of what the question consists of, what is the purpose and for whom it is intended for. This limits the re-use of questions in different scenarios and also by different systems.
```
Figure: Questions used only for SCORE output (black box model)
```
QuML introduces a model where a question incorporates the information that defines the question, information that is presented to a student, instructions of how to present to the student & instructions of how to score the question. For example, in QuML, every interaction of student with the question gets captured as response variables and scoring takes place when student responses are transformed into outcomes by response processing rules. The response processing rules and the response variables are well defined entities in QuML.
> ## In QuML, question is a first class citizen. It can be used for different purposes, and in different forms, mainly due to its modular structure.
There are multiple components in a QuML question. These components can be broadly classified into four categories.
### ➢ Content: The content category comprises of the data that is needed to render and process the question. It includes labels & texts, responses and outcomes of the question.
### ➢ Appearance: This category comprises of the sections that define the user experience of the question. It includes the HTML markup and styles to format the question.
### ➢ Behaviour: This category comprises of the processing logic for assigning values to template variables and evaluating the student’s responses.
### ➢ Metadata: This category comprises of the data that is needed to search, discover and compose questions into tests.
```
Figure: Anatomy of a QuML Question
```
Since a QuML question is not monolithic in nature and has a well-defined structure, a question can be deconstructed and used for different purposes. For example, a testing system can get questions which are tagged for “exam” purpose and render them without feedback or hints to the student. And another e-learning system can get questions, from the same repository, which are tagged for “practice” purpose and render them with feedback and hints enabled.
## Conceptual Model
The below figure shows an overview of different components and actors involved in the questions creation and delivery.
```
Figure: Overall system view for Questions
```
Question Bank is a repository of QuML questions. Authoring tools are used by authors to create questions which get stored in the question bank. Learning and Assessment systems are responsible for presenting the questions to teachers and/or students depending on the context where the question is being used. Teachers and students interact with questions via QuML players. QuML players are responsible for managing the question sessions.
A question session is the accumulation of all the attempts at a particular instance of a question made by a student. In some types of tests, the same question may be presented to the candidate multiple times e.g. during ‘drill and practice’. Each occurrence or instance of the question is associated with its own question session. The following figure illustrates the user-perceived states of the question session.
```
Figure: The life cycle of a question session
```
Not all states will apply to every scenario, for example feedback may not be provided for a question or it may not be allowed in the context in which the question is being used. Similarly, the student may not be permitted to review their responses and/or view the solution.
As mentioned earlier, a QuML question comprises of multiple components with a well-defined structure. Each component of the question comes into play at different states of a question session. QuML implementations (players) use these components and perform the required action in-between state transitions. The below figure shows the actions performed by QuML players in between states of a question session.
```
Figure: Processes in a question session
```
For example, body along with style sheet data is used to present the question to the student before entering interacting state. User responses are captured and set during interacting state. User’s responses are processed and outcomes are set before moving into finished state. Finally, QuML players generate telemetry events (explained later) that capture details of all user interactions with the question, including the user’s responses to the question.
## Question Data Model
A QuML Question comprises of the information model and associated binding that can be used to store, represent and deliver questions. A question can be defined as a set of interactions (possibly empty) collected together with any supporting material and an optional set of rules for converting the student’s response(s) into outcomes. Question contains the definition for the interactions, variables to store responses, logic to process the responses, outcome data and other components that are needed for question delivery.
QuML specification makes use of web standards like HTML, CSS, JSON, and Javascript to store the question data. The below figure shows the parts that comprise a QuML question:
```
Figure: QuML Question Structural Model
```
### Question Variables
QuML defines multiple types of variables that are useful for different purposes at different states of a question session. The specification defines the places where and how these variables can be declared, initialised, updated and used.
Question variables are declared by variable declarations: response, outcome or template variables. All variables must be declared except for the built-in variables, referred to below, which are declared implicitly and must not be declared. The purpose of the declaration is to associate an identifier with the variable and to identify the runtime type of the variable's value. The variables are also optionally initialised at the time of declaration.
**Response Variables**
Response variables are used to capture the response of student. These variables are generated as a result of an interaction of student with the question. The expected values for response variables are declared in “response declarations” and bound to interactions in the question. Each response variable declared may be bound to one and only one interaction. At runtime, response variables are instantiated as part of a question session.
A response variable with a NULL value indicates that the student has not offered a response, either because they have not attempted the question at all or because they have attempted it and chosen not to provide a response. If a default value has been provided for a response variable then the variable is set to this value at the start of the first attempt. If the student never attempts the question, in other words, the question session passes straight from the “start” state to the “finished” state without going through the interacting state, then the response variable remains NULL and the default value is never used.
Response variables can be used within the question body using the html data attribute: **“data-response-variable”**. This data attribute should be set on HTML elements that are used by candidates for interacting with the question. And when the candidate uses the HTML element for interaction, the corresponding response variable should be updated with the value of the HTML element.
```
```
In the above sample, there is response variable named “response\_01” defined on the HTML checkbox element. When a candidate selects the checkbox, the value of the checkbox element (Oxygen, in this example) is set as the value for the variable “response\_01”.
#### Built-in Response Variables
There are two built-in response variables: 'numAttempts' and 'duration'. These are declared implicitly and should be updated by QuML players when a candidate is interacting with the question.
* **numAttempts**: An integer that records the number of attempts at each question the candidate has taken (if multiple attempts are allowed in the context where the question is being used). The value defaults to 0 initially and then increases by 1 at the start of each attempt.
* **duration**: A float that records the accumulated time (in seconds) of all candidate sessions for all attempts. In other words, the time between the beginning and the end of the question session minus any time the session was in the suspended state.
**Asset Variables**
Asset variables are used to load assets during the rendering of a question. Values for asset variables refer to an asset in the QuML repository server. Asset variables can be used in a question using the html data attribute: **“data-asset-variable”**. This data attribute should be set on HTML elements that are used for rendering assets (e.g. img, audio and video elements). And when the question is being rendered, the location of the asset object represented by the asset variable is set as the source of the HTML element on which the asset variable is declared. All the assets used in a question are declared in Asset Declaration. This declaration is needed to package and pre-download the question assets for offline usage.
```
```
Optionally, a default source can also be provided for the HTML elements. If an asset variable is not declared, the default value of the source will be used by the QuML players.
*QuML implementations should upload assets to the QuML repository server at the time of creation. QuML players should use absolute path of the asset during online usage of the question and in case of offline usage, the asset should be pre-downloaded to the device/system where the question is being used.*
**Template Variables**
Template variables are used in templatized questions, which can be used to produce large number of similar questions. Template variables can be referred to by templateVariable objects in the question body.
Template variables can be referred to by using the html data attribute **“data-template-variable”** in the question body. This data attribute should be set on HTML elements which need to be templatized. And the template variables are resolved and replaced by an absolute value before the question is presented to a student. The rules for assigning values to template variables should be defined in the **“templateProcessing”** section of the question data.
This data attribute should be set on a element around a text that needs to be templatized. And when the question session starts, the text within the gets replaced with the value generated by template processing rule for the corresponding template variable.
```
Shyam has 5apples.
Ram borrowed 3apples from Shyam.
Shyam now has apples.
```
The above question is for subtraction, where the learner is asked to identify the number of apples Shyam has after Ram borrowed 3 of the 5 apples which Shyam had. Instead of showing the same numbers and fruit name always, the numbers and name of the fruit is templatized in the item body using template variables.
**Outcome Variables**
Outcome variables are declared by outcome declarations. Their value is set either from a default given in the declaration itself or during “response processing”. These variables are used for returning the score or for controlling the feedback to the student.
There are six reserved outcome variables:
* Questions that declare a numeric outcome variable representing the student’s overall performance on the question should use the outcome name **‘SCORE’** for the variable. SCORE needs to be a float.
* Questions that provide a feedback to student after the attempt should use the outcome variable **‘FEEDBACK’** to show the appropriate feedback modal. FEEDBACK needs to be a string.
* Questions that have in-built hints to help students in attempting the question should use the outcome variable **‘HINT’**. The value of this variable should be used by the QuML players to render the corresponding hint. HINT needs to be a string.
* Questions that declare a maximum score (in multiple response choice questions, for example) should do so by declaring the **‘MAXSCORE’** variable. MAXSCORE needs to be a float.
* Questions that declare a minimum score (minimum score to mark the student as passed) should do so by declaring the **‘MINSCORE’** variable. MINSCORE needs to be a float.
* Questions or tests that want to make the fact that the student scored above a predefined threshold available as a variable should use the **‘PASSED’** variable. PASSED needs to be a boolean. This variable is automatically set after response processing using the SCORE and MINSCORE outcome variables.
#### Built-in Outcome Variables
There is one built-in outcome variable, **‘completionStatus’**, that is declared implicitly and must not appear in outcome variables declaration. QuML players must maintain the value of the built-in outcome variable completionStatus. It starts with the reserved value “not\_attempted”. At the start of the first attempt, it changes to the reserved value “unknown”. It remains with this value for the duration of the question session unless set to a different value by a setOutcomeValue in responseProcessing. This variable changes to the reserved value “completed” by default if it is not updated during response processing.
There are four permitted values: 1. ‘completed’ - the student has experienced enough of the question to consider it completed; 2. ‘incomplete’ - the student has not experienced enough of the question to consider it completed; 3. ‘not\_attempted’ - the student is considered to have not used the question in any significant way; 4. ‘unknown’ - no assertion on the state of completion can be made.
### Body
Body contains the text, graphics, media objects and interactions that describe the question’s content and information about how it is structured. The body is presented by combining it with stylesheet and/or internationalization information, either explicitly or implicitly using the default style rules of the QuML player.
Body is presented to the student when the associated question session is in the *interacting* state. In this state, the student must be able to interact with each of the visible interactions and therefore set or update the values of the associated response variables.
The body may be presented to the student when the question session is in the *finished* or *review* state. In these states, although the student’s responses should be visible, the interactions or the setting and/or updating the values of the associated response variables must be disabled.
Finally, the body may be presented to the student in the *solution* state, in which case the correct values of the response variables must be visible and the associated interactions and/or response variable updates disabled.
The structural elements of the item body are taken from HTML with the following restrictions:
* It should contian only **structural, media and input HTML elements**. Structural elements include \\, \\, and . Media html elements are img, audio and video elements. Input html elements are text input, text area, select, options, checkbox, radio buttons, file upload and canvas elements.
* It should not contain **element**
* It should not have any **javascript code**, i.e. elements or onXYZ (onClick, etc) for HTML elements should not be used
* It should not have any **javascript import statements**
* It should not have any **css import statements**
In addition to the standard HTML elements and attributes, the specification allows the following in the question body:
* Usage for “data-” attributes to specify the interactions on HTML elements, bind response variables to the interactions, use template and i18n variables.
* Usage of standard style classes defined by the specificaiton. QuML players should provide implementation for the defined classes.
Body:
```
{
“itemBody”: “...”
}
```
### Interactions
A question can contain zero or more interactions with the user. Most of these interactions are typical question types (for example, multiple choice, order elements, fill in the blanks). We can also use interactions for activities like "upload document," "draw a picture," and "start a film".
Interactions can be defined in the item body using HTML data attributes as shown in the below sample (**“data--interaction”**). Each interaction should have an associated response variable that binds the interaction to a **“responseDeclaration”** with the same identifier. This is the link for the result and response processing, which are described later in this document.
```
```
QuML specification defines the following interaction types. QuML players should provide the implementation for all the defined interaction types. The behaviour of the interaction may vary in each implementation but should ensure that the defined specification is supported.
#### Simple Choice Interaction
The simple choice interaction is used when a set of choices are presented to the student. The student’s task is to select one of the choices. This interaction must be bound to a response variable with single cardinality (i.e. set only one value to the associated response variable).
| **Data Attribute** | **data-simple-choice-interaction** |
| ------------------ | -------------------------------------------------------------- |
| cardinality | *single* |
| type | string, integer or float |
| html elements | any HTML element which is clickable and have a value attribute |
#### Multi Choice Interaction
The multi choice interaction is used when a set of choices are presented to the student. The student’s task is to select one or more of the choices. This interaction must be bound to a response variable with multiple cardinality (i.e. set a list of values to the associated response variable). An array variable will be used to store the values of response variable for this interaction.
| **Data Attribute** | **data-multi-choice-interaction** |
| ------------------ | -------------------------------------------------------------- |
| cardinality | *multiple* |
| type | string, integer or float |
| html elements | any HTML element which is clickable and have a value attribute |
#### Text Interaction
A text interaction obtains a piece of text from the student. The QuML player must allow the student to review their choice within the context of the question. The text interaction must be bound to a response variable with single cardinality only. The type must be one of string, integer, or float.
| **Data Attribute** | **data-text-interaction** |
| ------------------ | ---------------------------------------------------------------------- |
| cardinality | *string* |
| type | string, integer or float |
| html elements | HTML elements where user can enter text, i.e. text input, or text area |
#### Order Interaction
In an order interaction, the student’s task is to reorder the choices. When specified, the student must select the choices and impose an ordering on them. If a default value is specified for the response variable associated with an order interaction then its value should be used to override the order of the choices specified here. The order interaction must be bound to a response variable with ordered cardinality (i.e. list of ordered values).
| **Data Attribute** | **data-ordered-interaction** |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------- |
| cardinality | *ordered* |
| type | string, integer or float |
| html elements | element. The values of all elements should be set in the same order as the value for response variable bound with the interaction |
> The order interaction implementation in the player should enable drag and drop of the elements in the element with this interaction. QuML players may choose to provide enhanced graphics/animation for order interaction.
#### Match Interaction
A match interaction presents candidates with two sets of choices and allows them to create associates between pairs of choices in the two sets, but not between pairs of choices in the same set. The match interaction must be bound to a response variable with type map and single cardinality.
| **Data Attribute** | **data-match-interaction** |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| cardinality | *single* |
| type | map |
| html elements | Two elements. Both the elements should be bound to the same response variable. The list of values of element pairs (mapped with each other) should be set as the value of the response variable. |
> Similar to order interaction, the match interaction implementation in the player should enable drag and drop of elements in one element on to elements in another element with this interaction. QuML players may choose to provide enhanced graphics/animation for match interaction.
#### Upload Interaction
The upload interaction allows the candidate to upload a pre-prepared file representing their response. It must be bound to a response variable with type file and single cardinality.
| **Data Attribute** | **data-upload-interaction** |
| ------------------ | --------------------------- |
| cardinality | *single* |
| type | uri |
| html elements | HTML file upload element |
> The upload interaction implementation in the player should upload the input file to a configured server and set the uploaded file url as the value for the response variable.
#### Map Interaction
The map interaction is a graphic interaction. The candidate’s task is to select one or more points on a canvas. The associated response may have a mapping that scores the response on the basis of comparing it against predefined areas but QuML players must not indicate these areas of the image. Only the actual point(s) selected by the candidate shall be indicated. The map interaction must be bound to a response variable with type points and multiple cardinality.
| **Data Attribute** | **data-map-interaction** |
| ------------------ | ------------------------ |
| cardinality | *multiple* |
| type | points |
| html elements | HTML canvas element |
#### Custom Interaction
The custom interaction provides an opportunity for extensibility of this specification to include support for interactions not currently documented. QuML implementations can implement a set of custom interactions that go beyond the standard interactions described above. Authors wishing to write questions for those implementations can use these custom interactions. However these custom templates should be published to ensure that these questions can be used with QuML players of other implementations. Otherwise, such questions will be limited to use only by that implementation.
### CSS classes
QuML recommends the use of Cascading Style Sheets (CSS) for controlling the content formatting. QuML has a defined set of standard classes for various different elements and interactions. A question can use those classes for formatting and QuML players should provide implementation for these classes.
* *1-col-layout*: style to create a one column layout
* *2-col-layout*: style to create a two column layout
* *3-col-layout*: style to create a three column layout
* *4-col-layout*: style to create a four column layout
* *6-col-layout*: style to create a six column layout
* *8-col-layout*: style to create a eight column layout
* *row*: style to add a row in the question
* *title*: for formatting title texts of the quesiton
* *sub-title*: for formatting sub-headings in the question
* *paragraph*: for formatting paragraph texts
* *small-image*: style to display small images
* *medium-image*: style to display medium size images
* *large-image*: style to display large images
* *horizontal-options*: style to layout options horizontally (can be used for choice, match and order interactions)
* *vertical-options*: style to layout options vertically (can be used for choice, match and order interactions)
* *file-upload*: styling for file upload elements (can be used for upload interaction)
* *canvas*: styling for canvas elements (can be used for map interaction)
Alternatively when there is a need for formatting that is not provided by the defined classes, a question can also have custom css. The support of the CSS elements used for formatting is totally dependent on the browser used to render the question. Hence it is advised to use only CSS elements that are supported by all the commonly used browsers and browser versions.
### Instructions
Some questions will have instructions on how to understand, attempt or how the question will be evaluated. Such instructions are defined in HTML format and stored in the instructions part of the question data.
Instructions:
```
{
“instructions”: “... … … ”
}
```
### i18n Data
Internationalization is supported by QuML to render texts inside body (answers, feedback and hints) in different locales. To enable this, QuML allows storage of content in multiple locales.
For example, **body** in different locales should be stored in body part of the question data as below:
body:
```
{
“body”: {
“”: "
...
",
“”: "
...
"
…
}
}
```
In cases when internationalization is not required, the value of body (or of answers, instructions, feedback and hints) can be defined without the locale information.
### Response Declaration
A **“responseDeclaration”** contains information about the answer (the response) to a question: When is it correct, and (optionally) how is it scored?
Response Declaration should have declaration for every response variable in the question body. Optionally, the declaration can have the following details:
* *Correct Response:* Correct (or optimal) values for the response variable
* *Mapping to Score:* Map different values to score so that a response can have more nuances than plain right or wrong, e.g.: a multiple choice question with more than one correct answer can support partial scoring
Response declaration is a JSON object in key-value format. The keys in the JSON are the response variables defined in the body and values are of type ResponseVariableDef.
ResponseDeclaration:
```
{
“responseDeclaration”: {
“”: ResponseVariableDef Object,
“”: ResponseVariableDef Object,
…
}
}
```
*ResponseVariableDef:*
Each response variable should have exactly one response variable definition in the response declaration.
| Attribute | Schema | Description |
| --------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| cardinality | dataType: string, required: true, range: “single”, “multiple”, “ordered” | Used to specify whether the response variable will have a single value, multiple values or an ordered list of values. |
| type | dataType: string, required: true, range: “string”, “integer”, “float”, “boolean”, “map”, “uri”, “points”, “coordinate” | |
| correctResponse | dataType: CorrectResponseDef, required: false | |
| mapping | dataType: MappingDef, required: false | Map different answers to scores so that an answer can have more nuances than plain right or wrong |
| areaMapping | dataType: AreaMappingDef, required: false | Similar to mapping attribute, but for response variables associated to map interactions |
*CorrectResponseDef:*
| Attribute | Schema | Description |
| --------- | ----------------------------- | ----------- |
| value | dataType: any, required: true | |
*MappingDef:*
| Attribute | Schema | Description |
| ------------- | ------------------------------------------------------- | ----------- |
| key | dataType: any, required: true | |
| value | dataType: float, required: true | |
| caseSensitive | dataType: boolean, required: false, defaultValue: false | |
*AreaMappingDef:*
| Attribute | Schema | Description |
| --------- | -------------------------------- | ----------- |
| key | dataType: points, required: true | |
| value | dataType: float, required: true | |
### Asset Declaration
A **“assetDeclaration”** contains information about the assets used in the question. Asset Declaration should have declaration for every asset used in the question across body, feedback, instructions, hints and answers.
Asset declaration is a JSON object in key-value format. The keys in the JSON are the asset variables defined in the question and values are of type [*Asset*](/v1/common#asset).
AssetDeclaration:
```
{
“assetDeclaration”: {
“”: Asset Object,
“”: Asset Object,
…
}
}
```
### Outcome Declaration
An “outcomeDeclaration” contains information about the outcome variables of the question, i.e the values that are output of a question session. QuML players should make the outcome variables available to the context in which the question is being used. For example, a test session uses the values of outcome variables of constituent questions to process the outcome of the test.
Outcome declaration is a JSON object in key-value format. The keys in the JSON are the outcome variables and values are of type OutcomeVariableDef.
OutcomeDeclaration:
```
{
“outcomeDeclaration”: {
“”: OutcomeVariableDef Object,
“”: OutcomeVariableDef Object,
…
}
}
```
*OutcomeVariableDef:*
Each outcome variable should have exactly one outcome variable definition in the outcome declaration.
| Attribute | Schema | Description |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| cardinality | dataType: string, required: true, range: “single”, “multiple”, “ordered” | Used to specify whether the outcome variable will have a single value, multiple values or an ordered list of values. |
| type | dataType: string, required: true, range: “string”, “integer”, “float”, “boolean”, “map”, “uri”, “points”, “coordinate” | |
| defaultValue | dataType: any, required: false | |
| range | dataType: List of any, required: false | |
### Response Processing
Response processing is the process by which QuML players assign outcomes based on the student’s responses. The state representation for the response processing is shown in the following figure. The outcomes may also be used to provide feedback to the student. Feedback is either provided immediately following the end of the student’s attempt or it is provided at some later time, perhaps as part of a summary report on the question session.
The end of an attempt, and therefore response processing, must only take place in direct response to a user action or in response to some expected event, such as the end of a test. A question session that enters the suspended state may have values for the response variables that have yet to be submitted for response processing.
```
Figure - The state diagram for response processing
```
There are two ways of configuring response processing: 1. Question authors can provide custom response processing logic as javascript code for their questions. This logic can use the available library methods to get/set question variables. 2. Question authors can use/configure one of the provided response processing templates to process the response of the question.
#### Custom Response Processing
The custom response processing logic using javascript should be defined as part of the “eval” attribute of “responseProcessing” data.
Sample Custom Response Processing:
```
{
“responseProcessing”: {
“eval”: “
mapResponse();
var score = getOutcomeVariable(\“SCORE\”);
If (score >= 1.0) {
setOutcomeVariable(\“FEEDBACK\”, “feedback_01”);
} else if (score > 0 && score < 1.0) {
setOutcomeVariable(\“FEEDBACK\”, “feedback_02”);
} else if (score <= 0) {
setOutcomeVariable(\“FEEDBACK\”, “feedback_03”);
}
“
}
}
```
> custom “eval” data is mandatory if response processing templates are not used for the question. If both template and evaluation logic are present, template is executed first and then the custom evaluation logic is executed.
#### Response Processing Templates
Custom response processing involves the application of a set of response rules, including the testing of response conditions and the evaluation of logic involving the question variables. For QuML implementations that are only designed to support very simple use cases, the implementation of a system for carrying out this evaluation, conditional testing and processing may pose a barrier to the adoption of the specification.
To alleviate this problem, the implementation of generalized response processing is an optional feature. Players that do not support it can instead implement a smaller number of standard response processors called response processing templates. These templates are also defined using javascript for evaluation of logic and should be implemented by the QuML implementation.
Following are two standard response processing templates defined in QuML (an additional template for response processing using templates is defined in the Template Processing section):
**MATCH\_CORRECT**
This response processing template matches the value of response variables with their correct value using **correctAnswer** attribute defined in **responseDeclaration**. It sets the outcome variable SCORE to either 0 or 1 depending on the outcome of the test. To use this template, all response variables must have been declared and have an associated correct value. Similarly, the outcome variable SCORE must also have been declared.
**MAP\_RESPONSE**
The map response processing template uses the **mapping** and **areaMapping** definitions of response variables in **responseDefinition** to set value for the outcome SCORE. To use this template, all response variables must have been declared and must have an associated mapping. Similarly, the outcome variable SCORE must also have been declared.
The map response processing template optionally takes a **mapping config** as input. This config is to set other outcome variables based on the value of SCORE outcome variable. If no config is provided, these templates set the value of SCORE outcome only. The structure of mapping config is as shown below.
*MappingConfig:*
Mapping config should be provided in JSON format. It comprises of the list of outcome variables and their values for various possible scores.
```
{
“mappingConfig”: [
{
“SCORE”: {
“”: ,
“”: , ...
“regex”:
},
“outcomeVariables”: {
“”:
}
}
]
}
```
Following is the sample usage of response processing templates. The sample javascript response processing provided above (in the Custom Response Processing section) can also be done using MAP\_RESPONSE template:
```
{
“responseProcessing”: {
“template”: “MAP_RESPONSE”,
“mappingConfig”: [
{
“SCORE”: {“ge”: 1.0},
“outcomeVariables”: {“FEEDBACK”: “feedback_01”}
},
{
“SCORE”: {“gt”: 0, “lt”: 1},
“outcomeVariables”: {“FEEDBACK”: “feedback_02”}
},
{
“SCORE”: {“le”: 1.0},
“outcomeVariables”: {“FEEDBACK”: “feedback_03”}
}
]
}
}
```
> QuML implementations that do not support generalized response processing (custom evaluation) but do support response processing mechanisms that go beyond the standard templates described above can define templates of their own. Authors wishing to write questions for those implementations can then refer to these custom templates. Publishing these custom templates will then ensure that these questions can be used with QuML players that do support generalized response processing.
### Feedback
Feedback is shown to the students following response processing. The value of outcome variable “FEEDBACK” is used to determine whether or not the feedback is shown. Feedback is shown only if it is allowed in the context where the question is being used.
Feedback is a JSON object in key-value format. The keys in the JSON are the identifiers of different feedbacks for the question and values are HTML snippet to be shown to the student. After the response processing, the QuML player renders the feedback HTML mapped to the value that is set to the FEEDBACK outcome variable.
Feedback:
```
{
“feedback”: {
“”: “
...
”,
“”: “
...
”,
…
}
}
```
To support internationalization, feedback in multiple locales should be provided (for each value of feedback) as shown below:
```
{
“feedback”: {
“”: {
"locale_1": “
...
”,
"locale_2": “
...
”,
...
}
“”: {
"locale_1": “
...
”,
"locale_2": “
...
”,
...
},
…
}
}
```
### Hints
Hints are shown to the candidates after response processing or when the student requests for hints.
* **HINT** outcome variable value is used to render the hint after response processing.
* Alternatively, QuML players can provide an option for students to request for hints. Upon request, the QuML player renders all the configured hints in a sequence (as defined in the hints configuration).
In either case, hints are shown only if they are allowed in the context where the question is being used. Hints is a JSON object in key-value format. The keys in the JSON are the identifiers of different hints for the question and values are HTML snippet for hints.
Hints is a JSON object in key-value format. The keys in the JSON are the identifiers of different hints for the question and values are HTML snippet for hints.
Hints:
```
{
“hints”: {
“”: “...”,
“”: “...”
…
}
}
```
> Similar to feedback, internationalization is supported for hints also.
### Answers
Providing exemplar answers for questions aid candidates in-depth learning and enhance candidate’s understanding of the concepts. These answers are mainly helpful for candidates in preparing for exams. Multiple answers can be configured for a question and each answer can have one or more parts, each part should be stored in HTML format. Similar to feedback & hints, answers in multiple locales can be specified.
Answers:
```
{
“answers”: [
{
"parts": [{
"body": “
...
”,
"comment": "optional comment for the answer part"
},
{ ... }
]
“comment”: “optional comment for the answer”
},
{ ... }
]
}
```
Answers for QuML questions are shown to the students in *solution* state of question session, and if the context in which the question is being used allows the students to view the solution.
### Template Declaration
Question templates are templates that can be used for producing large numbers of similar questions. Such questions are often called cloned questions. Question templates can be used to produce a dynamically chosen clone at the start of a question session.
Each question cloned from a template is identical except for the values given to a set of template variables. A question is therefore a question template if it contains **templateDeclaration** and a set of **templateProcessing** rules for assigning values for template variables. The value of the template variable is used to create an appropriate run of text that is displayed. When the question session starts, the template variable gets replaced with the value generated by template processing rule for the corresponding template variable.
Template declarations declare variables that are to be used specifically for the purposes of cloning items. They can have their value set only during **templateProcessing**. They are referred to within the question body in order to individualize the clone and possibly also within the **responseProcessing** rules if the cloning process affects the way the question is scored.
Template declaration is a JSON object in key-value format. The keys in the JSON are the template variables used in the question and values are of type TemplateVariableDef.
TemplateDeclaration:
```
{
“templateDeclaration”: {
“”: TemplateVariableDef Object,
“”: TemplateVariableDef Object,
…
}
}
```
*TemplateVariableDef:*
Each template variable should have exactly one template variable definition in the template declaration.
| Attribute | Schema | Description |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| cardinality | dataType: string, required: true, range: “single”, “multiple”, “ordered” | Used to specify whether the template variable will have a single value, multiple values or an ordered list of values. |
| type | dataType: string, required: true, range: “string”, “integer”, “float”, “boolean”, “map”, “uri”, “points”, “coordinate” | |
| defaultValue | dataType: any, required: false | |
A question can have template variable that are not used in the question “body”. There will be cases when some template variables are needed for template and/or response processing but not visible for the candidate. Below is a sample template declaration. In addition to the template variables used in the body, there is an additional template variable “template\_var\_temp\_number” defined. This additional template variable can be used in both template and response processing.
```
{
“templateDeclaration”: {
“template_var_fruit_number_1”: {
“cardinality”: “single”, “type”: “integer”, “defaultValue”: 5
},
“template_var_fruit_number_2”: {
“cardinality”: “single”, “type”: “integer”, “defaultValue”: 3
},
“template_var_fruit_name”: {
“cardinality”: “single”, “type”: “string”, “defaultValue”: “apples”
},
“template_var_temp_number”: {
“cardinality”: “single”, “type”: “integer”, “defaultValue”: 2
}
}
}
```
### Template Processing
Template processing consists of one or more templateRules that are followed by the QuML players in order to assign values to the template variables. Template processing is identical in form to responseProcessing except that the purpose is to assign values to template variables, not outcome variables, at the start of a question session.
```
Figure - The state diagram for template processing
```
Template processing is a JSON object in key-value format. The keys in the JSON are the template variables used in the question and values are of type TemplateProcessingDef.
TemplateProcessing:
```
{
“templateProcessing”: {
“”: List of TemplateProcessingDef Objects (one per locale),
“”: List of TemplateProcessingDef Objects,
…
}
}
```
*TemplateProcessingDef:*
Each template variable should have the template processing rule defined using TemplateProcessingDef object.
| Attribute | Schema | Description |
| --------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| random | dataType: RandomVariableDef, required: false | Used to generate a random value for the template variable |
| regex | dataType: string, required: false | Used to provide a regular expression for generating value for the template variable |
| eval | dataType: string, required: false | Used to provide custom logic (in javascript) for generating value for the template variable |
| locale | dataType: string, required: false, defaultValue: en | Internationalization is supported in template processing. Template processing specific to locale can be defined by setting the locale value. |
> For each template variable, at least one and only one of “random”, “regex” and “eval” should be defined as the processing rule.
*RandomVariableDef:*
| Attribute | Schema | Description |
| --------- | ------------------------------------------ | ---------------------------------------------------- |
| list | dataType: list of any, required: false | Used to pick a value randomly from the provided list |
| number | dataType: RandomNumberDef, required: false | Used to generate a random number (integer or float) |
*RandomNumberDef:*
| Attribute | Schema | Description |
| --------- | ------------------------------------------------------- | ----------- |
| type | dataType: string, required: true, range: integer, float | |
| min | dataType: integer/float, required: true | |
| max | dataType: integer/float, required: true | |
| step | dataType: integer/float, required: false | |
Below is the sample template processing configuration for the template variables referred to in the previous section.
```
{
“templateProcessing”: {
“template_var_fruit_number_1”: [{
“random”: {
“type”: “integer”, “min”: 3, “min”: 9
}
}],
“template_var_temp_number”: [{
“random”: {
“type”: “integer”, “min”: 3, “min”: 6
}
}],
“template_var_fruit_number_2”: [{
“eval”: “
if (getTemplateVariable(‘template_var_temp_number’) > getTemplateVariable(template_var_fruit_number_1)) {
return getTemplateVariable(template_var_fruit_number_1) - 3;
} else {
return getTemplateVariable(template_var_fruit_number_1) - getTemplateVariable(‘template_var_temp_number’);
}
”
}],
“template_var_fruit_name”: [{
“random”: {
“list”: [“apples”, “mangoes”, “bananas”, “oranges”, “pineapples”]
},
“locale”: “en”
}, {
“random”: {
“list”: [“सेब”, “आम”, “केले”, “संतरे”, “अनानास”]
},
“locale”: “hi”
}]
}
}
```
#### Using Template Variables in Response Processing
Template variables are also used in response processing in cases when these variables affect the outcome of the question. When the response processing has custom logic , the template variables can be accessed using the library method *getTemplateVariable*. Alternatively, the pre-defined response processing template **“MATCH\_TEMPLATE”** can also be used to process the response using template variables.
**MATCH\_TEMPLATE**
This response processing template matches the value of response variables with specified template variables and sets the outcome variable SCORE.
The map response processing template takes a match template config as input. This config is to set SCORE outcome variable based on the value of a template variable. Optionally, a mapping config (similar to the one provided for MAP\_RESPONSE template) can also be provided to set other outcome variables.
*MatchTemplateConfig:*
Template variable config should be provided in JSON format. It comprises of the list of outcome variables and their values for various possible scores.
```
{
“matchTemplateConfig”: [
{
“mapping”: {
: [
{
“operator”: ,
“templateVariables”: [, ]
},
{ … }
],
: [{ … }]
}
“SCORE”: 1.0
}
]
}
```
Following is the sample usage of MATCH\_TEMPLATE response processing template:
```
{
“responseProcessing”: {
“template”: “MATCH_TEMPLATE”,
“matchTemplateConfig”: [
{
“mapping”: {
“response_01”: [{
“eq”: “template_var_temp_number”
}]
},
“SCORE”: 1.0
}
]
}
}
```
In the above sample, the SCORE is set to 1.0 when the value of response variable “RESPONSE” is equal to the value of the template variable “template\_var\_temp\_number”.
## Question Metadata
As per definition, *metadata is a set of data that describes and gives information about other data*, the other data is question in this case. Question metadata should capture enough detail that is needed for discovering, delivering and composing questions into tests. QuML classifies question metadata into four categories:
### Learning Metadata
This category of metadata contains pedagogic information of the question - how difficult it is, what level of skill is the question assessing, where is it intended to be used, etc.
* *bloomsTaxonomyLevel*: the cognitive processes involved to answer the question - remember, understand, apply, analyse, evaluate, create.
* *difficultyLevel*: difficulty level of the question for a average learner - easy, medium, hard.
* *purpose*: the purpose served by the question - recall, explore, sense, assess, teach, revise.
* *expectedDuration*: expected time for one attempt of the question.
* *maxScore*: maximum score that can be awarded for the question.
### Technical Metadata
This category of metadata contains information that can be used by delivery engines to deliver the question. Some of the metadata fields of this category can be automatically derived and set by the creation tools, e.g.: mime type.
* *mimeType*: mime type of the question - HTML is the only supported mime type in this version of the specification.
* *version*: version of the QuML specification using which the question is created.
* *questionType*: one of the standard question types - mcq, mtf, ftb, mmcq, essay, short answers, programming, other. this can be auto-derived at times based on the interactions used in the question.
* *visibility*: if the question is visible for all or only for those who created it and/or for some specific systems or use cases - private, public.
* *isTemplate*: set to true if question data has template variables and template processing, else it is set to false.
* *interactions*: list of interactions present in the question.
* *solutionAvailable*: true, if question data has answers, else, set to false
* *scoringMode*: one of the values: responseProcessing (if question has inbuild response processing), offline (if scoring will be done offline and/or manually) or external (if an external system does the evaluation and submit the score).
### Curricular Metadata
This category of metadata captures the pedagogic intent of the question - which concepts are tested by the question, etc. The metadata fields in this category vary for each domain. For example, questions of K-12 domain may have the following curricular metadata fields: *board, grade, subject, medium and topics*.
### Usage Metadata
This category of metadata captures the statistical data about the usage of a question. These metadata fields are not provided by users but are set by analytics systems based on the usage telemetry generated by QuML players.
* *totalTimeSpent*: total cumulative time spent, in milliseconds, on the question by all users.
* *avgTimeSpent*: average time spent per attempt, in milliseconds.
* *numAttempts*: total number of attempts.
* *numCorrectAttempts*: number of attempts where the user response is correct.
* *numInCorrectAttempts*: number of attempts where the user response is incorrect.
* *numSkips*: total number of attempts where the user did not give a response.
* *avgRating*: average rating of the question.
* *totalRatings*: total number of ratings given for the question.
# samples
# mcq\_sample\_1
This sample demonstrates how QuML is used to model a simple multiple choice question (with multiple answers), configure the interactions & process the responses to produce results.
This question is a multiple choice question where the correct answer has two values - “Hydrogen” and “Oxygen”. Below is the logic for evaluating and processing the question:
* If student selects both the correct answers, the score should be 1.0
* If student selects only one of the correct answers, the score should be 0.5
* If student selects any of the wrong answers, a value of 0.5 should be removed from the total score
* If total score if 1.0, show the message “Well done!!!”
* If total score is between 0 and 1.0 (not including 0 and 1.0, i.e. 0 < score < 1.0), show the message “Better luck next time!!!”
* If total score is less than or equal to 0, show the message “You need to work harder!!!”
**Body**
```
Composition of water
Which of the following elements are used to form water?
”
}
}
```
# README
This folder contains the images used in this specification.
# QuML for Tests
Questions from the bank are pulled together to form a test. Selecting the questions requires a deep understanding of the subject and teaching process. For example, which concepts to cover in the test, how many questions should be picked, in what order? All of these have an impact on the design of a test. It is therefore a task that is generally done by the subject matter experts.
## Conceptual Model
At the simplest level, a test is a collection of pre-selected questions, carefully arranged in an order. The issue with such tests is that they are static, and multiple attempts become rote.
> ## In reality, tests need to be a lot more dynamic so that new questions, in random order are presented to the student. Even if the same student attempts the test multiple times, he/she is presented with a different set of questions during each attempt.
In addition to explicitly adding questions to a test, the test can also be created using a dynamic criteria/query - so that all questions in the question bank that match that criteria become candidates for inclusion in the test. Each such criteria is nothing but a set of questions as defined in the underlying question bank.
A test therefore is a group of such question sets and questions with an associated set of rules that determine which of the questions the student sees, in what order, and in what way the student interacts with them. The rules describe the valid paths through the test, when responses are submitted for response processing and when (if at all) feedback is to be given.
```
Figure - Test/Question Set and Questions
```
> “In the example shown above, the test has 5 questions that match the criteria defined in the first set. Let’s say the question bank has 20 questions that are “easy” difficulty level, and related to the concept of “Inertia”. Therefore, at runtime, when a test is being constructed, ANY 5 from the available 20 can be selected.”
Modeling the tests (Question Sets) like this maintains a Balance of the questions in the test, ensure that even though questions are picked at random for different students, they all still get questions that are similar but not necessarily identical (in this case they all get Easy questions) - ensuring fairness of the assessment.
A test (represented using question set in QuML) must contain at least one other question set or question. Question sets enable the following capabilities:
### ➢ Divide test into parts that could be undertaken in separate test sessions or a single test session
### ➢ Establish groups of questions that have some common pedagogic testing objective
### ➢ Collect together questions which will then be presented to the student. The order of presentation can be controlled using selection and ordering algorithms
The below figure shows an overview of different components and actors involved in tests creation and delivery.
```
Figure: Overall system view for Tests
```
Similar to questions, teachers and students interact with question sets via QuML players. QuML players are responsible for managing test sessions.
For each test session, question sets are selected and arranged into order according to rules defined in the containing question set. This process of selection and ordering defines a basic structure for each part of the test on a per-session basis. The paths that a student may take through this structure are then controlled by the mode settings for the question set and possibly by further preConditions or branchRules evaluated during the test session itself.
```
Figure - Structure of the test with question sets and questions
```
The below figure illustrates a specific instance of the same question set after the application of selection and ordering rules. A rule in question set S 01 selects just one of S 01 A and S 01 B, a rule in S 02 shuffles the order of the items contained by it and, finally, rules in S 03 select 1 out of the 2 items it contains and shuffles the result.
```
Figure - Delivered test after selection and ordering
```
### Navigation mode
This specification defines a way in which the overall behaviour of a question set can be controlled: the **navigation mode**. The navigation mode determines the general paths that the student may take.
A question set in linear mode restricts the student to attempt each question in turn. Once the student moves on they are not permitted to return. A question set in nonlinear mode removes this restriction - the student is free to navigate to any question in the question set at any time. Navigation mode is applicable only to question sets that have questions as members and not other question sets. QuML players are free to implement their own user interface elements to facilitate navigation provided they honour the navigation mode currently in effect.
### Pre-conditions and Branching
These are advanced concepts used to introduce an element of adaptivity into the specification of a test. Pre-conditions enable some question sets to be skipped depending on the outcomes of some of the question sets presented earlier in the test.
A branch rule is a simple expression attached to a question set that is evaluated after the question set has been presented to the student. If the expression evaluates to true the test jumps forward to the question set referred to by the target identifier or exit the test.
An implication with branch rules is that this configuration might result in cyclic loops some times. Though this can be used in drill and practice use cases, QuML players should not allow unbounded repetition. The player should exit the test after a certain number of repetitions.
### Time Limits
In the context of a test, a question or a question set, may be subject to a time constraint. This specification supports both minimum and maximum time constraints. The controlled time for a single question is simply the duration of the question session as defined by the built-in response variable *duration*. For question sets, the time limits relate to the duration of all the question sessions plus any other time spent navigating the question set. In other words, the time includes time spent in states where no question is being interacted with, such as dedicated navigation screens.
QuML players are required to track and report the time spent on each question set when time limits are in force. If no time limit is in force for a question set, then the time spent may be tracked and reported but it is not required.
The time spent on the question set is recorded using a built-in response variable called *duration*. The values of these durations can be referred to during **outcomeProcessing** by using the variable name duration.
## Question Set Data Model
This section describes the data model of a question set. Question set data is used by QuML players while rendering a question set. In case when the question set is added to another question set, then the settings of the containing question set will override the configurations of the contained question set.
### Instructions
Similar to questions, question sets can also have instructions. Such instructions are defined in HTML format and stored in the instructions part of the question set data (similar to question instructions).
Instructions:
```
{
“instructions”: “
...
”
}
```
### Feedback
The feedback configuration for question sets is same as that of questions. Multiple feedbacks can be configured and each feedback should be defined in HTML format.
Feedback:
```
{
“feedback”: {
“”: “
...
”,
“”: “
...
”,
…
}
}
```
### Hints
The hints configuration for question sets is same as that of questions. Multiple hints can be configured and each hint should be defined in HTML format.
Hints:
```
{
“hints”: {
“”: “
...
”,
“”: “
...
”,
…
}
}
```
### i18n data
Texts in a question set (instructions, feedback, hints) can also be rendered in multiple locales. QuML allows instructions, feedback and hints to be defined in multiple locales (in the same way as defined for questions).
### Navigation mode
The navigation mode determines the general paths that the student may take during the test session. It is an enumeration with two possible values: “linear” and “non-linear”.
Navigation Mode:
```
{
“navigationMode”: “linear | non-linear”
}
```
* A question set in linear mode restricts the student to attempt each question in turn. Once the student moves on they are not permitted to return.
* A question set in nonlinear mode removes this restriction - the student is free to navigate to any question in the question set at any time.
### Time limits
The time limits (if any) for a question set can be defined as **timeLimits** data. Both minimum and maximum time constraints can be provided for the complete set and/or for one question as well. This configuration is used by QuML players to impose time limits (both minimum and maximum) for attempting a question set.
Time Limits:
```
{
“timeLimits”: {
“questionSet”: { // time limits for the question set and for any member sets
“min”: ,
“max”:
},
“question”: { // time limits for the questions in the question set
“min”: ,
“max”:
}
}
}
```
### Show Hints
This configuration is used by QuML players to enable/disable hints for the student while using the question set. It should be set to either true or false based on the context in which the question set is being used.
Show Hints:
```
{
“showHints”: “true | false”
}
```
### Questions
If a question set contains questions, the association should be defined in the “questions” section. This section is mutually exclusive with “questionSets” section, i.e. a question set contain only either question sets or questions but not both.
Questions:
```
{
“questions”: {QuestionDef Object}
}
```
*QuestionDef*
| Attribute | Schema | Description |
| -------------- | ------------------------------------------------------- | ---------------------------------------------------------------- |
| shuffle | dataType: boolean, required: false, defaultValue: false | |
| totalQuestions | dataType: integer, required: true | |
| maxQuestions | dataType: integer, required: true | |
| list | dataType: list of string, required: true | List of question identifiers that are added to the question set. |
### Question Sets
A question set can contain either other question sets or individual questions. If a question set contains other question sets, the association should be specified in the “questionSets” section.
QuestionSets:
```
{
“questionSets”: [{QuestionSetDef Object}, {QuestionSetDef Object}, … ]
}
```
*QuestionSetDef*
| Attribute | Schema | Description |
| -------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| questionSetId | dataType: string, required: true | identifier of the member question set |
| shuffle | dataType: boolean, required: false, defaultValue: false | if questions in the member question set should be shuffled or not when presented to the student |
| totalQuestions | dataType: integer, required: true | total number of questions in the member question set. applicable only if the member question set has questions |
| maxQuestions | dataType: integer, required: true | number of questions in the member question set that should be used in one session. applicable only if the member question set has questions |
| preConditions | dataType: list of PreConditionDef objects, required: false | conditions to be validated before rendering the question set. generally depends on the outcomes of question sets presented earlier in the session |
| branchRules | dataType: list of BranchRuleDef objects, required: false | Evaluated after question set is complete and jumps forward to the specified target. |
*PreConditionDef*
| Attribute | Schema | Description |
| ------------- | --------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| questionSetId | dataType: string, required: true | Identifier of the question set whose outcome will be used to do the precondition check |
| match | dataType: list of OutcomeMatchDef objects, required: true | |
*OutcomeMatchDef*
| Attribute | Schema | Description |
| --------------- | ---------------------------------------------------------------- | ----------- |
| outcomeVariable | dataType: string, required: true | |
| operator | dataType: boolean, required: true, range: lt, le, eq, gt, ge, in | |
| value | dataType: any, required: true | |
*BranchRuleDef*
| Attribute | Schema | Description |
| --------- | --------------------------------------------------------- | ----------- |
| target | dataType: TargetSetDef, required: true | |
| match | dataType: list of OutcomeMatchDef objects, required: true | |
*TargetSetDef*
Target of a branch rule can be either another question set or exit of the question set. One (and only one) of these two must configured as a target for a branch rule.
| Attribute | Schema | Description |
| ------------- | ------------------------------------------------------- | ----------- |
| questionSetId | dataType: string, required: false | |
| exit | dataType: boolean, required: false, defaultValue: false | |
### Outcome Declaration
Question Sets also have outcome variables similar to Question outcome variables. They are also declared by outcome declarations. The same set of reserved and built-in variables defined for questions are applicable to question sets also. The only difference is that their value is set during outcome processing (in contrast to response processing of questions).
Outcome declaration is a JSON object in key-value format (same as for questions). The keys in the JSON are the outcome variables and values are of type OutcomeVariableDef.
OutcomeDeclaration:
```
{
“outcomeDeclaration”: {
“”: OutcomeVariableDef Object,
“”: OutcomeVariableDef Object,
…
}
}
```
*OutcomeVariableDef*
| Attribute | Schema | Description |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| cardinality | dataType: string, required: true, range: “single”, “multiple”, “ordered” | Used to specify whether the outcome variable will have a single value, multiple values or an ordered list of values. |
| type | dataType: string, required: true, range: “string”, “integer”, “float”, “boolean”, “map”, “uri”, “points”, “coordinate” | |
| defaultValue | dataType: any, required: false | |
| range | dataType: list of any, required: false | |
### Outcome Processing
Outcome processing takes place each time the student submits the responses for a question. It happens after any (question level) response processing triggered by the submission. Because outcome processing occurs each time the student submits responses, the resulting values of the question set outcomes may be used to activate question set level feedback during the test or to control the behaviour of subsequent parts through the use of preConditions and branchRules.
Similar to response processing of questions, outcome processing rules of a question set can be defined using custom evaluation logic or use one of the existing outcome processing templates.
#### Custom Outcome Processing
The custom outcome processing logic using javascript should be defined as part of the “eval” attribute of “outcomeProcessing” data.
```
{
“outcomeProcessing”: {
“eval”: ““
}
}
```
> “eval” data is mandatory if outcome processing templates are not used for the question set. If both template and evaluation logic are present, template is executed first and then the custom evaluation logic is executed.
#### Outcome Processing Templates
1. **SUM\_OF\_SCORES**: This outcome processing template adds the value of SCORE outcome variables of questions or question sets that are part of the question set and sets the computed value as the SCORE outcome of the question set.
2. **AVG\_OF\_SCORES**: This outcome processing template computes the average of SCORE outcome values of questions or question sets that are part of the question set and sets the computed value as the SCORE outcome of the question set.
3. **WEIGHTED\_AVG\_OF\_SCORES**: This template is similar to the AVG\_OF\_SCORES templates with an additional configuration to specify weightage for each question or question set. The weightage configuration can be provided using an additional parameter **weightageConfig** in outcome processing.
Outcome processing schema for using outcome processing templates:
| Attribute | Schema | Description |
| ---------------- | ----------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| template | dataType: string, required: false, range: SUM\_OF\_SCORES, AVG\_OF\_SCORES, WEIGHTED\_AVG\_OF\_SCORES | name of the outcome processing template to be used for the question. “template” is mandatory if outcome processing templates are used for the question |
| ignoreNullValues | dataType: string, required: false, defaultValue: false | If set to true, the processing will ignore any SCORE outcomes that are null. This is relevant while using average based templates |
| weightageConfig | dataType: map, required: false | configuration to set weightages for questions or question sets when WEIGHTED\_AVG\_OF\_SCORES template is used. If not provided, same weightage is given to all |
| mappingConfig | dataType: string, required: false | configuration to set additional outcome variables (other than SCORE). Same as the mappingConfig defined in question responseProcessing specification |
## Question Set Metadata
Similar to question metadata, question set metadata should also capture enough detail that is needed for discovering, delivering and composing question sets into tests.
### questionSetType
A question set can be comprised of a materialized list of questions, or can also be dynamically built at runtime by using a criteria to select member questions. A set can be either - *materialized* or *dynamic*.
### criteria
Criteria to be used when the set type is dynamic . Criteria should be provided in JSON map format where key should be a metadata field of question and value should be either a single value or list of values.
```
{
“”: ,
“”: [ , , ... ],
...
}
```
### usedFor
A question set can be used either for formative or summative assessments. A question set tagged for usage in summative assessment should not be used in formative assessment. Allowed values for this field are - *practice* (formative assessments) or *exam* (summative assessments).
### purpose
This metadata field should be used to tag the purpose served by the question set - *recall, explore, sense, assess, teach, revise*.
### visibility
Some question sets can be made available only for those who created it and/or for some specific systems or use cases. Also, a question set could be created to be used only within one test and others cannot use it. This metadata can be used to tag the question set visibility - *private, public or parent*.
### version
Version of the QuML specification using which the question set is created.
### members
List of questions or question set identifiers that are added to the question set.
### Curricular Metadata
This is a group of metadata fields to capture the pedagogic intent of the question set - which concepts are tested by the question set, etc. The metadata fields in this category vary for each domain. For example, questions of K-12 domain will have the following curricular metadata fields: *board, grade, subject, medium and topics*.
```
Optionally, a default source can also be provided for the HTML elements. If an asset variable is not declared, the default value of the source will be used by the QuML players.
*QuML implementations should upload assets to the QuML repository server at the time of creation. QuML players should use absolute path of the asset during online usage of the question and in case of offline usage, the asset should be pre-downloaded to the device/system where the question is being used.*
**Template Variables**
Template variables are used in templatized questions, which can be used to produce large number of similar questions. Template variables can be referred to by templateVariable objects in the question body.
Template variables can be referred to by using the html data attribute **“data-template-variable”** in the question body. This data attribute should be set on HTML elements which need to be templatized. And the template variables are resolved and replaced by an absolute value before the question is presented to a student. The rules for assigning values to template variables should be defined in the **“templateProcessing”** section of the question data.
This data attribute should be set on a element around a text that needs to be templatized. And when the question session starts, the text within the gets replaced with the value generated by template processing rule for the corresponding template variable.
```
Shyam has 5 apples.
Ram borrowed 3 apples from Shyam.
Shyam now has apples.
```
The above question is for subtraction, where the learner is asked to identify the number of apples Shyam has after Ram borrowed 3 of the 5 apples which Shyam had. Instead of showing the same numbers and fruit name always, the numbers and name of the fruit is templatized in the item body using template variables.
**Outcome Variables**
Outcome variables are declared by outcome declarations. Their value is set either from a default given in the declaration itself or during “response processing”. These variables are used for returning the score or for controlling the feedback to the student.
There are six reserved outcome variables:
* Questions that declare a numeric outcome variable representing the student’s overall performance on the question should use the outcome name **‘SCORE’** for the variable. SCORE needs to be a float.
* Questions that provide a feedback to student after the attempt should use the outcome variable **‘FEEDBACK’** to show the appropriate feedback modal. FEEDBACK needs to be a string.
* Questions that have in-built hints to help students in attempting the question should use the outcome variable **‘HINT’**. The value of this variable should be used by the QuML players to render the corresponding hint. HINT needs to be a string.
* Questions that declare a maximum score (in multiple response choice questions, for example) should do so by declaring the **‘MAXSCORE’** variable. MAXSCORE needs to be a float.
* Questions that declare a minimum score (minimum score to mark the student as passed) should do so by declaring the **‘MINSCORE’** variable. MINSCORE needs to be a float.
* Questions or tests that want to make the fact that the student scored above a predefined threshold available as a variable should use the **‘PASSED’** variable. PASSED needs to be a boolean. This variable is automatically set after response processing using the SCORE and MINSCORE outcome variables.
#### Built-in Outcome Variables
There is one built-in outcome variable, **‘completionStatus’**, that is declared implicitly and must not appear in outcome variables declaration. QuML players must maintain the value of the built-in outcome variable completionStatus. It starts with the reserved value “not\_attempted”. At the start of the first attempt, it changes to the reserved value “unknown”. It remains with this value for the duration of the question session unless set to a different value by a setOutcomeValue in responseProcessing. This variable changes to the reserved value “completed” by default if it is not updated during response processing.
There are four permitted values: 1. ‘completed’ - the student has experienced enough of the question to consider it completed; 2. ‘incomplete’ - the student has not experienced enough of the question to consider it completed; 3. ‘not\_attempted’ - the student is considered to have not used the question in any significant way; 4. ‘unknown’ - no assertion on the state of completion can be made.
### Body
Body contains the text, graphics, media objects and interactions that describe the question’s content and information about how it is structured. The body is presented by combining it with stylesheet and/or internationalization information, either explicitly or implicitly using the default style rules of the QuML player.
Body is presented to the student when the associated question session is in the *interacting* state. In this state, the student must be able to interact with each of the visible interactions and therefore set or update the values of the associated response variables.
The body may be presented to the student when the question session is in the *finished* or *review* state. In these states, although the student’s responses should be visible, the interactions or the setting and/or updating the values of the associated response variables must be disabled.
Finally, the body may be presented to the student in the *solution* state, in which case the correct values of the response variables must be visible and the associated interactions and/or response variable updates disabled.
The structural elements of the item body are taken from HTML with the following restrictions:
* It should contian only **structural, media and input HTML elements**. Structural elements include \\, \\, and . Media html elements are img, audio and video elements. Input html elements are text input, text area, select, options, checkbox, radio buttons, file upload and canvas elements.
* It should not contain **element**
* It should not have any **javascript code**, i.e. elements or onXYZ (onClick, etc) for HTML elements should not be used
* It should not have any **javascript import statements**
* It should not have any **css import statements**
In addition to the standard HTML elements and attributes, the specification allows the following in the question body:
* Usage for “data-” attributes to specify the interactions on HTML elements, bind response variables to the interactions, use template and i18n variables.
* Usage of standard style classes defined by the specificaiton. QuML players should provide implementation for the defined classes.
Body:
```
{
“itemBody”: “...”
}
```
### Interactions
A question can contain zero or more interactions with the user. Most of these interactions are typical question types (for example, multiple choice, order elements, fill in the blanks). We can also use interactions for activities like "upload document," "draw a picture," and "start a film".
Interactions can be defined in the item body using HTML data attributes as shown in the below sample (**“data--interaction”**). Each interaction should have an associated response variable that binds the interaction to a **“responseDeclaration”** with the same identifier. This is the link for the result and response processing, which are described later in this document.
```
```
QuML specification defines the following interaction types. QuML players should provide the implementation for all the defined interaction types. The behaviour of the interaction may vary in each implementation but should ensure that the defined specification is supported.
#### Simple Choice Interaction
The simple choice interaction is used when a set of choices are presented to the student. The student’s task is to select one of the choices. This interaction must be bound to a response variable with single cardinality (i.e. set only one value to the associated response variable).
| **Data Attribute** | **data-simple-choice-interaction** |
| ------------------ | -------------------------------------------------------------- |
| cardinality | *single* |
| type | string, integer or float |
| html elements | any HTML element which is clickable and have a value attribute |
#### Multi Choice Interaction
The multi choice interaction is used when a set of choices are presented to the student. The student’s task is to select one or more of the choices. This interaction must be bound to a response variable with multiple cardinality (i.e. set a list of values to the associated response variable). An array variable will be used to store the values of response variable for this interaction.
| **Data Attribute** | **data-multi-choice-interaction** |
| ------------------ | -------------------------------------------------------------- |
| cardinality | *multiple* |
| type | string, integer or float |
| html elements | any HTML element which is clickable and have a value attribute |
#### Text Interaction
A text interaction obtains a piece of text from the student. The QuML player must allow the student to review their choice within the context of the question. The text interaction must be bound to a response variable with single cardinality only. The type must be one of string, integer, or float.
| **Data Attribute** | **data-text-interaction** |
| ------------------ | ---------------------------------------------------------------------- |
| cardinality | *string* |
| type | string, integer or float |
| html elements | HTML elements where user can enter text, i.e. text input, or text area |
#### Order Interaction
In an order interaction, the student’s task is to reorder the choices. When specified, the student must select the choices and impose an ordering on them. If a default value is specified for the response variable associated with an order interaction then its value should be used to override the order of the choices specified here. The order interaction must be bound to a response variable with ordered cardinality (i.e. list of ordered values).
| **Data Attribute** | **data-ordered-interaction** |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------- |
| cardinality | *ordered* |
| type | string, integer or float |
| html elements | element. The values of all elements should be set in the same order as the value for response variable bound with the interaction |
> The order interaction implementation in the player should enable drag and drop of the elements in the element with this interaction. QuML players may choose to provide enhanced graphics/animation for order interaction.
#### Match Interaction
A match interaction presents candidates with two sets of choices and allows them to create associates between pairs of choices in the two sets, but not between pairs of choices in the same set. The match interaction must be bound to a response variable with type map and single cardinality.
| **Data Attribute** | **data-match-interaction** |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| cardinality | *single* |
| type | map |
| html elements | Two elements. Both the elements should be bound to the same response variable. The list of values of element pairs (mapped with each other) should be set as the value of the response variable. |
> Similar to order interaction, the match interaction implementation in the player should enable drag and drop of elements in one element on to elements in another element with this interaction. QuML players may choose to provide enhanced graphics/animation for match interaction.
#### Upload Interaction
The upload interaction allows the candidate to upload a pre-prepared file representing their response. It must be bound to a response variable with type file and single cardinality.
| **Data Attribute** | **data-upload-interaction** |
| ------------------ | --------------------------- |
| cardinality | *single* |
| type | uri |
| html elements | HTML file upload element |
> The upload interaction implementation in the player should upload the input file to a configured server and set the uploaded file url as the value for the response variable.
#### Map Interaction
The map interaction is a graphic interaction. The candidate’s task is to select one or more points on a canvas. The associated response may have a mapping that scores the response on the basis of comparing it against predefined areas but QuML players must not indicate these areas of the image. Only the actual point(s) selected by the candidate shall be indicated. The map interaction must be bound to a response variable with type points and multiple cardinality.
| **Data Attribute** | **data-map-interaction** |
| ------------------ | ------------------------ |
| cardinality | *multiple* |
| type | points |
| html elements | HTML canvas element |
#### Custom Interaction
The custom interaction provides an opportunity for extensibility of this specification to include support for interactions not currently documented. QuML implementations can implement a set of custom interactions that go beyond the standard interactions described above. Authors wishing to write questions for those implementations can use these custom interactions. However these custom templates should be published to ensure that these questions can be used with QuML players of other implementations. Otherwise, such questions will be limited to use only by that implementation.
### CSS classes
QuML recommends the use of Cascading Style Sheets (CSS) for controlling the content formatting. QuML has a defined set of standard classes for various different elements and interactions. A question can use those classes for formatting and QuML players should provide implementation for these classes.
* *1-col-layout*: style to create a one column layout
* *2-col-layout*: style to create a two column layout
* *3-col-layout*: style to create a three column layout
* *4-col-layout*: style to create a four column layout
* *6-col-layout*: style to create a six column layout
* *8-col-layout*: style to create a eight column layout
* *row*: style to add a row in the question
* *title*: for formatting title texts of the quesiton
* *sub-title*: for formatting sub-headings in the question
* *paragraph*: for formatting paragraph texts
* *small-image*: style to display small images
* *medium-image*: style to display medium size images
* *large-image*: style to display large images
* *horizontal-options*: style to layout options horizontally (can be used for choice, match and order interactions)
* *vertical-options*: style to layout options vertically (can be used for choice, match and order interactions)
* *file-upload*: styling for file upload elements (can be used for upload interaction)
* *canvas*: styling for canvas elements (can be used for map interaction)
Alternatively when there is a need for formatting that is not provided by the defined classes, a question can also have custom css. The support of the CSS elements used for formatting is totally dependent on the browser used to render the question. Hence it is advised to use only CSS elements that are supported by all the commonly used browsers and browser versions.
### Instructions
Some questions will have instructions on how to understand, attempt or how the question will be evaluated. Such instructions are defined in HTML format and stored in the instructions part of the question data.
Instructions:
```
{
“instructions”: “... … … ”
}
```
### i18n Data
Internationalization is supported by QuML to render texts inside body (answers, feedback and hints) in different locales. To enable this, QuML allows storage of content in multiple locales.
For example, **body** in different locales should be stored in body part of the question data as below:
body:
```
{
“body”: {
“