This document is intended as a resource for IT organizations and procurement offices within educational institutions. It provides basic background information on the QTI (Question and Test Interoperability®) and APIP® (Accessible Portable Item Protocol) standards, sample RFP language, and additional considerations when specifying support for 1EdTech-certified QTI as part of the procurement process.
The 1EdTech QTI interoperability standards provide a set of specifications that enable a rich collection of digital assessment content to be accessed from a wide variety of learning platforms, including learning management systems, authoring platforms, item/test repositories, assessment delivery engines, and web applications. Downloadable or web accessible content developed to conform to the specifications will "run" in a wide variety of conformant assessment platforms.
Basics: Content, Platforms, Specification Versions and Profiles
At a basic level, assessment content refers to the questions or types of interactions (such as true/false, multiple choice, fill-in the blank, essay, etc.) that comprise a given assessment. The QTI and APIP standards provide a data model and structure for the interchange of assessment content that contains information and resources required to make that content display to students, and in the case of APIP, includes information to accommodate a variety of specific needs and preferences (such as magnification, color overlay, Braille, sign language, etc.).
There are different specification version numbers to be aware of:
- QTI 3.0 is the newest version incorporating the accessibility & accommodation supports of APIP as well as a more web-friendly approach to content markup
- QTI v2.2 is backwards compatible with QTI v2.1 and contains new supports such as MathML v3, W3C ARIA tags, some HTML5 elements, etc.
- APIP v1.0 is an extension standard used in QTI 2.1 and included in QTI v2.2. APIP content is transferred in a specific APIP profile of content packaging.
- QTI 2.1 was the first widely adopted version of QTI 2.x
Version numbers are very important as they prescribe the features and formats that must be supported by conformant applications and content.
The reason there are multiple versions of the standards is that the specifications are evolving over time to support more functionality, much in the way that other digital formats evolve over time (.pdf, .doc, etc.). Higher version numbers indicate support for additional features beyond lower numbered versions.
To determine if QTI and APIP content will run in a certain applications, match the version number declared inside the content to the version supported by the conformant platform. For example, if digital content is designated as conformant to version QTI 3.0, it must be run in a platform that also conforms to version 3.0. It is possible for platforms to support more than one version or all of the versions of QTI/APIP. 1EdTech provides a list of the products that have achieved conformance and the version of the specification supported: http://www.imscert.org/
Support for QTI Features by Profile
A profile is a set of minimum features required for a certification. 1EdTech has pre-defined certifications adopted by suppliers and institutions around the world. Other QTI profiles are currently under development for specific assessment programs such as Smarter Balanced Assessment Consortium, New Meridian (formerly PARCC), the Dutch Ministry of Education, and the Australian government.
See the Suggested RFP Language section for how to request features beyond the minimum required features defined in a certification profile.
Metadata & Packages
Test publishers may use metadata to describe and categorize the content within a QTI or APIP package. While metadata is only required at the schema version notation inside a package, it may be placed on any resource. Developers must use a subset of LOM (Learning Object Metadata) to describe their resources. Curriculum standards metadata at the package, resource, and question item level is also available. Ask publishers to supply the metadata you need.
Suggested RFP Language and Resources for Procurement
To ensure that your educational technology ecosystem is interoperable, it is recommended that you require 1EdTech conformance certification as part of your procurement process and RFP documents.
The following RFP language is made with the understanding that there are many and various kinds of assessment applications/systems, and these depend upon the overall assessment and/or learning solution architecture.
When specifying standards for use in an assessment program, there are often differences across the whole system architecture that may require a separation of requirements. The guidance provided below will likely need adjustment for your particular assessment program.
The language below assumes the use of QTI version 3.0, commonly referred to as QTI 3. QTI 3 includes the most recent changes that reflect the current state-of-the-art assessment solutions. There may be subtle or significant differences in the language used for other versions of QTI which would need to be adjusted for the version specified.
All certified products are issued a certification number and are listed in the official 1EdTech Certified Product Directory at imscert.org. If you experience an interoperability issue with a product listed here, 1EdTech will work with the supplier to resolve the problem. If a product is not listed in the 1EdTech product directory it has either not passed 1EdTech certification testing or its certification has expired.
Suggested RFP Wording and Clauses for a QTI Applications
- [Institution name] is committed to an open standards-based, interoperable assessment technology ecosystem. We are requesting [vendor/product] obtain certification by 1EdTech for QTI [version number] [profile name] no later than [specific date]. *Specify specific properties, values, and extensions—see below.
- [Institution name] is requiring [name of product/supplier] to be 1EdTech-certified for QTI [version number] [profile name]. The vendor is required to provide a valid and current 1EdTech certification registration number. See imscert.org. The supplier is also required to remain current with subsequent versions of the QTI standard and receive certification within a “reasonable” [define] time period. Additionally, [vendor] will describe how the optional QTI features and supports may be added to the application at the request of [name of institution]. *The latest version—QTI 3.0—could be specified.
- Describe how the application conforms to the latest 1EdTech QTI standard as available on the 1EdTech website.
- Applications handling assessment content MUST have 1EdTech QTI certification at the version, level, capabilities, and features required in this contract.
- All content produced for the life of this contract must be valid instances of QTI [version number], as determined by the 1EdTech supplied validator.
Suggested RFP Wording Specific to QTI 3 Assessment Delivery Applications
The Assessment Delivery Application is defined as the system that presents assessment content to candidates and records the responses of candidates.
The Assessment Delivery System MUST, at a minimum, have a current 1EdTech certification for QTI 3, Level 1 (Entry) [or Level 2 (Core), OR any named QTI certification profile].
In addition, the assessment delivery application MUST include the following features:
- [a list of features is defined in the Appendix of the QTI 3 Conformance & Certification document – examples are listed below. Note you only need specifically state a feature if that feature is NOT included in the minimum certification level stated in the above paragraph]
- EXAMPLE – Q18 Select Point Interaction
- EXAMPLE – A36 Sign Language
- EXAMPLE – I17 Composite Items
- [include any other features required but not defined in the QTI 3 standard, including connections to other related applications/systems. You should include a definition of said features.]
OPTIONAL
The assessment delivery application MUST be able to adjust the presentation of assessment content to the specific needs of candidates based upon the Personal Needs and Preferences (PNP) profile using the 1EdTech Access for All (AfA) 3.0 standard (QTI 3 profile). The assessment delivery application MUST support the following AfA features:
- OPTIONAL – The assessment delivery application MUST have a current 1EdTech certification for QTI 3 Elevated Accessibility
- EXAMPLE – I14 Catalog Resources
- EXAMPLE – A41 Tactile Reference
- EXAMPLE – Custom Support: Illustrated Glossary
OPTIONAL
The Assessment Delivery Application MUST provide a demonstration of a fully compliant system by delivering a sample test, including all the features required for assessment delivery detailed in this contract. The demonstration MUST be given [3 months, or your preferred time frame] before the first operational assessment of this contract unless otherwise detailed in the signed contract between the contractor and the contractee.
OPTIONAL
The Assessment Delivery Application MUST be WCAG 2.1 Level AA compliant.
[Note that QTI 3 supports and encourages the W3C Accessibility standards.]
OR
The Assessment Delivery Application MUST meet the Section 508 Refresh of the Rehabilitation Act of 1973.
[Note that the Section 508 Refresh requires compliance with WCAG 2.0 Level AA requirements, an earlier version than the other optional requirement stated above.]
Suggested Wording for QTI 3 Authoring Applications
The assessment authoring application is defined as capable of creating new assessment content, or modifying stored assessment content. While QTI certified AUTHORING systems are not required to store content in QTI, the system MUST be capable of creating or modifying content that aligns with QTI features, which includes any of the required features of the certification levels or profiles for which they have been certified.
The assessment authoring application MUST, at a minimum, have a current 1EdTech certification for QTI 3, Entry Level. [Alternatively Core Level OR any named QTI certification profile].
In addition, the assessment delivery application MUST include the following features:
- [a list of features is defined in the Appendix of the QTI 3 Conformance & Certification document – examples are listed below. Note you only need specifically state a feature if that feature is NOT included in the minimum certification level stated in the above paragraph]
- EXAMPLE – Q18 Select Point Interaction
- EXAMPLE – A36 Sign Language
- EXAMPLE – I17 Composite Items
- [include any other features required but not defined in the QTI 3 standard. You should include a definition of said features.]
OPTIONAL
The assessment authoring application MUST be able to include supplemental and/or alternative assessment content for the specific needs of candidates. The assessment delivery application MUST support the following 1EdTech Access for All (AfA) 3.0 features:
- OPTIONAL – The assessment authoring application MUST have a current 1EdTech certification for QTI 3 Elevated Accessibility
- EXAMPLE – I14 Catalog Resources
- EXAMPLE – A41 Tactile Reference
- EXAMPLE – Custom Support: Illustrated Glossary
- OPTIONAL – any other accommodation or accessibility support that requires authoring prior to an online delivery session.
OPTIONAL
The Assessment Authoring Application MUST provide an exported sample test, including all the assessment content features required for the assessment authoring system detailed in this contract. The sample content MUST pass official 1EdTech validation. The example content MUST be provided [state your timeline].
Future
QTI Scoring Application
Elevated Accessibility Feature Set
This list and full definitions are available in the QTI 3 Conformance and Certification document, but is listed here for your convenience. Many of these features are associated with WCAG 2.1 AA compliance.
- I14 Catalog Resources
- A2a WAI-ARIA Subset
- A6 Additional Testing Time
- A8 Answer Masking
- A9 Audio-Description (of video)
- A10a Braille
- A11 Candidate breaks
- A13b Captions (track support)
- A29 Long Description
- A30a Magnification
- A36 Sign Language
- A40a Spoken (reading-type=screen-reader)
- A40b Spoken (reading-type=computer-read-aloud)
- A41 Tactile reference
- A44 Transcript (of media)
For QTI 2.2
There are two predefined certification profiles for QTI 2.2 (also aligning to QTI 3.0 predefined profiles), Entry and Core. Entry are the basic, minimum set of features required to be an interoperable QTI application. Core Level includes more advanced features for larger assessment programs.
For QTI 2.1 and APIP
View existing certifications in the 1EdTech Product Directory. The certification will list the specific content or type of application and level of certification. The Conformance Registration Number is listed with the product listing.
To view the specific conformance features required for the various certification versions and levels, see the QTI 2.1/APIP Conformance Matrix .
Procuring assessment applications:
- Include in the RFP a requirement for the assessment applications you’re looking to acquire (authoring platform, delivery, item/test bank, or other platform).
- For versions before QTI 3.0, look at the features supported by the various conformance profiles in the QTI/APIP Conformance Matrix to determine the functionality you would like the platform to support. Indicate in your RFP requirement which version of the specification or profile you would like supported.
- Specify that a Conformance Registration Number must be provided in the proposal (Check the registration numbers online when you receive the proposals). Alternatively, any contract may stipulate certification for a specific version and level by an agreed upon date, with penalties or contract cancellation if the certification is not complete by the stated date.
If you need assistance, please send an inquiry via our contact form, or reach out to us at support@1edtech.org.