IMS Global TECH TALK
Contributed by Dr. Colin Smythe, IMS Chief Architect
Using Compatibility Check to Improve Interoperability: From Conformance to Characterization
In a previous blog, I discussed the importance of Conformance and Certification as part of the IMS specification process. I also explained why IMS certification is required and why it is not "compliance." Certified products published in the IMS Product Directory have successfully passed rigorous IMS conformance testing. Defining the conformance requirements and providing the associated conformance test capabilities is essential for the IMS specification development process.
While certification is critical, it is fully product-focused and does not cover the configuration of an operationally deployed version of the product. There is a limit to the degree of interoperability guaranteed through certification only. It is possible for two products certified for a standard not to interoperate. Fortunately, the IMS Product Directory provides sufficient details so that confirmation of the correct type of certification is available. In the case of deployed operational systems, there are many different ways in which a certified system can be configured. As the next step beyond certification, IMS has created the Compatibility Check (CCx) system that is used to provide a characterization of a deployment.
Conformance testing is undertaken on a vendor's product configured for a testing environment. Typically, service providers are configured with the in-house test data sets i.e., real data is not used. IMS conformance testing ensures that invalid data is not sent by the service provider. Unless all of the data stored by the service provider is entered through the API undergoing conformance testing, this form of testing does not explore how the data set is entered into the service provider. Therefore, once the test data set has been verified as correct, an inherent part of conformance testing, there is no further check on the service provider's ability to prohibit the storage of invalid data sourced through other interfaces. I've already explained how two certified products may not provide data interoperability. However, there is one more likely reason why certified systems may not have full interoperability. The actual configuration of a product will depend upon the business models being used by the vendor e.g., specific endpoints may only be available for some business models. The fact that a product is certified does not mean that all of the features checked under conformance are available in an operational configuration.
Therefore, actual interoperability can ONLY be determined by examining the configurations of the deployed operational systems. We call this process Characterization.
Characterization enables the coverage of a specification in an operationally deployed implementation to be determined and recorded. Characterization is also the process by which data instances can be analyzed, i.e., covering the scenarios where only the data formats, not the exchange mechanism, are defined in a specification.
-
For service providers, the characterization is automated, whereas an engineer must complete a detailed functionality questionnaire for consumers.
-
For data import systems, a model of the import capabilities must be manually created, whereas, for data export systems, functionality coverage is created through the analysis of a set of exported data instances.
-
For REST API services, the characterization covers all endpoints, the associated query parameters, and the error handling mechanisms.
From the data perspective, characterization checks all of the data-typing, the supplied range of content, and any use of the extension mechanisms. The checking of the range of content enables the characterization to provide insight into the frequency of usage of data fields. For example, whether or not an optional field is always populated, the coverage of the set of enumerated tokens, etc. This content checking is important when confirming that the data required by a consuming system is being supplied by the provider (such a requirement would be an extra constraint on the optionality defined in a specification).
Compatibility Check is the software solution that provides the characterization capability. Characterization is only one of the features available through Compatibility Check. Two other features are available:
-
Verification – confirmation that the actual data being exchanged is valid with respect to the specification. Once characterization has been completed, any other data instances can be verified with respect to that characterization;
-
Compatibility Comparisons – the capability for a CCx user to explore their set of characterizations and to make detailed comparisons between matched characterizations. For example, the compatibility between a specific service provider deployment and the corresponding consumer(s) can be completed. This comparison provides a definitive, predictive statement on the interoperability matches and mismatches.
It is important to stress that while all of these analytics are stored (the characterization data), NONE of the actual data being exchanged is stored. Compatibility Check has been implemented such that a human never has access to, and cannot gain access to, the data being analyzed.
At present, Compatibility Check covers the IMS OneRoster and Common Cartridge standards. It also covers the set of apps under the IMS TrustEd Apps process i.e., apps that have been checked with respect to the IMS TrustEd Apps Rubric and that have the corresponding TrustEd Apps Seal. Access to CCx is available to IMS Contributing Members and Affiliate Suppliers who have OneRoster or Common Cartridge certification. CCx is also available to all Educational Institutions.
Compatibility Check is also a part of the Standards First Initiative. Standards First is a call-to-action for the edtech community. This initiative aims to ensure that we can achieve open standards-based integrations as the foundation for enabling product choice, improving cost, and enhancing data availability and student privacy. Standards First begins with the Pledge to make open standards the first and primary choice for edtech integrations. Compatibility Check is made available to confirm interoperability through the use of open standards and the correct usage of those open standards. Over time, CCx will be extended to cover other IMS specifications, particularly Learning Tools Interoperability (LTI) Advantage.
IMS Members wishing to find more on Standards First and Compatibility can contact Lisa Mattson at lmattson@imsglobal.org.
