Executive Summary

The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make healthcare safer, higher quality, more accessible, equitable, and affordable and to work across the U.S. Department of Health and Human Services and in collaboration with other partners to make sure that the evidence is understood and used. A related AHRQ priority is to improve healthcare quality by accelerating the real-world clinical implementation of patient-centered outcomes research (PCOR) findings. AHRQ is also exploring the dissemination of PCOR findings as clinical decision support (CDS) resources that can be integrated with health information technology (IT), helping to move evidence into practice, while ensuring that CDS is more patient centered.

Working with stakeholders across the healthcare community, AHRQ assists CDS developers and implementers to transform PCOR findings into CDS that provides patient-specific information and evidence-based knowledge to clinicians, patients, or other individuals. CDS includes processes and mechanisms that aim to deliver the right information, to the right people, using the right formats, in the right channels, and at the right times during workflow (often referred to as the “5 Rights” framework). Well-implemented CDS can inform healthcare decisions and improve the quality and efficiency of the care provided to patients.

An additional AHRQ goal is to make CDS more shareable and publicly available in a format compliant with health IT standards. To help realize that goal, AHRQ contracted with the Centers for Medicare & Medicaid Services (CMS) Alliance to Modernize Healthcare Federally Funded Research and Development Center (the Health FFRDC), operated by the MITRE Corporation, to build and maintain CDS Connect. The CDS Connect project aims to generate a systematic and replicable process for transforming PCOR findings into shareable, standards-based, publicly available CDS, and to develop prototype tools to facilitate this transformation process. CDS Connect systems, resources, and tools provide the framework for improving healthcare outcomes by: (1) making interoperable CDS expressions (or “artifacts”) easier to create, (2) developing and sharing CDS artifacts and implementation guidance, (3) disseminating evidence-based research expressed as CDS, and (4) developing and releasing open-source health IT integration and implementation tools.

Clinicians today face an unending stream of new research findings, new or updated clinical practice guidelines, and best practices identified by peers that they must incorporate into daily practice. Transforming these large volumes of research into actionable knowledge that can be integrated into clinical care is a lengthy and expensive process that stretches the limits of what any one healthcare system can reliably accomplish on its own. Currently in its third year, the CDS Connect project provides an opportunity for healthcare organizations to share evidence-based knowledge expressed as CDS, enabling other organizations to leverage the publicly available expressions. The ability to share CDS expressions enhances efficiency by removing the need for subsequent organizations to develop CDS from “scratch.” It also contributes to a learning health community where CDS developers and implementers collaborate and enhance the shared resources.

CDS Connect is both a platform and a community of contributors and users, supported by open-source prototype tools for authoring, testing, implementing and sharing interoperable CDS. Central to CDS sharing is the CDS Connect Repository (the “Repository”) of CDS knowledge artifacts. Through the Repository, CDS contributors and CDS consumers have equal access to CDS artifacts generated from cutting-edge CDS research, and clinical and regulatory guidelines. Additionally, organizations can leverage advanced technical resources and tools posted within artifacts to aid in the implementation of the CDS logic and secure information testing. Contributions to the Repository continue to increase, expanding the breadth of resources available to stakeholders across the Nation.

In year 1 of the project, the CDS Connect team developed and delivered the initial (alpha) and second (beta) versions of the Repository to AHRQ. Year 2 saw the release of the third (production) version, with continued refinement based upon feedback from the CDS Connect Work Group (WG) and CDS contributors. In year 3, a number of enhancements were made to the Repository, including the addition of an application programming interface (API) to streamline the import and export of CDS artifacts, enhancements to the metadata fields provided to describe each artifact, a new approach to browse and search for artifacts (i.e., artifact discovery), and expanded capabilities for user accounts. Section 6 of this document contains more detailed information on each of the enhancements, and Appendix B lists all of the contributed artifacts to date. The CDS Connect team also developed several prototype tools to facilitate the development, testing, and sharing of CDS artifacts. The open-source CDS Authoring Tool (AT), developed in year 1 and enhanced in years 2 and 3, allows non-software engineers to build standards-based CDS logic using the Health Level 7 (HL7) Clinical Quality Language (CQL) standard. The AT allows people unfamiliar with CQL to develop structured, well-formatted CDS artifacts through a user-friendly interface, leveraging interoperable standards so that CDS can be written to common specifications. In year 3, the CDS Connect team designed and released three phases of enhancements to the AT. Additional details about the AT and enhancements made can be found in Section 7 of this document.

The CDS Connect team also developed a new prototype tool, the CQL Testing Framework, and enhanced an existing prototype tool, CQL Services, in year 3. The CQL Testing Framework allows CQL authors to develop and run test cases for validating CQL-based CDS logic. CQL Services is an open-source service framework for exposing CQL-based logic using the HL7 CDS Hooks application programming interface. This capability allows implementers to integrate CQL-based CDS into systems that do not yet support CQL natively. All software developed through the CDS Connect project is open-source and freely available.

Each year, the CDS Connect team develops one or more CDS artifacts and pilots the artifacts in partnership with a healthcare organization to: demonstrate a repeatable process for translating evidence-based knowledge into interoperable CDS artifacts, identify potential challenges while integrating the CDS artifacts into health IT systems and document lessons learned, test the CDS in a “live” environment and enhance the CDS expression as needed, and assess the Repository’s capability to host and share the CDS artifact. An overview of the CDS artifacts developed each year can be found in Table 1.

In year 3, AHRQ selected patient-facing CDS in the domain of preventive health as the “use case” (i.e., scenario to frame CDS development). Working with a pilot partner, the CDS Connect team developed four patient-facing CDS artifacts based on U.S. Preventive Services Task Force (USPSTF) recommendations. This process included integration of the CDS artifacts into the pilot partner’s health IT platform, providing real-world experience to inform the development and implementation of future CDS artifacts. Additional details on artifact development are found in Section 2, and more information on the pilot process is in Section 3.

Key to all streams of work, a CDS Connect Work Group (WG) was formed in year 1, and continued in years 2 and 3, to provide insight and advise the CDS Connect team on all aspects of CDS Connect work, including the identification and prioritization of key features and capabilities for CDS Connect systems and tools. Monthly WG meetings were attended by a broad array of CDS stakeholders (subject matter experts from across government, industry, academia, clinical settings, and nonprofits). In year 3, the WG members provided valuable input to the CDS Connect team on topics such as artifact development derived from the USPSTF recommendations, feedback on the CDS Connect Sustainability project to help shape the sustainability path and approaches, prioritization of prototype tool development and features, and feedback and ideas on enhancements to existing tools.

The CDS Connect leadership team also conducted extensive outreach via conference presentations, demonstrations, webinars, and strategic discussions to inform and maximize work efforts and increase adoption of the CDS Connect systems. Other noteworthy milestones and accomplishments as well as stakeholder engagement activities are described in more detail in additional sections of this document.

Throughout the third year of the CDS Connect project, the CDS Connect leadership team noted several valuable “lessons learned,” impacting the year 3 work and providing valuable insight for future CDS efforts:

1. Lessons Learned affecting the development of and enhancements to CDS Connect prototype tools:
   • The CDS Connect team considered adding support for additional terminology servers in the AT; however, after further research and discussion with the CDS Connect WG, no candidate terminology servers were identified. Organizations using the AT should choose between using the Value Set Authority Center (VSAC) interface currently available or modifying the AT source code to use their own proprietary terminology server.
   • VSAC was enhanced to add support for intensional value set creation, providing the ability to define value set codes using rules instead of enumerating every code. However, the VSAC Fast Healthcare Interoperability Resource (FHIR) API provided by the National Library of Medicine (NLM) has not been updated to support intensional value sets; therefore, the AT will not be able to display and execute them until this support is added.
   • When using CQL libraries with dependencies, all references to the same CQL library must use the same version of that library; therefore, CQL authors must ensure matching versions in dependency libraries.
   • The CQL Testing Framework prototype tool improved the efficiency and quality of CQL development for the CDS Connect technical team. The pilot partner, b.well, also benefitted through the receipt of bug-free CDS logic and automated test cases to streamline their testing efforts and ensure the accuracy of the CDS.
2. Lessons learned throughout the CDS pilot process:
   • During the pilot, the CQL Services software failed to return results for several patients due to the very large volume of data. This was resolved by increasing the default size limit in CQL Services and adding the capability to edit the default size. Implementers should ensure that tests include validation for similar large volume test cases.
   • Aggregating data from multiple sources presents great opportunities for coverage of required data, while introducing challenges in the completeness and specificity of the data, which may lack standard codes or specific data attributes. This compounds data mapping efforts, impacting resource needs as well as the implementation timeline. In general, the health IT industry would benefit from broader adoption of standardized terminologies and the FHIR data model, supporting increased interoperability and data aggregation and reducing mapping effort.
   • Pilot technical team experience with evolving technology tools such as CQL, CDS Hooks, and FHIR should be considered when planning the implementation timeline and work effort in future pilot efforts.
   • Future patient-facing CDS pilots should consider the importance of selecting a pilot partner with consumer/patient-facing experience and expertise, as well as ensuring that the pilot timeframe allows personalization and consumer/patient engagement opportunities to be realized.
3. Lessons learned affecting CDS artifact development:
   • The CDS Connect team desired to reuse value sets available on VSAC whenever possible but found that there were inconsistencies in the definition and ongoing maintenance, as well as duplicate value sets representing the same or similar concepts. A governance process to help ensure the validity and maintenance of available value sets would increase trust and improve value set reuse, while decreasing the number of “similar” value sets created.
   • The environmental scan performed early in the project provided key information to help shape the project strategy and decision making. The findings informed many aspects of the project, including the selection of the pilot partner and the focus of the preventive care CDS recommendations.
   • Access to and collaboration with knowledge authors ensures the accurate translation and representation of the CDS logic and interventions.
   • When defining clinical concepts in CDS logic, if the lookback period requires patient data recorded prior to the implementation of International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) on October 1, 2015, ICD-9-CM should be included in the concept definitions.
4. Feedback and engagement via the CDS WG and with other stakeholders provided major value to CDS Connect project efforts, informing the usefulness, usability, and longevity of CDS Connect resources and systems, while raising awareness and increasing contributions to the Repository and use of the AT.

These are discussed in greater detail in the Lessons Learned section of the document.

CDS Connect is helping to further AHRQ’s vision of advancing evidence into practice through the dissemination of shared, interoperable CDS and the development of publicly available tools and resources to facilitate integration of CDS into health IT systems. CDS Connect is also contributing to a learning health community by documenting lessons learned across all project activities and publishing the lessons learned, along with recommendations for future efforts in the CDS domain. Collaboration with stakeholders will guide CDS Connect activities moving forward, enabling new efforts to effectively serve the healthcare community.

Executive Summary

The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable and to work within the U.S. Department of Health and Human Services and with other partners to make sure that the evidence is understood and used. AHRQ’s priority areas of focus are: investing in research to make healthcare safer and improve quality, creating materials to train healthcare systems and professionals to put research results into practice, and generating measures and data used by providers and policymakers. AHRQ’s evidencebased tools and resources can be used to propel stakeholders toward improved quality, safety, effectiveness, and efficiency of health care.

The Patient Protection and Affordable Care Act (ACA) of 2010 directs AHRQ to disseminate and build capacity in patient-centered outcomes research (PCOR). Working with clinical organizations, AHRQ is assisting health information technology (IT) users with incorporating patient-centered research findings expressed as clinical decision support (CDS) into clinical practice. CDS provides patient-specific information and evidence-based knowledge via health IT to clinicians, patients, or other individuals. CDS includes processes and mechanisms that aim to deliver the right information, to the right person, using the right format, in the right channel, and at the right time during workflow (often referred to as the “5 Rights” framework). Well-implemented CDS can improve health care decisions and the quality and efficiency of the care provided to patients.

CDS provides patient-specific information and knowledge, enabled by health IT, to clinicians, patients, or other individuals to enhance health and health care. CDS includes processes and mechanisms that aim to deliver the right information, to the right person, using the right format, in the right channel, and at the right time during workflow (often referred to as the “5 Rights” framework). Well-implemented CDS can improve health care processes. AHRQ has launched an initiative that fulfills its ACA requirements related to PCOR and CDS.

The goals of AHRQ’s overall CDS initiative are to advance evidence into practice through CDS, and to make CDS more shareable, health IT standards-based, and publicly available. This effort includes the following components:

1. Engaging stakeholders to catalyze the development and use of PCOR-based CDS throughout the health care system and provide CDS developers and stakeholders with constructive advice and feedback.
2. Developing prototype infrastructure to create and share CDS, including coded clinical knowledge, implementation guides, and a publicly accessible repository of CDS resources or “artifacts.”
3. Advancing CDS research through demonstration and dissemination grant funding opportunities.
4. Evaluating the overall initiative, including lessons learned and future recommendations.

To realize the second component of this initiative, AHRQ sponsored the CDS Connect project, a contract with CMS Alliance to Modernize Healthcare, a federally funded research and development center operated by the MITRE Corporation. The CDS Connect team developed a production-level “CDS Connect” repository to host and share standards-based, interoperable CDS artifacts (knowledge expressions implemented in EHR systems to inform care) via a web-based, publicly available system. Through the Repository, CDS contributors and CDS consumers have equal access to knowledge generated from cutting-edge CDS research, as well as clinical and regulatory standards. Additionally, organizations that work to balance limited resources can leverage advanced technical resources and secure information critical to the CDS implementation process. Contributions to the Repository grow every day, expanding the breadth of resources available to stakeholders across the Nation.

The CDS Connect project also developed a production-level CDS Authoring Tool that leverages interoperable standards so that CDS can be written to common specifications. This capability was designed to be intuitive and easy to use, with the intention of reducing the development burden that health care organizations undertake. By making it easier to compose and express CDS artifacts, this software improves the quality of CDS design, accelerates the velocity of CDS development, ensures open access to supporting CDS resources, and enables integrated software systems for interoperable CDS. Furthermore, the code for this system is open-source, enabling greater access, adoption, and community involvement.

To demonstrate a repeatable CDS development process and the Repository’s capability to host and share CDS in this second year of the project, the CDS Connect team developed, tested, and implemented a CDS artifact that generated a summary view of patient data relevant to pain management. The “Pain Management Summary” markedly reduced clinician burden by compiling clinical data that normally needs to be searched for across several sections of an EHR. Documentation and sharing of the development and implementation processes promotes transparency and increased awareness to future developers and implementers—furthering efficiency and effectiveness.

Key to all streams of work, CDS Connect leaders hosted a monthly Work Group meeting attended by a broad array of CDS stakeholders (subject matter experts from across government, industry, academia, clinical settings, and nonprofits) and conducted outreach via conference presentations, demonstrations, webinars, and strategic discussions to inform and maximize work efforts and increase adoption of the CDS Connect systems.

This second year of the CDS Connect project enabled the two central systems (the Repository and Authoring Tool) to be developed into robust, valuable assets to the health care and CDS communities. This work provides the framework for improving health care outcomes via CDS creation, discovery, integration, and implementation using evidence-based interoperable CDS artifacts.

Executive Summary

The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable. AHRQ works within the U.S. Department of Health and Human Services and with other partners to make sure that the evidence is understood and used. AHRQ’s priority areas of focus are for improving health care quality by accelerating implementation of patient-centered outcomes research (PCOR). Further, AHRQ is focused on making health care safer, increasing accessibility to health care, and improving health care affordability, efficiency, and cost transparency.

The Patient Protection and Affordable Care Act (ACA) of 2010 directs AHRQ to disseminate and to build capacity in PCOR. Working with relevant medical and clinical associations, AHRQ is assisting users of health information technology (IT) focused on clinical decision support (CDS) to incorporate findings into clinical practice and to promote the ease of use of such incorporation. AHRQ supports processes for receiving feedback from stakeholders, including physicians, providers, patients, and commercial vendors of health IT focused on CDS about the value of the information and the assistance provided.

CDS provides patient-specific information and knowledge, enabled by health IT, to clinicians, patients, or other individuals to enhance health and health care. CDS includes processes and mechanisms that aim to deliver the right information, to the right person, using the right format, in the right channel, and at the right time during workflow (often referred to as the “5 Rights” framework). Well-implemented CDS can improve health care processes. AHRQ has launched an initiative that fulfills its ACA requirements related to PCOR and CDS.

The goals of the initiative are to advance evidence into practice through CDS and to make CDS more shareable, health IT standards-based, and publicly available. This effort includes the following components:

1. Engaging stakeholders to catalyze the development and use of PCOR-based CDS throughout the health care system and provide CDS developers and stakeholders with constructive advice and feedback.
2. Developing prototype infrastructure to develop and share CDS, including coded clinical knowledge, implementation guides, and a publicly accessible repository of CDS resources or “artifacts.”
3. Advancing CDS research through demonstration and dissemination grant funding opportunities.
4. Evaluating the overall initiative.

Through this project, named “CDS Connect,” the CMS Alliance to Modernize Healthcare v (CAMH) is piloting a process that accounts for the agile nature of CDS development. CAMH is a Federally Funded Research and Development Center (FFRDC) operated by the MITRE Corporation. As part of this work, this CDS Connect project has included clinical and technical translation of guidelines into computable CDS, testing and monitoring, implementation protocols, and feedback loops. Initial work is grounded in the domain of cholesterol management and designed to promote the transformation of findings into actionable, relevant, and interoperable clinical capabilities.

The CDS Repository is a web-based software service that offers structured data, aggregated resources, and the ability to leverage the international standard Clinical Quality Language (CQL). Through the CDS Repository, CDS contributors and CDS consumers have equal access to knowledge driven by cutting-edge research in CDS, as well as clinical and regulatory standards. Additionally, organizations that work to balance limited resources are able leverage advanced technical resources and secure information critical to the CDS implementation process.

The CDS Connect Project has convened two advisory Work Groups (Cholesterol Management and CDS Repository) consisting of CDS subject matter experts from across government, industry, academia, clinical settings, and non-profits. These Work Groups guide the development, implementation, and inclusion of cholesterol management CDS artifacts while providing ongoing collaboration in support of AHRQ’s commitment to developing a repository with the features and functions critical to the CDS community.

The CDS Connect project also supported the design and development of a prototype CDS Authoring Tool that leverages interoperable standards so that CDS can be written to common specifications. This capability was designed to be intuitive and easy to use, with the purposes of reducing provider burden. By making it easier to compose and express CDS artifacts, this software improves the quality of CDS design, accelerates the velocity of CDS development, ensures open access to supporting CDS resources, and enables integrated software systems for interoperable CDS.

This CDS Connect project has resulted in a proof-of-concept that will lead to the increased use of findings in clinical healthcare practice. This work is providing the foundation for improved healthcare outcomes via CDS creation, discovery, integration, and implementation using interoperable data standards to express the logic of the CDS for use by health IT software systems.

Executive Summary