Executive Summary

This year’s report details the project’s efforts relating to the Repository, Authoring Tool, open-source tools and CDS artifacts.  Highlights include developing a Repository User Guide and updating the CDS artifact Implementation Guides, which cover the topics of mitigating the risk of cardiovascular disease, pain management, diabetes, and statin use.  Recommendations, lessons learned and next steps in the transition are identified.  The report concludes with project resources to encourage readers to engage with the project and its transition activities.

Executive Summary

Healthcare is undergoing a digital revolution that will lead to innovations in using technology and data to impact care. As part of the Agency for Healthcare Research and Quality’s (AHRQ’s) clinical decision support (CDS) initiative, the CDS Connect project was launched in 2016 to put patient-centered outcomes research (PCOR) findings into practice. CDS Connect provides a web-based public platform for integrating evidence-based care more rapidly into clinical practice through electronic health systems.

The sixth year of the CDS Connect project initially focused on system maintenance and updates to ensure that the tools remained shareable, standards-based, and publicly available, with a shift after the first quarter of the project to supporting other AHRQ initiatives. This annual report organizes the project’s year-long accomplishments into four task areas: Task Management, Trust Framework, Standards Conformance, and Outreach.

Task Management (Manage Project Resources; Section 2)

With reduced tasking in Year 6 the CDS Connect project team (hereafter “the team”) was leaner, primarily with a single lead and support staff across respective tasks.

Trust Framework (Prioritize Activities Based on Promotion of Trust; Section 3)

The team engaged with end users and other stakeholders to identify and prioritize updates. This activity resulted in a framework focused on two primary “must have” themes and one secondary “should have” theme for CDS Connect updates.

The primary themes were “must have”—

• Enforce Standards Compliance (CDS Artifacts): Update CDS artifacts already in the Repository to ensure compliance with applicable standards.

Other themes were evenly supported, including “Expand the Repository,” “Increase Trust,” and “Expand the Use of Existing Artifacts.”

Standards Conformance
(Ensure that Tools Adhere to Standards and are Innovative; Sections 4 – 7)

CDS Connect tools and artifacts are more likely to be used if they adhere to existing, widely accepted standards and if they include useful innovations for end users. During this period of maintenance and update, the team introduced the following improvements based on the selected themes in the Trust Framework task:

• Implemented a standardized process to review and update (where needed) all CDS artifacts in the CDS Connect Repository.

• Updated CDS Connect project artifacts to follow best practices for Clinical Quality Language (CQL) development and reflect evidence updates from trusted clinical sources.

• Upgraded the CDS Connect Repository to Drupal 9 for ongoing vendor support.

• Developed the Repository User Guide.

• Created tiles and landing page for Clinical Decision Support Innovation Collaborative (CDSiC) project.

• Enhanced CDS Authoring Tool capabilities related to FHIR versions, artifact downloads, system integration, usability, and maintainability.

• Continued to maintain open-source software offerings to use latest libraries, including updated libraries for CQL execution.

• Continued the “Lessons Learned” series for the CDS Connect Work Group (WG), a community of more than 140 CDS members who provide diverse perspectives and help the team identify and prioritize key features and capabilities for CDS Connect.

Outreach (Engage with the CDS Community; Section 8)

Engaging with the CDS community through various channels promotes adoption of CDS Connect and increases use of the resources. During Year 6, the team continued to adapt to varied engagement formats and redirection from AHRQ on CDS community needs. The team engaged with the CDS community in the following ways:

• Participated in one in-person conference at the start of the year, with a focus on the use of CQL and the CDS Authoring Tool.

• Developed handouts and prioritized patient partnering in CDS through expansion of the Patient-Partnering Panel held during Year 5 of the project.

Collaborated and supported AHRQ’s CDSiC project and technical migration to Amazon Web Servicers (AWS) secure enclave.

These accomplishments during Year 6—primarily in the areas of prioritization of CDS needs in development and implementation, particularly with patient partnering—will enable the team to further its success in any possible future phases of the CDS Connect project.

Executive Summary

Healthcare is undergoing a digital revolution that will lead to innovations in using technology and data to impact care. As part of the Agency for Healthcare Research and Quality’s (AHRQ’s) clinical decision support (CDS) initiative, the CDS Connect project was launched in 2016 to put patient-centered outcomes research (PCOR) findings into practice. CDS Connect provides a web-based public platform for integrating evidence-based care more rapidly into clinical practice.

The fifth year of the CDS Connect project1 focused on system maintenance and updates to ensure that the tools remained shareable, standards-based, and publicly available. The team also focused on ensuring that the project’s activities and work product accurately reflect the needs of the community—addressing the perspectives of patients and caregivers as well as the needs of clinicians and content developers. This annual report organizes the project’s year-long accomplishments into four task areas: Task Management, Trust Framework, Standards Conformance, and Outreach.

Task Management (manage project resources; Section 2)
New staff with development expertise joined the CDS Connect project team (hereafter “the team”) to promote the adoption of standards and implementation of CDS Connect’s offerings.

Trust Framework (prioritize activities based on promotion of trust; Section 3)
The team engaged with end users and other stakeholders to identify and prioritize updates. This activity resulted in a framework focused on two primary “must have” themes and one secondary “should have” theme for CDS Connect updates.
Two primary themes were “must have”—

1. Enforce Standards Compliance (CDS Artifacts): Update CDS artifacts already in the Repository to ensure compliance with applicable standards

2. Expand the Use of Existing CDS Artifacts: Improve usability and update the Repository to support expanded use of existing CDS artifacts.

Another secondary theme was differentiated from the others as “should have”—

3. Increase Trust: Implement recommendations from the Trust Framework Work Group (WG).Standards

Conformance (ensure that tools adhere to standards and are innovative; Sections 4-7)
CDS Connect tools and artifacts are more likely to be used if they adhere to existing, widely accepted standards and if they include useful innovations for end users. During this period of maintenance and update, the team introduced the following improvements based on the selected themes in Trust Framework task:

• Implemented a process to review and update (where needed) all CDS artifacts in the CDS Connect Repository.
• Updated CDS Connect project artifacts to use the Fast Healthcare Interoperability Resources (FHIR®) Release 4 (R4) data model and follow best practices for Clinical Quality Language (CQL) development.
• Applied Clinical Practice Guidelines (CPG)-on-FHIR® to the Repository to ease their implementation by the CDS community.
• Upgraded the CDS Connect Repository to Drupal 9 for ongoing vendor support.
• Updated and enhanced CDS Application Programming Interface (API) to facilitate access to integrate tools with CDS Connect.
• Upgraded open-source software offerings to align more closely with CQL 1.5.
• Continued the “Lessons Learned” series for the WG, a community of more than 150 CDS members who provide diverse perspectives and help the team identify and prioritize key features and capabilities for CDS Connect).

Outreach (engage with the CDS community; Section 8)
Engaging with the CDS community through various channels promotes adoption of CDS Connect and increases use of the resources. During Year 5, the team continued to adapt to varied engagement formats, innovated its outreach based on limitations posed by the COVID-19 pandemic, and sought diverse outlets to interact with the CDS community. The team engaged with the CDS community in the following ways:

1. Participated in four virtual conferences throughout the year, with a focus on practical engagements that directly impact CDS Connect (e.g., immediate use or improvement).
2. Expanded its practices to track visit and download metrics across CDS Connect tools.
3. Pursued a collective effort to encourage patient partnering in CDS development and implementation (including site updates, WG discussions, and panel).
4. Delivered periodic updates to the CDS community through opt-in email notification.

These accomplishments during Year 5—particularly in the areas of system improvements and diversified stakeholder feedback—will enable the team to further its mission success throughout the next period of performance.

Executive Summary

AHRQ launched the Clinical Decision Support (CDS) Connect project in 2016 as part of its CDS initiative to put into practice patient-centered outcomes research (PCOR) findings, with focus on the topics, questions, and outcomes most important to patients and those who care for them. The first 3 years of the project included development and testing of the tools included in the CDS Connect platform. The fourth year of the CDS Connect project1 focused on maintenance and updates to ensure that the tools are shareable, standards-based, publicly available, and patient centered. This annual report documents the project’s accomplishments over the fourth year, organized into four task areas: Task Management, Trust Framework, Standards Conformance, and Outreach.

Task Management (manage project resources)

To meet the needs of the new maintenance and update phase, project leadership changed to provide expertise in program evaluation to support refinement of processes and an increased focus on growth and use.

In addition, new staff with expertise in user experience and engagement joined the CDS Connect project team (hereafter “the team”) to promote the adoption and implementation of CDS Connect’s offerings.

Trust Framework (prioritize activities based on promotion of trust)

To foster trust in CDS Connect, the team engaged with end users and other stakeholders to identify and prioritize updates. The team’s analysis of stakeholder feedback resulted in a framework focused on two primary themes for CDS Connect updates:

1. Validate and update artifacts, as needed, to ensure that they reflect the most recent evidence-based clinical practice.
2. Update artifacts already in the Repository to ensure compliance with applicable health information technology (IT) standards.

Standards Conformance (ensure tools adhere to standards and are innovative)

CDS Connect tools and artifacts are more likely to be used if they adhere to existing, widely accepted standards and include useful innovations for end users. Accordingly, during this period of maintenance and update, the team introduced the following improvements based on the selected themes in Trust Framework task:

1. Implemented a process to review and update, as needed, all CDS artifacts in the CDS Connect Repository.
2. Applied Clinical Practice Guidelines (CPG)-on-Fast Healthcare Interoperability Resources (FHIR®) to the Repository and CDS Authoring Tool to align CDS Connect’s tools and promote ease of implementation in the CDS community.
3. Upgraded software to FHIR Reference 4 (R4) to encourage innovative use.
4. Added a lessons learned series to the Work Group- 130+ CDS community members who provide diverse perspectives and help the team identify and prioritize key features and capabilities for CDS Connect
5. Refreshed the content across the CDS Connect website to accurately reflect the maturity of CDS Connect (e.g., using past tense for completed items as opposed to future tense).
6. Improved accessibility across all CDS Connect resources for persons with disabilities through broader application of standards established to support Section 508 accessibility requirements.

Aided by these improvements, members of the CDS community repurposed the CDS Authoring Tool to develop a prototype drug-drug interaction alerting system and query form while others built on the Factors to Consider in Managing Chronic Pain: A Pain Management Summary artifact.

The improvements to CDS Connect and deeper engagement with its stakeholders have established a foundation for ongoing success.

Executive Summary

The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce evidence to make healthcare safer, higher quality, more accessible, equitable, and affordable and to work across the U.S. Department of Health and Human Services and in collaboration with other partners to make sure that the evidence is understood and used. A related AHRQ priority is to improve healthcare quality by accelerating the real-world clinical implementation of patient-centered outcomes research (PCOR) findings. AHRQ is also exploring the dissemination of PCOR findings as clinical decision support (CDS) resources that can be integrated with health information technology (IT), helping to move evidence into practice, while ensuring that CDS is more patient centered.

Working with stakeholders across the healthcare community, AHRQ assists CDS developers and implementers to transform PCOR findings into CDS that provides patient-specific information and evidence-based knowledge to clinicians, patients, or other individuals. CDS includes processes and mechanisms that aim to deliver the right information, to the right people, using the right formats, in the right channels, and at the right times during workflow (often referred to as the “5 Rights” framework). Well-implemented CDS can inform healthcare decisions and improve the quality and efficiency of the care provided to patients.

An additional AHRQ goal is to make CDS more shareable and publicly available in a format compliant with health IT standards. To help realize that goal, AHRQ contracted with the Centers for Medicare & Medicaid Services (CMS) Alliance to Modernize Healthcare Federally Funded Research and Development Center (the Health FFRDC), operated by the MITRE Corporation, to build and maintain CDS Connect. The CDS Connect project aims to generate a systematic and replicable process for transforming PCOR findings into shareable, standards-based, publicly available CDS, and to develop prototype tools to facilitate this transformation process. CDS Connect systems, resources, and tools provide the framework for improving healthcare outcomes by: (1) making interoperable CDS expressions (or “artifacts”) easier to create, (2) developing and sharing CDS artifacts and implementation guidance, (3) disseminating evidence-based research expressed as CDS, and (4) developing and releasing open-source health IT integration and implementation tools.

Clinicians today face an unending stream of new research findings, new or updated clinical practice guidelines, and best practices identified by peers that they must incorporate into daily practice. Transforming these large volumes of research into actionable knowledge that can be integrated into clinical care is a lengthy and expensive process that stretches the limits of what any one healthcare system can reliably accomplish on its own. Currently in its third year, the CDS Connect project provides an opportunity for healthcare organizations to share evidence-based knowledge expressed as CDS, enabling other organizations to leverage the publicly available expressions. The ability to share CDS expressions enhances efficiency by removing the need for subsequent organizations to develop CDS from “scratch.” It also contributes to a learning health community where CDS developers and implementers collaborate and enhance the shared resources.

CDS Connect is both a platform and a community of contributors and users, supported by open-source prototype tools for authoring, testing, implementing and sharing interoperable CDS. Central to CDS sharing is the CDS Connect Repository (the “Repository”) of CDS knowledge artifacts. Through the Repository, CDS contributors and CDS consumers have equal access to CDS artifacts generated from cutting-edge CDS research, and clinical and regulatory guidelines. Additionally, organizations can leverage advanced technical resources and tools posted within artifacts to aid in the implementation of the CDS logic and secure information testing. Contributions to the Repository continue to increase, expanding the breadth of resources available to stakeholders across the Nation.

In year 1 of the project, the CDS Connect team developed and delivered the initial (alpha) and second (beta) versions of the Repository to AHRQ. Year 2 saw the release of the third (production) version, with continued refinement based upon feedback from the CDS Connect Work Group (WG) and CDS contributors. In year 3, a number of enhancements were made to the Repository, including the addition of an application programming interface (API) to streamline the import and export of CDS artifacts, enhancements to the metadata fields provided to describe each artifact, a new approach to browse and search for artifacts (i.e., artifact discovery), and expanded capabilities for user accounts. Section 6 of this document contains more detailed information on each of the enhancements, and Appendix B lists all of the contributed artifacts to date. The CDS Connect team also developed several prototype tools to facilitate the development, testing, and sharing of CDS artifacts. The open-source CDS Authoring Tool (AT), developed in year 1 and enhanced in years 2 and 3, allows non-software engineers to build standards-based CDS logic using the Health Level 7 (HL7) Clinical Quality Language (CQL) standard. The AT allows people unfamiliar with CQL to develop structured, well-formatted CDS artifacts through a user-friendly interface, leveraging interoperable standards so that CDS can be written to common specifications. In year 3, the CDS Connect team designed and released three phases of enhancements to the AT. Additional details about the AT and enhancements made can be found in Section 7 of this document.

The CDS Connect team also developed a new prototype tool, the CQL Testing Framework, and enhanced an existing prototype tool, CQL Services, in year 3. The CQL Testing Framework allows CQL authors to develop and run test cases for validating CQL-based CDS logic. CQL Services is an open-source service framework for exposing CQL-based logic using the HL7 CDS Hooks application programming interface. This capability allows implementers to integrate CQL-based CDS into systems that do not yet support CQL natively. All software developed through the CDS Connect project is open-source and freely available.

Each year, the CDS Connect team develops one or more CDS artifacts and pilots the artifacts in partnership with a healthcare organization to: demonstrate a repeatable process for translating evidence-based knowledge into interoperable CDS artifacts, identify potential challenges while integrating the CDS artifacts into health IT systems and document lessons learned, test the CDS in a “live” environment and enhance the CDS expression as needed, and assess the Repository’s capability to host and share the CDS artifact. An overview of the CDS artifacts developed each year can be found in Table 1.

In year 3, AHRQ selected patient-facing CDS in the domain of preventive health as the “use case” (i.e., scenario to frame CDS development). Working with a pilot partner, the CDS Connect team developed four patient-facing CDS artifacts based on U.S. Preventive Services Task Force (USPSTF) recommendations. This process included integration of the CDS artifacts into the pilot partner’s health IT platform, providing real-world experience to inform the development and implementation of future CDS artifacts. Additional details on artifact development are found in Section 2, and more information on the pilot process is in Section 3.

Key to all streams of work, a CDS Connect Work Group (WG) was formed in year 1, and continued in years 2 and 3, to provide insight and advise the CDS Connect team on all aspects of CDS Connect work, including the identification and prioritization of key features and capabilities for CDS Connect systems and tools. Monthly WG meetings were attended by a broad array of CDS stakeholders (subject matter experts from across government, industry, academia, clinical settings, and nonprofits). In year 3, the WG members provided valuable input to the CDS Connect team on topics such as artifact development derived from the USPSTF recommendations, feedback on the CDS Connect Sustainability project to help shape the sustainability path and approaches, prioritization of prototype tool development and features, and feedback and ideas on enhancements to existing tools.

The CDS Connect leadership team also conducted extensive outreach via conference presentations, demonstrations, webinars, and strategic discussions to inform and maximize work efforts and increase adoption of the CDS Connect systems. Other noteworthy milestones and accomplishments as well as stakeholder engagement activities are described in more detail in additional sections of this document.

Throughout the third year of the CDS Connect project, the CDS Connect leadership team noted several valuable “lessons learned,” impacting the year 3 work and providing valuable insight for future CDS efforts:

1. Lessons Learned affecting the development of and enhancements to CDS Connect prototype tools:
   • The CDS Connect team considered adding support for additional terminology servers in the AT; however, after further research and discussion with the CDS Connect WG, no candidate terminology servers were identified. Organizations using the AT should choose between using the Value Set Authority Center (VSAC) interface currently available or modifying the AT source code to use their own proprietary terminology server.
   • VSAC was enhanced to add support for intensional value set creation, providing the ability to define value set codes using rules instead of enumerating every code. However, the VSAC Fast Healthcare Interoperability Resource (FHIR) API provided by the National Library of Medicine (NLM) has not been updated to support intensional value sets; therefore, the AT will not be able to display and execute them until this support is added.
   • When using CQL libraries with dependencies, all references to the same CQL library must use the same version of that library; therefore, CQL authors must ensure matching versions in dependency libraries.
   • The CQL Testing Framework prototype tool improved the efficiency and quality of CQL development for the CDS Connect technical team. The pilot partner, b.well, also benefitted through the receipt of bug-free CDS logic and automated test cases to streamline their testing efforts and ensure the accuracy of the CDS.
2. Lessons learned throughout the CDS pilot process:
   • During the pilot, the CQL Services software failed to return results for several patients due to the very large volume of data. This was resolved by increasing the default size limit in CQL Services and adding the capability to edit the default size. Implementers should ensure that tests include validation for similar large volume test cases.
   • Aggregating data from multiple sources presents great opportunities for coverage of required data, while introducing challenges in the completeness and specificity of the data, which may lack standard codes or specific data attributes. This compounds data mapping efforts, impacting resource needs as well as the implementation timeline. In general, the health IT industry would benefit from broader adoption of standardized terminologies and the FHIR data model, supporting increased interoperability and data aggregation and reducing mapping effort.
   • Pilot technical team experience with evolving technology tools such as CQL, CDS Hooks, and FHIR should be considered when planning the implementation timeline and work effort in future pilot efforts.
   • Future patient-facing CDS pilots should consider the importance of selecting a pilot partner with consumer/patient-facing experience and expertise, as well as ensuring that the pilot timeframe allows personalization and consumer/patient engagement opportunities to be realized.
3. Lessons learned affecting CDS artifact development:
   • The CDS Connect team desired to reuse value sets available on VSAC whenever possible but found that there were inconsistencies in the definition and ongoing maintenance, as well as duplicate value sets representing the same or similar concepts. A governance process to help ensure the validity and maintenance of available value sets would increase trust and improve value set reuse, while decreasing the number of “similar” value sets created.
   • The environmental scan performed early in the project provided key information to help shape the project strategy and decision making. The findings informed many aspects of the project, including the selection of the pilot partner and the focus of the preventive care CDS recommendations.
   • Access to and collaboration with knowledge authors ensures the accurate translation and representation of the CDS logic and interventions.
   • When defining clinical concepts in CDS logic, if the lookback period requires patient data recorded prior to the implementation of International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) on October 1, 2015, ICD-9-CM should be included in the concept definitions.
4. Feedback and engagement via the CDS WG and with other stakeholders provided major value to CDS Connect project efforts, informing the usefulness, usability, and longevity of CDS Connect resources and systems, while raising awareness and increasing contributions to the Repository and use of the AT.

These are discussed in greater detail in the Lessons Learned section of the document.

CDS Connect is helping to further AHRQ’s vision of advancing evidence into practice through the dissemination of shared, interoperable CDS and the development of publicly available tools and resources to facilitate integration of CDS into health IT systems. CDS Connect is also contributing to a learning health community by documenting lessons learned across all project activities and publishing the lessons learned, along with recommendations for future efforts in the CDS domain. Collaboration with stakeholders will guide CDS Connect activities moving forward, enabling new efforts to effectively serve the healthcare community.