Planned for August 29, 2024, the CDS Connect Project will be transitioning into a temporary hiatus. This transition is part of establishing a new sustainment model for CDS Connect related to the recent challenge competition and Request for Information. The creation of new user accounts will continue routinely until 09/12/2024. After that time, account creation may be delayed. Platform services are expected to remain available during this time but may experience occasional outages during this hiatus. For updates on the hiatus or to report an issue, please contact the AHRQ team at clinicaldecisionsupport@ahrq.hhs.gov.
The Clinical Decision Support (CDS) Authoring Tool is a web-based application to help CDS authors develop production-ready Health Level Seven (HL7®) Clinical Quality Language (CQL) logic without the need to fully understand the CQL specification.
Authoring is based on defining elements using a broad type, such as Condition, and applying one or more value sets and codes to specify a more specific concept, such as Diabetes. These elements can be further refined using modifiers to indicate a specific status, a most recent value, a comparison against a specific value, and more.
The CDS Authoring Tool is part of the CDS Connect project, sponsored by the Agency for Healthcare Research and Quality (AHRQ), and initially developed under contract with AHRQ by MITRE's Health FFRDC . For an overview of CDS Connect and how the CDS Authoring Tool fits in to the CDS Connect lifecycle, watch the ONC Tech Forum: Clinical Decision Support Series Session #2 , recorded in September 2023.
For an in-depth tutorial on using the CDS Authoring Tool, watch "A National Web Conference on the Clinical Decision Support Authoring Tool", recorded in February 2019. While new features have been added since then, the basic approach to authoring remains the same. This webinar is an excellent introduction for new users of the CDS Authoring Tool.
1. Starting with the CDS Authoring Tool
Before you can build CDS artifacts in the CDS Authoring Tool, you must have two accounts:
A CDS Authoring Tool account
A UMLS Terminology Services account
1.1 Requesting a CDS Authoring Tool Account
You must request a CDS Authoring Tool account in order to log in to the CDS Authoring Tool. This account provides you with your own space within the CDS Authoring Tool to create and manage CDS artifacts.
Click on the "SIGN UP" button in the middle of the screen.
This will launch a new window with a form that must be filled out and submitted.
After your request is reviewed, you will receive an email with further instructions to set up your account.
1.2 Requesting a UMLS Terminology Services Account
UMLS Terminology Services accounts are managed by the National Library of Medicine (NLM). This account is used within the CDS Authoring Tool to access services provided by the Value Set Authority Center (VSAC). These services enable you to search for value sets, review value set contents, and test CQL that uses value sets.
Follow the instructions from NLM to create your account.
1.3 Accessing Your UMLS API Key
To use your UMLS Terminology Services account in the CDS Authoring Tool, you must provide your UMLS API key. While previous versions of the CDS Authoring Tool used your UMLS username and password, recent changes to UMLS Terminology Services now require an API key instead. To find your UMLS API key, navigate to https://uts.nlm.nih.gov/uts/ and click the "Sign In" button in the banner.
Follow the directions to sign in and then click on the "My Profile" button in the banner.
Your API key is listed in your profile details. If you do not yet have an API key, you will see a hyperlink that allows you to generate one.
Since your API key is a long sequence of random characters, you may want to store it safely in a file on your system that you can easily copy and paste from in the future. Otherwise you will need to do the above sequence each time you authenticate to VSAC in the CDS Authoring Tool. NOTE: Your API key is personal to you and should not be shared with others. If you are using a public or shared computer, do not copy your API key to a local file.
1.4 Logging In to the CDS Authoring Tool
Once you have a CDS Authoring Tool account, you can log in to start creating and managing CDS artifacts. To log in, navigate to the CDS Authoring Tool home page and click the "Login" button in the Clinical Decision Support Authoring Tool banner near the top of the page.
When you click "Login", a new dialog will be displayed with a simple login form. Type your registered username and password into the form and then hit the return key or click the "Login" button.
If you encounter any issues logging in, click the "Contact Us" link on the CDS Authoring Tool home page to request support.
2. Creating and Managing Artifacts
You can create and manage CDS Artifacts by clicking the "Artifacts" link in the Clinical Decision Support Authoring banner near the top of the page. This will bring you to the "Artifacts" view that contains a list of all your CDS artifacts.
Your list of artifacts can only be seen by you. No other users have access to your artifacts or any of the content within them. You can only share artifacts by downloading the CQL and distributing it yourself. If you have an artifact that may be useful to others, we encourage you to consider posting it on the CDS Connect Repository.
2.1 Creating New Artifacts
To create a new artifact, click the "Create New Artifact" button in the "Artifacts" view. This will reveal a form, allowing you to enter an artifact name and version. There are no specific rules about artifact names or versions, but FHIR® Clinical Guidelines suggests using the Apache APR Versioning Scheme .
Once you've entered an artifact name and version, click the "Save" button. Your new artifact will appear at the top of the artifact list.
2.1.1 FHIR Clinical Guidelines (a.k.a. CPG on FHIR) Metadata
The Authoring Tool supports metadata conforming to the FHIR Clinical Guidelines Publishable Library specification. By clicking the "Show CPG Fields" button, form data may be entered.
A progress bar at the top of the form shows a score relative to the completion percentage of entered CPG metadata.
2.2 Managing Existing Artifacts
The "Artifacts" view contains a list of existing artifacts, showing the following information and controls for each artifact:
Name
Version
When it was last updated
When it was created
Edit Info button
Duplicate button
Delete button
To edit an artifact's logic, click the artifact's hyperlinked name. This will bring you to the artifact's "Workspace" view.
To edit an artifact's name or version, click its Edit Info button. This will open a modal dialog with a form for editing the name and version.
To duplicate an artifact, click its Duplicate button. This will create a completely new copy of your artifact in the artifact table.
To delete an artifact, click its Delete button. This will open a modal dialog asking you to confirm that you would like to delete the artifact. After clicking the "Delete" button to confirm, the artifact will be permanently deleted. This cannot be undone.
To sort the table by any of the columns simply click the column name. To reverse the order of the sort, click the column name again and the arrow indicating ascending / descending sort will change directions.
3. Building Artifacts
The first step in building an artifact is to create a blank artifact. If you have not yet done this, see the previous section, Creating and Managing Artifacts. Once you've created an artifact, click on its hyperlinked name to go to its "Workspace" view.
The workspace view contains a number of components that are used when building an artifact.
The following components can be found in the "Workspace" header:
Artifact Name: The artifact name. Click the pencil to the left of the artifact name to edit it.
View CQL Button: Allows the author to view the FHIR-based CQL of the artifact. When you click this button, you can choose which version of FHIR to use in the CQL view (DSTU2, STU3, or R4). For FHIR R4, you can choose between 4.0.0 and 4.0.1 depending on which CQL data model your system supports. If you are unsure, FHIR R4 (4.0.1) is preferred. NOTE: The FHIR version may be restricted depending on several factors, including external CQL that you've uploaded, custom modifiers that you've built, and/or your use of version-specific element types (e.g., Service Request).
Download CQL Button: Allows the author to download the artifact as both FHIR-based CQL and as a Clinical Practice Guidelines (a.k.a. CPG on FHIR) Publishable Library. When you click this button, you can choose which version of FHIR to use in the exported CQL (DSTU2, STU3, or R4). For FHIR R4, you can choose between 4.0.0 and 4.0.1 depending on which CQL data model your system supports. If you are unsure, FHIR R4 (4.0.1) is preferred. NOTE: The FHIR version may be restricted depending on several factors, including external CQL that you've uploaded, custom modifiers that you've built, and/or your use of version-specific element types (e.g., Service Request).
Save Button: Allows the author to explicitly save their progress. NOTE: Progress will also be automatically saved at certain points during the editing workflow.
Workspace Tabs: A set of tabs for organizing the different aspects of the CDS Logic: Summary, Inclusions, Exclusions, Subpopulations, Base Elements, Recommendations, Parameters, Handle Errors, and External CQL. If a tab has a checkmark icon , it contains user-provided content. If a tab has an exclamation icon , its content contains errors.
3.1 Creating and Editing Elements
The CDS Authoring Tool allows authors to create CDS by building and combining "elements" in specific contexts. Elements describe the criteria that is used to determine if a given patient qualifies for a CDS recommendation ("Inclusions"), should be disqualified from a recommendation ("Exclusions"), or should receive a more specific recommendation ("Subpopulations").
The general approach for creating elements is the same across all contexts:
Select an element type, usually corresponding to a FHIR resource type (e.g., Condition or Observation).
Depending on the type that was selected,
Associate the element with at least one value set or code to give it more specific meaning (e.g., "Diabetes" or "LDL Cholesterol Test"), and/or...
Provide additional information as prompted by specific forms.
Modify the results by building a custom modifier or selecting built-in expression modifiers that further filter the results (e.g., "Verified"), get a specific property of the result (e.g., "Quantity Value"), or test the result (e.g., "> 130 mg/dL").
3.1.1 Select an Element Type
The first step to create an element is to select the element's type. The element type will determine what kind of data the element retrieves from the patient record. To select an element type, click in the dropdown box next to the "Add element" label.
Depending on your artifact you many see any or all of these choices:
Allergy Intolerance: Instances of the FHIR AllergyIntolerance resource type
Base Elements: Re-usable elements defined in the "Base Elements" tab
Condition: Instances of the FHIR Condition resource type
Demographics: Age or Gender as specified in an instance of the FHIR Patient resource type
Device: Instances of the FHIR Device resource type
Encounter: Instances of the FHIR Encounter resource type
External CQL: Named CQL definitions, parameters, and functions from CQL files uploaded in the "External CQL" tab
Immunization: Instances of the FHIR Immunization resource type
Medication Statement: Instances of the FHIR MedicationStatement resource type
Medication Request: Instances of the FHIR MedicationRequest(STU3/R4) or MedicationOrder(DSTU2) resource type
Observation: Instances of the Observation resource type
Parameters: Parameter values for parameters defined in the "Parameters" tab
Procedure: Instances of the FHIR Procedure resource type
ServiceRequest: Instances of the FHIR ServiceRequest resource type, available only in FHIR R4
Click on the choice corresponding to the type of data this element should be based on.
3.1.2 Associate the Element
After choosing a type, you may need to associate it with a Value Set or Code. If you see a button labeled "Authenticate VSAC", "Add Value Set", or "Add Code", then you have selected an element type which requires you to narrow it down further via terminology. These types also show a small key icon next to the name to indicate you will need to authenticate with the National Library of Medicine's Values Set Authority Center (VSAC).
If you see an "Authenticate VSAC" button, you must click it and then enter in your UMLS Terminology Services API key. If you do not have a UMLS Terminology Services account, see Requesting a UMLS Terminology Services Account. If you do not know your UMLS API key, see Accessing Your UMLS API Key.
After you authenticate, you will see two new buttons appear in your element box: "Add Value Set" and "Add Code".
3.1.3 Add Value Set
In the CDS Authoring Tool, authors use value sets to indicate a grouping of codes that should be used to find matching items in a patient's health record. For example, a "Diabetes" value set might contain many different codes that all indicate some diagnosis of Diabetes. These codes may differ in specificity or sub-types (e.g., Type 1 Diabetes, Type 2 Diabetes) or may be from different code systems (e.g., ICD-9, ICD-10, SNOMED-CT). If an author associates a value set with an element, then patient data need only match one of the codes in the value set to be considered a match for the element.
To associate a value set with an element, click on the "Add Value Set" button. If you do not see an "Add Value Set" button, first authenticate with the VSAC as described in the section above. After clicking the "Add Value Set" button, a modal dialog will appear allowing you to search for a value set by keyword. Enter a keyword representing the value set you want to find and click the "Search" button.
After clicking "Search", a set of results will be displayed. Each item in the results represents a value set in the VSAC. Each row contains information about the value set, including its name, object identifier (OID), steward, reviewed and updated dates, and the number of codes in the value set. Each row can be expanded for further details about the value set. Click on the value set you'd like to add, or to see the contents of a specific value set, click its eye icon under codes.
After clicking on a specific value set's eye icon , its full set of codes will be displayed. Each item in the list shows a contained code, its name, and its system. This allows authors to confirm that the value set contents match their intent. If this value set is not a good match or you want to inspect other value sets in the results, click the left arrow button near the top of the dialog. Otherwise, click the specific value set or "Select" button while viewing the contained codes to confirm the selection of the displayed value set.
Upon selection of the value set, the element will be updated with the value set association and a name (which defaults to the value set name). Feel free to modify the name to more closely match the final intent of the element. At this point, you can associate additional value sets or codes. These value sets and codes will be treated as a unioned set, meaning that an item in a patient's health record need only match one of any of them in order to be matched on the element as a whole.
Associated value sets will be reflected in the expression phrase as well as listed in the element's metadata. You can review a value set's contents by clicking on the eye icon to the right of the listed value set. You can delete the value set association by clicking the "X" icon to the far right of the listed value set.
Depending on the context of the element, you may also see a warning message indicating that the element does not have the correct return type. For example, if this is an element in the Inclusions or Exclusions criteria, it must have a return type "boolean" (e.g., a true or false answer). This is because every element in the criteria is combined using "and" or "or" clauses to arrive at an aggregate answer: true (the patient meets the criteria) or false (the patient does not meet the criteria). To satisfy the requirement for a specific return type, you must modify the results until the required return type is achieved.
3.1.4 Add Code
In cases where only a certain code should be used to match patient data for an element, authors can associate a single code with an element without using a value set. This feature can also be used to support matching on a code that is valid but is not a part of a value set that is also associated to the element.
To associate a specific code with an element, click on the "Add Code" button. If you do not see an "Add Code" button, first authenticate with the VSAC as described above. After clicking the "Add Code" button, a modal dialog will appear allowing you to enter a code and select the system from which it came. Currently, the CDS Authoring Tool supports the following systems, which are also supported by the VSAC: SNOMED, ICD-9, ICD-10, NCI, LOINC, and RXNORM. To choose a different code system, select "Other" in the code system dropdown menu and provide the FHIR-compatible identifying URL for the code system. Once you've entered the code and selected the appropriate system, click the "Validate" button.
After clicking "Validate", the CDS Authoring Tool will send the code to the VSAC for validation. If the code is from a supported code system and verified by VSAC, the CDS Authoring Tool will display the code, its system, and its display text. You can confirm the selection by clicking on the "Select" button at the bottom of the dialog.
If the code fails validation, this means the code is invalid or the code system is not supported. In either case, an error will be shown at the bottom of the dialog. You can choose to modify the code and try again, cancel the code selection process, or select the code despite the validation error. To cancel the code selection, click the "Cancel" button at the bottom of the dialog. To select the code anyway, click the "Select" button at the bottom of the dialog.
Upon selection of the code, the element will be updated with the code association and a name. Feel free to modify the name to more closely match the final intent of the element. At this point, you can associate additional codes or value sets. These codes and value sets will be treated as a unioned set, meaning that an item in a patient's health record need only match one of any of them in order to be matched on the element as a whole.
Associated codes will be reflected in the expression phrase as well as listed in the element's metadata. You can delete the code association by clicking the "X" icon to the far right of the listed value set.
Depending on the context of the element, you may also see a warning message indicating that the element does not have the correct return type. For more information see the last paragraph in the section above.
3.1.5 Name the Element
All elements provide a user-configurable field for the element name. In some cases, the element name may be pre-populated, but you may change the name if desired. It's considered a best practice to reflect the intent of the element in its name. For example, if an element checks for an LDL cholesterol test with a result over 130 mg/dL, then "LDL-c > 130 mg/dL" is a better name than just "LDL-c Test".
3.1.6 Annotate the Element
All elements support comments. The comments field allows authors to provide additional information about an element. By default, comments are collapsed in order to preserve space in the user interface. To toggle a comment field's visibility, click on the comment icon in the element header.
Comments can be used in many ways. Recommended uses of comments include:
Providing a rationale for why the element is created or indicating a decision that was made when creating the element and why that decision was made
Providing a simple summary of complex logic that is encapsulated in the element, or just further explaining the element built
Indicating a source from which the element's logic is derived and providing any necessary references
Indicating a section of logic that implementers may want to modify based on specific site needs
All comments are added to the generated CQL above the element's definition. However, they are not executed as part of the artifact and just serve as an annotation to the element.
When an element's comment field contains information, the comments icon changes to a subtle blue color and contains three dots. This allows authors to quickly see which elements have comments, even when the comments are collapsed.
3.1.7 Provide Additional Information
Some element types reqest additional information. Currently, the following two element types require additional information:
Demographics -> Age Range requires at least one of: Minimum Age, Maximum Age; as well as the unit of time (e.g., years).
Demographics -> Gender requires one of the following values to be selected: Male, Female, Other, Unknown.
3.1.8 Modify Results
Once you have defined the core element, including associated value sets and codes (if applicable), you can further filter the results or change the result type by applying modifiers. Authors may build their own modifier or select a built-in modifier from a list. The options available to the author differ depending on the element's return type. For elements with FHIR-based types (e.g., Condition, Observation, etc.), the initial return type is a list of items matching that type (e.g., List of Conditions). As modifiers are added to the element, the return type may change. For example, an author can use the "Exists" modifier to change an element's return type to a "Boolean" value in order use the element as inclusion criteria.
To apply a modifier to an element, click on the "Add Modifiers" button near the bottom of the element box. When you do this, a context menu will be displayed prompting a selection between the options 'Select Modifiers' and 'Build New Modifier'. Selecting modifiers allows authors to choose from a library of built-in modifiers. Building a new modifier allows for more detailed control over the fields, operations, and parameters used to filter the element's results. Section 3.1.9 covers the selection of built-in modifiers. Section 3.1.10 covers how to build custom modifiers.
3.1.9 Select Modifiers
The CDS Authoring Tool comes pre-loaded with a variety of modifiers for many different common situations. CDS Authors can use modifiers to further define or narrow an element's intent. To see the available modifiers for your current element, click the 'Select Modfiers' button within the Modifiers context menu. This will display the modifier selection menu.
By clicking the dropdown within this menu, you can see all of the eligible built-in modifiers that are available for use with the current return type. You may also successively chain modifiers within this menu. When you have finished adding your modifiers, press the 'Add' button to apply the changes and go back to the main element editor.
For some modifiers, you may be asked to provide additional information about the operation you would like to perform. For example, you may need to enter a quantity value to compare against an element's result quantity. In the example below, a 'Value Comparison' modifier has been applied to the 'Quantity' type returned by the 'Average Observation Value' modifier. The checkmark icon next to the new return type of 'Boolean' indicates that the new return type meets the requirements of the given context (e.g., inclusion criteria).
Since modifiers form a chain of operations, authors can only remove the last modifier or a modifier that does not change the previous return type. This preserves the integrity of the chain of operations. To remove a modifier, click on the "X" on the far right hand side of the modifier's row in the element's content.
3.1.10 Build Modifiers
While the CDS Authoring Tool provides a robust set of built-in modifiers, some authors may require (or prefer) more precise control over the mechanics of the modifier. Authors can accomplish this by building their own custom modifiers. While building custom modifiers is not as flexible as importing external CQL, it is much simpler and is well-suited for intermediate level users.
To build a custom modifier, open the modifiers context menu by pressing the 'Add Modifiers' button on an element; then click the 'Build New Modifier' button.
If your artifact has not been previously locked to a FHIR® version, you will be prompted to select one. If your artifact has been previously locked to a given version, it will be automatically selected for you.
Once a FHIR® version has been selected, the modifier builder will be displayed. A top-level group is shown with options to add a new rule or add a new group. Rules represent specific field-based criteria to filter on, while groups allow for rules to be combined and nested using boolean logic (and/or). A pink background indicates that a rule is incomplete or invalid.
To add a rule to the modifier, click the 'Add rule' link and then select a property from the dropdown list. The list of properties to choose from is based on the return type of the element. The CDS Authoring Tool does not support every possible property for the given type, but supports many of the properties commonly used in CDS use cases. Once you have chosen a property, choose from the list of operators that are available for that property.
After a field and operator have been chosen, the rule form will be updated to collect any additional information that is needed. The available inputs will differ depending on the field and operator. Once they have been properly specified, the background will no longer be pink, indicating that the rule is now valid. You may continue to add rules in this manner.
Sometimes elements may require more complex criteria with nested groups. For example, a modifier may need to specify 'A and B and (C or D)'. This type of boolean logic can be accomplished via groups. To add a new group, click the 'Add group' link. Select 'AND' or 'OR' depending on whether all the rules in the group must be satisfied or if it is sufficient for only one (or more) rules to be satisfied. Rules are added to nested groups in the same way they are added to the top-level group of the modifier.
To save the modifier and apply it to the element, click the 'Add' button. Once you've built a custom modifier, you can edit it by clicking the pencil icon next to the modifier text in the element editor.
The ability to build custom modifiers is a new feature of the CDS Authoring Tool. In its current iteration, custom modifiers can only filter lists of FHIR resource instances. Future versions may provide additional capabilities such as sorting and returning specific properties.
3.2 Deleting Elements
The CDS Authoring Tool allows authors to delete elements they no longer need or use. To delete an element, click the "X" button in the top-right corner of the element's box. Currently, the CDS Authoring Tool does not require authors to confirm deletion; the element is deleted immediately. This action cannot be undone.
In some cases, the CDS Authoring Tool will not allow you to delete an element. If the CDS Authoring Tool prevents you from deleting an element, this means that it is referenced and used elsewhere. To delete the element, you must first delete (or modify) all usages of that element.
3.3 Collapsing Elements
Because elements can contain a lot of detail, they may take up a large amount of space on the screen. In order to allow for a more streamlined view of the artifact's logic, the CDS Authoring Tool allows authors to collapse elements. When an element is collapsed, only its name and expression phrase are displayed.
To collapse an element, click the double-down arrow between the indent button and the delete button in the element's box. To re-expand an element, click the double-right arrow between the indent button and the delete button in the element's box.
3.4 Combining Elements
While elements can be useful on their own, usually they are combined with other elements in order to represent more complex ideas or requirements. The CDS Authoring Tool supports the following ways of combining elements:
And: Require every element in a set of boolean elements to be true
Or: Require at least one element in a set of boolean elements to be true
Indented Group: Group a set of elements together to represent a single logical unit
Intersect: Find the set of items that occur in each element of a set of elements
Union: Combine items from multiple elements into a single set of items
3.3.1 And
Combining elements using "And" indicates that each of the elements should evaluate to a "true" result in order for the logic to be satisfied. For example, you might combine an Age Range element and a Gender element using "And" to indicate that a patient must be 40 - 75 years old AND male. Elements must have a boolean return type in order to be combined using "And".
The Inclusions, Exclusions, and Subpopulations tabs all require elements to be combined using And, Or, or indented groups. In these tabs, you can combine elements using "And" by clicking the dropdown between elements and selecting "And". The CDS Authoring Tool requires that all sibling elements (e.g., elements indented at the same level) use the same combination logic, so when you select "And" between two elements, it will also be selected for all other elements at the same indent level.
In the Base Elements tab, elements can be combined using the "List Operations" element type. Do this by choosing "List Operations" in the element type dropdown and then selecting "And" in the second dropdown. This will create a group that you can give a unique name and begin adding elements to.
3.3.2 Or
Combining elements using "Or" indicates that at least one of the elements should evaluate to a "true" result in order for the logic to be satisfied. For example, you might combine an LDL-c element and a Diabetes element to indicate that a patient should have an LDL cholesterol reading above a certain threshold or a diagnosis of diabetes. Elements must have a boolean return type in order to be combined using "Or".
The Inclusions, Exclusions, and Subpopulations tabs all require elements to be combined using And, Or, or indented groups. In these tabs, you can combine elements using "Or" by clicking the dropdown between elements and selecting "Or". The CDS Authoring Tool requires that all sibling elements (e.g., elements indented at the same level) use the same combination logic, so when you select "Or" between two elements, it will also be selected for all other elements at the same indent level.
In the Base Elements tab, elements can be combined using the "List Operations" element type. Do this by choosing "List Operations" in the element type dropdown and then selecting "Or" in the second dropdown. This will create a group that you can give a unique name and begin adding elements to.
3.3.3 Indented Group
Sometimes authors need to create complex logic that requires the use of "And" and "Or" at the same time. This is accomplished by creating and combining sub-groups of elements. For example, you might want to indicate that a patient must be 40 - 75 years old and have either an LDL cholesterol reading above a certain threshold or a diagnosis of diabetes. To do this, you would create the first two elements as described in the And section above, but then you would convert the second element to an indented group and add the third element to that indented group, combining it using Or.
To create a new indented group, create the first element that should go in the indented group as normal. Then click on the "indent" button to the right of the element name to automatically create a new indented group and place the element in it. This new group can (and should) be given a unique name. To add more elements in the group, use the new "Add element" box that is displayed at the bottom of the indent group.
Groups can be indented multiple levels deep. In addition, groups can be outdented to return their elements back to the outer indent level. Use the "outdent" button to outdent groups.
3.3.4 Intersect
Combining elements using "Intersect" indicates that each of the elements should be inspected and only those items that match every element should be included in the intersection results. For example, if you combine an element representing confirmed myocardial infarctions with an element representing myocardial infarctions in the last six months, using "intersect", the result will be only myocardial infarctions that are in both element sets (i.e., confirmed and within the last six months).
The "Intersect" combination can only be applied in the Base Elements tab using the "List Operations" element type. Do this by choosing "List Operations" in the element type dropdown and then selecting "Intersect" in the second dropdown. This will create a group that you can give a unique name and begin adding elements to. All of the elements in the group will be intersected together. Since intersection requires elements to have returned items in common, the elements in an intersected group should all have the same result type, otherwise it is impossible for any item to match against all of the elements.
3.3.5 Union
Combining elements using "Union" indicates that all of the items across all of the elements should be combined together into a single set of items. For example, combining an LDL-c element with an HDL-c element using "Union" will result in all of LDL-c and HDL-c observations.
The "Union" combination can only be applied in the Base Elements tab using the "List Operations" element type. Do this by choosing "List Operations" in the element type dropdown and then selecting "Union" in the second dropdown. This will create a group that you can give a unique name and begin adding elements to. All of the elements in the group will be unioned together. If elements in the union have different return types, the return type of the overall union will be "Any".
3.5 Summary Tab
The Summary tab provides an overview of the artifact, including its name, version, date last changed, date created, and the high-level structure of the CDS logic. For the CDS logic, only the inclusions, exclusions, and recommendations are displayed. Each element within the inclusions and exclusions shows its type, name, and a link to the element definition in its respective authoring tab. Each recommendation shows its text, associated subpopulation (if applicable), associated rationale (if applicable), and a link to the recommendation definition within the Recommendations tab.
3.6 Inclusions Tab
Authors use the Inclusions tab to specify the criteria that patients should meet in order to receive a recommendation from the artifact. If a patient meets the criteria in the Inclusions tab and does not meet any criteria in the Exclusions tab, then they are considered to be part of the general population for which this artifact's recommendations apply.
Authors specify the Inclusions criteria by creating and combining elements as described in the sections above. Typically, Inclusions criteria will use "And" combinations so that all of the criteria must be met in order to receive a recommendation.
3.7 Exclusions Tab
Authors use the Exclusions tab to specify criteria that should disqualify patients from receiving a recommendation from the artifact. Even if a patient meets the criteria in the Inclusions tab, the criteria in the Exclusions tab can prevent them from being in the general population for which this artifact's recommendations apply. For example, some recommendations should not be provided when a patient is pregnant, even if they are otherwise indicated.
Authors specify the Exclusions criteria by creating and combining elements as described in the sections above. Typically, Exclusions criteria will use "Or" combinations so that any of the criteria can disqualify the patient from the recommendation.
3.8 Subpopulations Tab
Authors use the Subpopulations tab to specify criteria that groups patients into subpopulation that can be associated with more specific recommendations. While subpopulations are not required, they can be useful for artifacts that need to deliver more nuanced recommendations or recommendations that are population-specific. For example, a statin artifact might deliver a different strength recommendation for a patient with a 10-year risk score of 8% versus a patient with a 10-year risk score of 12%.
The Subpopulations tab allows authors to create as many subpopulations as needed. For each subpopulation, the author must specify a unique name, then create and combine elements as described in the sections above. To create a new subpopulation, click the "New subpopulation" button in the Subpopulations tab.
3.9 Base Elements Tab
Authors use Base Elements to create elements that can be re-used across several elements and contexts within the artifact. This allows for common elements to be defined once and used, as well as further modified, where ever they may be needed. This also allows authors to define standalone elements outside of any specific context and have these elements exported as-is in the CQL.
3.9.1 Creating Base Elements
Authors specify Base Elements by creating and (optionally) combining elements as described in the sections above. Since these elements are not in the context of specific logical constructs, they are not required to have any specific return type.
3.9.2 Using Base Elements
Authors can reference and use Base Elements in the Inclusions tab, Exclusions tab, and Subpopulations tab. To use a base element, it must be created in the Base Elements tab first. After that, use it in any valid location by selecting the "Base Elements" type in the element type dropdown. A second dropdown will appear allowing you to choose the Base Element you want.
An element that is based on a base element is shaded light blue to make it more easily distinguishable. In addition, it lists the original base element's name in the content of its element definition using the "Base Element" label. If you click the link icon to the far right of the base element's name, you will go directly to the base element definition.
If the base element does not have the return type required by its context (for example, if it is used in the Inclusions tab but does not have a boolean return type), you can add modifiers to it in much the same way as you would any element. In this case, the modifiers added to the use of the base element will not affect the original definition of the base element in the Base Elements tab. This allows base elements to used in a broad set of contexts in differing ways.
When a Base Element has been used, the definition of the element in the Base Elements tab will show where it is being used. This is indicated by rows labeled "Element Use" in the content of the element definition. If you click the link icon to the far right of the use's elementname, you will go directly to the element's definition.
Note that when a base element is used, certain restrictions are put into place. One restriction is that you cannot delete it. To delete a base element that is being used, you must first delete (or edit) all of its uses.
Another restriction is that applied modifiers cannot be removed if doing so would change the Base Element's return type. This ensures that modifying a base element does not make its uses become invalid. In order to remove the modifier, first delete all uses of the Base Element in the artifact.
In addition, modifiers cannot be added to base elements that are being used, unless the modifier does not change the overall return type of the element. If the modifier would change the return type of the base element, then you must first remove all uses of the Base Element in the artifact before you can add the modifier.
3.10 Recommendations Tab
Recommendations are the resulting notices that should be delivered to the clinician after the CDS Artifact is executed. Recommendations are written as free text and can have an accompanying Rationale. For the simplest cases, authors specify a single recommendation which will be delivered for any patient who meets the Inclusions criteria but does not meet any Exclusions criteria. For more advanced cases, authors can specify recommendations tailored to specific subpopulations.
3.10.1 Creating Recommendations
To create a recommendation, click the "New recommendation" button. This will add a blank recommendation below the last recommendation, if any exist. Note that recommendations are processed in the order they appear, and patients will receive the first recommendation for which they are eligible.
Sometimes authors may want to include a rationale for the recommendation as a separate data field. To do this, click on the "Add rationale" button in any recommendation element. This will display another text field which can be used to enter the rationale.
3.10.2 Associating Recommendations with Subpopulations
For CDS artifacts that may deliver more than one recommendation, authors must associate recommendations with the subpopulations to which they apply. The CDS Authoring Tool includes the following subpopulations by default:
Doesn't Meet Inclusion Criteria: Patients who don't meet the inclusion criteria usually will not receive a recommendation, but associating a recommendation with this subpopulation allows you to still deliver a notice for these patients.
Meets Exclusion Criteria: Patients who meet the exclusion criteria usually will not receive a recommendation, but associating a recommendation with this subpopulation allows you to still deliver a notice for these patients.
In addition, any subpopulations that you created can also be associated with recommendations. For More information on creating subpopulations, see Subpopulations documentation.
To associate a recommendation with subpopulations, click the "Add subpopulation" button. This will display a section above the recommendation text where you can select a subpopulation. After selecting a subpopulation, you can optionally select another subpopulation. The recommendation will only be delivered if the patient meets every subpopulation criteria that has been associated with the recommendation.
3.10.3 Adding Links to Recommendations
Recommendations can include links to reference information, educational materials, or SMART applications. This capability fits well with implementations that deliver CDS using HL7 CDS Hooks.
To add links to a recommendation, click the "Add Link" button. For each link, you must select either 'absolute' or 'smart' for the link type. Then enter the link text (the text to display) and the link address/url (the destination when clicking on the link). You may add more than one link.
3.10.4 Adding Suggestions to Recommendations
Suggestions present clinicians with one or more actions that they may choose to take as part of the recommendation. Common examples might include prescribing a medication or referring to a specialist. This capability fits well with implementations that deliver CDS using HL7 CDS Hooks.
To add suggestions to a recommendation, click the "Add Suggestion" button. For each suggestion, you should add a label for the suggestion.
Each suggestion can have one or more actions. To add an action, use the "Add Action" button. Currently, only "create" actions are supported, and only MedicationRequest and ServiceRequest resources can be created by these actions. Select the resource type that will be created for this action. This will open a dialog where you can add values for the elements of the chosen resource. Click "Create" to add the action to the suggestion.
Each action is displayed within the suggestion. To edit an action, click its Edit button. To delete an action, click its Delete button.
You may add multiple suggestions for each recommendation, and you may add multiple actions for each suggestion.
3.10.5 Re-ordering and Deleting Recommendations
Since patients will receive the first recommendation that applies to them, the order that recommendations are listed will affect how the CDS artifact behaves. In some cases, authors may need to re-order their recommendations with this in mind. To re-order recommendations, click the up arrow or down arrow on any recommendation to move it up or down.
Authors may also need to delete recommendations if they were entered by mistake or are no longer applicable. To do this, click the "X" button in the recommendation element box. This will prompt you to confirm the deletion. Once you have confirmed this, the recommendation will be permanently deleted. This action cannot be undone.
3.11 Parameters Tab
Parameters allow authors to create named elements whose values can be supplied by the CDS execution environment at run-time. Authors can specify default values for parameters, but are not required to do so. Parameters provide a useful approach for allowing certain aspects of CDS behavior to be tailored for specific environments without the need to modify the CDS logic itself. For example, an artifact that is capable of delivering grade B and grade C recommendations might specify a parameter called "Allow Grade C" with a default value: true. At sites where grade C recommendations are not allowed, the CDS can be run with the parameter value set to false in order to supress grade C recommendations. The author, of course, needs to properly use the parameter in their logic to implement this behavior. To support this, parameters can be referred to in elements throughout the CDS logic (much like base elements).
3.11.1 Creating Parameters
Authors can define as many parameters as they'd like. To create a parameter, click the "New parameter" button. An empty parameter box will be displayed with fields for the parameter name, user-provided comments, the parameter type, and an optional default value.
The parameter name should reflect the purpose of the parameter, making it easier for implementers to understand what it represents and how it might affect the artifact. The comments box, however, can be used to provide additional information about the parameter and its affect on the behavior of the artifact.
The parameter type provides a dropdown selection of available parameter types: Boolean, Code, Concept, Integer, DateTime, Decimal, Quantity, String, Time, Interval<Integer>, Interval<DateTime>, Interval<Decimal>, and Interval<Quantity>. Once you've selected a parameter type, the default value editor will provide an appropriate interface for editing the type of value you chose. Specifying a default value is optional, but if no default value is specified, then implementers must specify a value for that parameter at run-time.
3.11.2 Using Parameters
Authors can reference and use Parameters in the Inclusions tab, Exclusions tab, Subpopulations tab, and Handle Errors tab. To use a parameter, it must be created in the Parameters tab first. After that, use it in any valid location by selecting the "Parameters" type in the element type dropdown. A second dropdown will appear allowing you to choose the Parameter you want.
An element using a parameter indicates the parameter in its expression phrase as well as listing the parameter's name in the content of its element definition using the "Parameter" label. If you click the link icon to the far right of the parameter's name, you will go directly to the parameter definition.
If the parameter does not have the return type required by its context (for example, if it is used in the Inclusions tab but does not have a boolean return type), you can add modifiers to it in much the same way as you would any element.
Note that when a parameter is used, certain restrictions are put into place: it cannot be deleted nor can its parameter type be changed. To delete a parameter or change its type, you must first delete (or edit) all of its uses.
3.11.3 Deleting Parameters
Authors may want to delete parameters if they were entered by mistake or are no longer applicable. To do this, click the "X" button in the parameter element box. This will immediately delete the parameter without requesting further confirmation. This action cannot be undone. If the "X" button is disabled, this means you cannot delete the parameter because it is currently being used by another element. To delete it, you must first remove all uses of it from other elements.
3.12 Handle Errors Tab
Authors use the "Handle Errors" tab to define what messages should be provided to end users when certain error conditions arise. For example, an author might want to return an error message if required data is missing from the patient's health record. Handling errors is optional and may not be required or useful for certain use cases.
NOTE: Although CQL 1.2 introduced the "Messages" function to allow authors to return error messages to the operating environment at run-time, the CDS Authoring Tool does not currently use this approach. Instead, it creates a CQL statement called "Errors" that will return a list of the errors that are applicable to the patient in context. Each error in the list is a simple text string. Although the result is a list, the CDS Authoring Tool is currently only capable of returning one error per invocation, so the list length will never be more than one.
3.12.1 Error Conditions
Authors specify error handling by chaining together conditional "If" statements, indicating that if a certain condition (or set of conditions) is met, then a certain error message should be returned. By default, the CDS Authoring Tool provides three initial conditions that can be used when specifying errors:
Recommendations is null: indicates that no recommendation was provided.
Doesn't Meet Inclusion Criteria: indicates that the patient didn't meet the initial criteria to receive the recommensation.
Meets Exclusion Criteria: indicates that the patient met the criteria that automatically excludes them from receiving the recommendation regardless of whether or not they met the inclusion criteria.
In addition to the conditions above, authors can select subpopulations or boolean-valued parameters as conditions to indicate when an error message should be returned.
3.12.2 If Condition Then
To setup the simplest error handling case, choose a condition from the dropdown menu to the right of the "If" label in the "Errors" box. After you've selected the condition on which the error should be returned, enter the error message into the text box below the label "Then".
3.12.3 And Also If...
To create more complex conditional logic, click on the "And Also If..." button. This allows you to specify more detailed error handling for when the top-level condition is met. Clicking the "And Also If..." button will display a new set of indented error handling controls. These controls work the same as the top-level set of error handling controls, but will only be active when the top-level condition is met first.
3.12.4 Or Else If...
To create alternate error messages for other conditions, click on the "Or Else If..." button. This allows you to choose a condition that will be evaluated only if none of the other "If" or "Else if" conditions above it were met. If it evaluates to true (and the previous conditions did not), then its error message will be returned.
3.12.5 Else
The "Else" text box allows authors to specify an error if no other error messages were generated. This is rarely used at the top-level, as most CDS should not return any error if everything runs smoothly. In nested conditions (started by "And Also If..."), however, the "Else" box can be used to specify an error if the top-level condition was met, but none of the sub-conditions were met. If the "Else" text box is left empty, no error message will be returned if no other error condition has been satisfied.
3.13 External CQL Tab
Authors can use the "External CQL" tab to upload CQL files that contain logic that the author wants to use in the current CDS artifact. This is useful when you want to re-use existing logic rather than re-write it or when you want to use CQL logic that the CDS Authoring Tool is not capable of producing (for example, complex mathematical operations or timing relationships).
Note that uploading a CQL library does not make it editable in the CDS Authoring Tool. The External CQL library is considered read-only. Authors, however, can use External CQL library elements as the basis of authored elements and apply further modifiers to customize them for their artifacts.
3.13.1 Uploading External CQL
The CQL Authoring Tool allows CQL files to be uploaded individually or as a group of files contained in a zip file. CQL files that are uploaded individually must not depend on any external libraries aside from FHIRHelpers. If you want to upload a library that depends on other libraries (using CQL's "include" keyword), you must upload a zip file containing the library and its dependencies.
There are two ways to upload External CQL files (whether .cql or .zip):
Find the file on your local disk and drag it into the drop zone (outlined by a dashed line).
Click anywhere in the drop zone to open a dialog you can use to find and select your file.
The CDS Authoring Tool will attempt to compile your CQL file when you upload it. If there are any errors, it will display an error message and the file upload will be abandoned. If there are no errors, the upload is successful and the file will appear in the list of External CQL libraries below the drop zone.
3.13.2 Managing External CQL
The "External CQL" tab contains a list of uploaded CQL libraries, showing the following information and controls for each artifact:
Library
Last Updated
Version
FHIR® Version
View Details button
Delete button
To view a summary of the contents of the External CQL library, click on the View Details button. This will display a modal window with high-level metadata about the library and a listing of the library's definitions, parameters, and functions. For each of these, the name and return type are shown. For functions, the list of arguments is also shown.
To delete an External CQL library, click its Delete button. This will open a modal dialog asking you to confirm that you would like to delete the External CQL file. After clicking the "Delete" button to confirm, the External CQL library will be permanently deleted. This cannot be undone.
If the Delete button is disabled, then this CQL library is being used within the artifact or is declared as a dependency of one of the other External CQL libraries. To delete an External CQL library in this case, you must first delete any uses of it in the artifact and/or the external libraries that declare it as a dependency.
3.13.3 Using External CQL Elements
Authors can reference and use definitions, parameters, and functions from External CQL libraries in the Inclusions tab, Exclusions tab, Subpopulations tab, and Base Elements tab. To use an External CQL element, select the "External CQL" type in the element type dropdown in a place an element can be created. A second dropdown will appear allowing you to choose the External CQL library that contains the definition, parameter, or function you wish to use. After selecting a library, a third dropdown will appear allowing you to choose the specific definition, parameter, or function to use.
After selecting the External CQL definition, parameter, or function, its name and return type will be reflected in the element's expression phrase and listed in the element's metadata using the label "External CQL Element". If you wish (or if required), you can add modifiers to the element in much the same way as you would any element. In this case, the modifiers added to the use of the External CQL element will not affect the original definition of the External CQL element.
If the External CQL element you select is a function that has required arguments, the appropriate editors will be supplied for you to populate with values. These editors consist of two dropdowns: one which allows you to select the source of the argument and the other that is used to select a value according to your chosen source. The available sources for arguments are Base Elements, Editors, External CQL, and Parameters. Note that Base Elements, External CQL, and Parameters are only available for selection when an appropriate element exists that matches the type of the argument you are attempting to populate. If a particular argument source is greyed out within the first dropdown, this is an indication that there are no elements from that source that match the type of your particular function argument.
Once an argument source is selected, you may populate the second field to complete that particular function argument. If "Editor" is selected as the argument source, an appropriate editor will be displayed for completion. If one of Base Elements, External CQL, or Parameters is selected, you will be prompted with a second dropdown listing the matching elements of that type. Note that when using External CQL Elements as arguments you will be restricted to External CQL Functions with zero arguments, External CQL Parameters, and external CQL definitions. You may, however, use elements from a different External CQL library as an argument to an external function of another.
Note that when an External CQL element is used, it prevents the External CQL library from being deleted from the artifact. To delete the External CQL library, you must first delete its use. In addition, when an External CQL library is uploaded in an artifact, that artifact will become locked into the same version of FHIR that the External CQL uses. After this, you cannot download the artifact as other versions of FHIR, you cannot test the artifact against synthetic patient data using another version of FHIR, and you cannot upload additional External CQL using a different version of FHIR.
3.13.4 Using External CQL Modifiers
In certain situations, authors can use functions from External CQL libraries as modifiers on elements in an artifact. An external function can be used to modify an element in this way if all of the following conditions are met:
The function has at least one argument.
The first argument of the function matches the return type of the element.
One of the following is true for each argument:
There exists a Base Element whose return type matches the argument's type.
There exists a Parameter whose return type matches the argument's type.
There exists an External CQL Element whose return type matches the argument's type.
Note: Only External CQL definitions, parameters, and functions with zero arguments are valid as arguments to external modifiers.
The argument's type is one of the following:
Boolean, Code, Concept, Integer, DateTime, Decimal, Quantity, String, Time, Interval<Integer>, Interval<DateTime>, Interval<Decimal>, and Interval<Quantity>.
If these conditions are met, the modifier will appear just like any other after selecting the "Add Modifier" button. The modifier will be labeled with the function's name and the library's name, as well as a book icon to indicate that it is from an External CQL library.
Once an External CQL modifier is selected, it is displayed on the element just like any other modifier. The return type of the element will now be the return type of the External CQL function. If the External CQL function only has one argument, then the modifier is complete and no further action is required because this argument will be populated by the existing element. If the function has additional arguments, then the modifier will populate with editors that will allow you to select an argument source and an appropriate value for that source.
Like above with External CQL functions with arguments, the available sources for arguments are Base Elements, Editors, External CQL, and Parameters. Note that Base Elements, External CQL, and Parameters are only available for selection when an appropriate element exists that matches the type of the argument you are attempting to populate. If a particular argument source is greyed out within the first dropdown, this is an indication that there are no elements from that source that match the type of your particular function argument.
Once an argument source is selected, you may populate the second field to complete that particular function argument. If "Editor" is selected as the argument source, an appropriate editor will be displayed for completion. If one of Base Elements, External CQL, or Parameters is selected, you will be prompted with a second dropdown listing the matching elements of that type. Note that when using External CQL Elements as arguments you will be restricted to External CQL Functions with zero arguments, External CQL Parameters, and external CQL definitions. You may, however, use elements from a different External CQL library as an argument to an external function of another.
Similarly to when an External CQL element is used, when an External CQL modifier is used, its corresponding External CQL library cannot be deleted until the modifier is removed.
4. Testing Artifacts
While CDS artifacts should always be tested in the environment where they are deployed, testing is an important part of the artifact development process as well. Testing CDS logic before it is deployed in a test environment allows authors to discover and fix bugs earlier in the process, saving both time and money. This also makes it easier to distinguish between bugs in the logic and bugs in the data or integration environment. To test your artifacts, click on the "Testing" link in the Clinical Decision Support Authoring banner near the top of the page.
The CDS Authoring Tool testing view allows authors to upload their own synthetic test patients as FHIR bundles of synthetic patient data. The author can then run their CDS logic against one or more of these patients and inspect the results to determine if they are as expected. The CDS Authoring Tool does not currently provide a test patient editor, nor does it provide a mechanism for automated verification of results (e.g., "test assertions"). For more advanced testing capabilities, consider CDS Connect's CQL Testing Framework .
4.1 Uploading Test Patients
The first step in testing an artifact is to upload synthetic patient data to the CDS Authoring Tool. The CDS Authoring Tool environment is not appropriate for real patient data (whether de-identified or not). Only synthetic data should be used. Patient data should be uploaded as a FHIR bundle (DSTU2, STU3, or R4). Authors familiar with FHIR can use their tool of choice to create patient test data. Popular options include CDS Connect's CQL Testing Framework , Synthea , ClinFHIR , and FHIR® Shorthand .
There are two ways to upload synthetic test patient bundles (in .json format):
Find the file on your local disk and drag it into the drop zone (outlined by a dashed line).
Click anywhere in the drop zone to open a dialog you can use to find and select your file.
When you upload a patient, the CDS Authoring Tool will try to determine which version of FHIR the patient data conforms to. If it can detect a FHIR version, the patient will be added and the FHIR version will be chosen automatically. If the FHIR version cannot be detected, a modal dialog will be displayed asking you to indicate which version of FHIR the patient data conforms to. Choose DSTU2, STU3, or R4.
This may affect what artifacts can be tested using this patient and/or what FHIR version of the artifact will be used to do the testing on this patient.
4.2 Managing Test Patients
The "Testing" tab contains a list of uploaded synthetic test patients, showing the following information and controls for each patient:
Name
Birth Date
Gender
FHIR® Version
Last Updated
View button
Download button
Delete button
To view a summary of a synthetic test patient's data, click on the View button. This will display a modal window with the patient's demographic data and the individual entries within the patient's health record under the "Summary" tab. The entries are grouped according to type (e.g., Conditions, Medications, Encounters, etc.). Groups can be expanded and collapsed using the right arrow and down arrow (respectively). When expanded, each group shows a table with the most relevant data for that type (e.g., conditions show onset and abatement, medications show date written, etc.).
To view the full details of a synthetic test patient's data, click the "Details" tab in the modal window. This will display the exact FHIR representation of the patient. This can be helpful for more detailed debugging or when the patient view summary data does not show the relevant fields of interest.
To download a bundle with the synthetic test patient's data, click the Download button. This will download a JSON file with a FHIR Bundle containing the synthetic patient's data.
To delete a patient, click its Delete button. This will open a modal dialog asking you to confirm that you would like to delete the patient. After clicking the "Delete" button to confirm, the patient will be permanently deleted. This cannot be undone.
4.3 Test Execution
Once your test patients have been uploaded (see section above), you can run any compatible CDS artifact against them. Compatible artifacts are artifacts that can be exported using the same version of FHIR as the patient data is encoded in. Note that some artifacts may not be compatible with your patients because the artifact is locked to a specific version of FHIR based on the External CQL it uses.
In order to run the CDS artifact's CQL, the CDS Authoring Tool must request additional value set information (e.g., the set of codes) from the Value Set Authority Center. If you see an "Authenticate VSAC" button, you must first click it and then enter in your UMLS Terminology Services account username and password (this is not the same as your CDS Authoring Tool username and password). After you authenticate, you will see a new button titled "Execute CQL on Selected Patients".
To run your CDS artifact against your test patients, select the patients you want to test by checking the box to the left of each patient's name. Once you've chosen the first test patient, you can only select other patients that use the same version of FHIR. The CDS Authoring Tool does not support operating on multiple versions of FHIR in a single test run. After selecting the patients you want to test click the "Execute CQL on Selected Patients" button to open the testing details modal dialog.
The testing details dialog contains a dropdown select box allowing authors to choose a FHIR compatible artifact. Choose the artifact you want to test from the list. If you do not see your artifact, it is likely locked to a different version of FHIR than the patients you have selected. If the artifact you selected has parameters defined, an additional field will display for each parameter with the default value preselected. You can use these fields to override the default values. After populating the parameter values (if applicable), click on the "Execute CQL" button to close the dialog and execute the CQL.
4.4 Test Execution Results
After executing the CQL, the results will be displayed on the main testing page under the dropzone in a box labeled "CQL Execution Results". Under the label, high-level metadata will be shown, including:
Artifact: the name of the artifact tested.
Meets Inclusion Criteria: The number of patients that met the inclusion criteria (e.g., "2 of 3 patients").
Meets Exclusion Criteria: The number of patients that met the exclusion criteria (e.g., "1 of 3 patients").
Below the execution results summary, a list of the test patients is displayed.
This lists the five most important CQL result elements for each patient:
MeetsInclusionCriteria: A green checkmark indicates the patient met the inclusion criteria, a red "X" indicates the patient did not meet the inclusion criteria, and the phrase "No Value" indicates that there was not enough data to make a determination.
MeetsExclusionCriteria: A green checkmark indicates the patient met the exclusion criteria, a red "X" indicates the patient did not meet the exclusion criteria, and the phrase "No Value" indicates that there was not enough data to make a determination.
Recommendation: If the CDS indicates a recommendation for this patient, the recommendation text will be displayed, otherwise the phrase "No Value" is displayed.
Rationale: If the CDS indicates a rationale for this patient, the rationale text will be displayed, otherwise the phrase "No Value" is displayed.
Errors: If the CDS returns an error for this patient, the error text will be displayed, otherwise the phrase "No Value" is displayed.
Each patient also has a "View Detailed Results" button allowing detailed test execution results to be viewed for that patient. Clicking on this button brings up the CQL logic for the artifact displayed with the results of executing each CQL statement on the patient.
4.5 Detailed Test Execution Results
After clicking the "View Detailed Results" button for a patient, the CQL for the artifact is displayed with the detailed execution results for each CQL statement.
The detailed execution results popup shows the primary CQL from the artifact with the syntax highlighted. The results of executing each CQL statement in the artifact against the selected patient are displayed preceded with the characters "==>". Results can include boolean values, numerical values, strings, quantities, codes, and FHIR object type and ID references.
The detailed execution results can be closed using the "close" button at the bottom of the popup.