RE: RIN 0955-AA09 Health Data, Technology, and Interoperability: ASTP/ONC Deregulatory Actions To
Unleash Prosperity

Dear Assistant Secretary Keane:
The Health IT End-Users Alliance (the Alliance) applauds the Assistant Secretary for Technology Policy/Office of the National Coordinator for Health Information Technology (ASTP/ONC) for its review of the Health IT Certification Program (Certification Program) and other ASTP/ONC initiatives seeking opportunities to reduce ecosystem burden and promote innovation. The Alliance supports the intent behind many of the proposals within the Health Data, Technology, and Interoperability: ASTP/ONC Deregulatory Actions to Unleash Prosperity (HTI-5) proposed rule and looks forward to collaborating with ASTP/ONC to refine other proposals to ensure the rule achieves its stated regulatory goals.

The Alliance brings together health information professionals, physicians, hospitals, and other front-line healthcare providers and organizations that use health IT in the provision of care to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use technology tools for care. By working collaboratively across settings of care, the Alliance is focused on advancing end-user perspectives in health IT policy and standards development to support clinical care and operations.

We recognize that HTI-5 as proposed is intended to reduce regulatory burden; however, we are concerned that what is proposed will result in additional burden to the end-user. In effect, this regulatory burden would continue to place end-users in the position of needing to develop or fund technology to ensure they maintain compliance with other regulatory requirements throughout HHS. This additional burden would potentially cause end-users to need to choose between maintaining regulatory compliance and focusing on new innovative solutions to improve patient outcomes.

End-users consistently struggle to find a seat at the health IT development and implementation table despite efforts to engage in these processes. Both policy and technology development often happen before the end-user is involved in the process, making it difficult for end-user pain points to be addressed prior to implementation. This existing burden on end-users increases the cost for technology development, as the developed technology often does not meet its intended goals or policy outcomes once implemented in the end-user environment. The Alliance believes many of the proposals in HTI-5 will exacerbate these ongoing problems and recommends ASTP/ONC refrain from finalizing these proposed changes until a time when end-user concerns, experiences, and needs are accounted for during technology and policy development.

For example, ASTP/ONC proposes to remove the Condition and Maintenance of Certification Real World Testing requirement from the Certification Program. While the Alliance recognizes this is a codification of previously released enforcement discretion, we recommend ASTP/ONC refrain from removing these crucial requirements from the Certification Program. Real-world testing plays a crucial role in ensuring Certified Electronic Health Record Technology (CEHRT) meets the needs of end-users once implemented. Should this requirement be removed, it may cause increased burden on the end-user due to poorly designed technology or technology that does not meet real-world workflows. Other consequences due to lack of adequate real-world testing include:
• adoption and implementation of standards that require significant workarounds by healthcare
organizations;
• adoption and implementation of incomplete or immature standards;
• standards and policies that do not achieve the desired goal when deployed;
• excessive burden added to end-users;
• wasted money on failed implementations; and
• confusion from patients with respect to technological capabilities.

We urge ASTP/ONC to refrain from removing the real-world testing requirement from the Certification Program, and instead strengthen the required use of real-world testing. By working together with end-users earlier, developers and policymakers can ensure innovation is realized in health IT with fewer burdens. Ensuring real-world testing remains a robust requirement in the Maintenance and Conditions of Certification criteria is one way to realize that potential. As part of the Appendix to this letter, we have included our Real-world Testing Consensus Statement that includes further policy recommendations.

ASTP/ONC also proposes to remove the decision support intervention (DSI) transparency requirements from the Certification Program criteria. ASTP/ONC cites a lack of evidence of utility as the reasoning behind removal of the criteria, as well as the desire to reduce burden on developers as they seek compliance with the program. The Alliance recommends ASTP/ONC refrain from removing these criteria without proposing alternative transparency criteria or an alternative regulatory pathway for giving end-users access to transparency information relating to DSI technology. If the DSI transparency requirements are preserved, we recommend ASTP/ONC solicit and publish feedback from end-users on the usefulness of this information to understand the industry’s needs and allow developers to prioritize DSI transparency information that is most useful for end-users.

Artificial intelligence (AI) technology continues to proliferate throughout healthcare at an ever-expanding rate. Such rapid adoption increases the need for end-users to have transparency, trust, and predictability that the technology will deploy successfully. Without trust, it will not only be difficult for end-users to identify technology suitable for adoption, but could also lead to an erosion of trust within the patient- provider relationship. The existing DSI criteria helps narrow the gap for end-users by giving them insight into the data the tools utilized and how decisions were made. It also assists them in making smart purchase decisions to ensure the technology implemented is cost effective and suitable for their environment.

Without the DSI criteria, there will be no regulatory framework in place for health IT relating to AI. If ASTP/ONC were to replace DSI with a new framework – as recommended above – we recommend ASTP/ONC prioritize creating a framework that embodies the following characteristics:
• A risk-based approach to oversight;
• aligns across federal agencies;
• exhibits shared responsibility between developers and end-users in prioritizing safety and
transparency;
• ensures AI does not compromise the security of end-user IT systems and is strengthened against
cyberattacks;
• reduces workflow burden of end-users;
• supports and provides robust end-user education, participation, and leadership; and
• accounts for accessibility and usability in the end-user facing technology.

The Alliance believes preserving the DSI criteria, or pursuing another AI transparency framework, within the Certification Program is important to advancing the use of AI within health IT. Without these provisions in the Certification Program, end-users will be hindered from understanding what they are purchasing and implementing, potentially leading to wasted money in the healthcare system if the implementations fail. The Appendix to this letter contains our Artificial Intelligence Consensus Statement which provides additional recommendations for how ASTP/ONC should approach maintaining an AI and DSI regulatory framework.

The Alliance would also like to commend ASTP/ONC for its work to review and update the information blocking program. Within the proposed rule, ASTP/ONC proposes changes to tighten the ability of actors to exploit exceptions to prevent the transmittal of patient data. We urge ASTP/ONC to ensure that future reviews of the information blocking exceptions account for the actors that rely on exceptions to reflect their information exchange capabilities, while also searching for opportunities to limit the ability for other actors to misuse those exceptions. These proposed changes demonstrate ASTP/ONC’s focus on ensuring an environment exists where patient data flows freely to those who need it most, when they need it while also ensuring the information blocking program is suitable for all actors.

At the same time, the Alliance cautions ASTP/ONC from further altering the information blocking program and its exceptions once HTI-5 is finalized. With a continued state of change, it remains difficult for end- users to build strong compliance and education programs in preparation for future enforcement. Further changes will create additional burden and unpredictability, leading to an enforcement environment where an information blocking actor may be found non-compliant because of a confusing change made to the program. Programmatic consistency is the best way to attain high levels of compliance, and we believe this will be true for the information blocking program in the long-term.

We thank you again for the opportunity to provide our input on this important set of issues and welcome the opportunity to share more about the Alliance and discuss our work further. We look forward to collaborating with ASTP/ONC on issues relating to HTI- 5 and other key interoperability matters.

Jim O’Neill
Deputy Secretary
US Department of Health and Human Services
200 Independence Avenue SW
Washington, DC 20201

Dr. Thomas Keane
Assistant Secretary and National Coordinator
Assistant Secretary for Technology Policy / Office of the National Coordinator for Health Information
Technology
330 C Street NW
Floor 7, Mary E. Switzer Building
Washington, DC 20201

Submitted electronically to www.regulations.gov

RE: HHS Health Sector AI RFI

Dear Deputy Secretary O’Neill and Dr. Keane:

The Health IT End-Users Alliance (the Alliance) appreciates the opportunity to provide input on the Office of
the Deputy Secretary and Assistant Secretary for Technology Policy / Office of the National Coordinator for
Health Information Technology (ASTP/ONC) Request for Information: Accelerating the Adoption and Use of
Artificial Intelligence as Part of Clinical Care, or HHS Health Sector AI RFI, as published in the December
23, 2025, Federal Register (RIN 0955-AA13).

The Alliance brings together health information professionals, physicians, hospitals, and other front-line
healthcare providers and organizations that use health information technology (IT) to ensure that policy
and standards development activities reflect the complex web of clinical and operational challenges facing
those who use such technologies today. By working collaboratively, the Alliance is focused on priorities for
how technology can best support clinical care and operations.

The Alliance appreciates HHS’ interest in creating a policy and regulatory environment that is optimal to
realizing the benefits of AI. AI is rapidly transforming healthcare, with developments and innovations
producing promising non-clinical and clinical benefits in various settings and specialties. Concerns remain
related to how AI models are developed, trained, used, and monitored, and the significant impact AI has on
healthcare operations, patient care delivery, and health outcomes. Due to those concerns, widespread
adoption of AI in healthcare requires thoughtful oversight and governance frameworks to minimize risks
and ensure the appropriate, safe, and ethical use of AI. Health IT end-users are at the forefront of AI use in
healthcare and are well-equipped to collaborate on common principles to ensure the proper balance
between innovation and use of AI with appropriate guardrails.

The Alliance published a consensus statement in April 2025 reflecting the current state of AI in healthcare
including principles intended to guide policymakers as they work to ensure appropriate oversight of AI tools
without hampering innovation. As end-users are often brought into the development cycle for technology
during implementation, the Alliance’s consensus statement highlights the need for end-users to be
engaged throughout the entire AI development lifecycle. The Alliance recommends HHS prioritize
policymaking activities that bring developers, policymakers, and end-users to the table from a
project’s outset to ensure AI tools are built in ways that fit organizations’ unique needs, integrate with
existing workflows, promote trust and confidence in the use of such tools, and inform organization-
specific governance frameworks for end-users.

As HHS pursues policies and regulations to determine the appropriate methods and areas for governance
and oversight of AI in clinical care, the Alliance urges HHS to consider implementing policies that assist the
industry in fostering innovation in AI while maintaining the protection of health data, lowering
administrative burden, and improving health outcomes. Recommendations on specific areas of policy and
regulatory oversight include:

Regulation and oversight of AI. AI tools should augment, not replace, end-users’ expertise, and policy
must ensure these tools supplement cognitive and administrative tasks while preserving human judgment.
AI tools in healthcare require a risk-based approach to oversight where the level of scrutiny and validation
should be proportionally accounted for in policymaking to minimize the disparate harm and consequences
the tool might introduce. The Alliance urges HHS to leverage ASTP/ONC’s statutory coordinating role and its
component agencies, including the Office for Civil Rights (OCR), Centers for Medicare and Medicaid
Services (CMS), and other relevant agencies to ensure policies are aligned across the federal government
to avoid competing and confusing standards that could lead to non-compliance and increased burden.
This includes ensuring HIPAA regulatory frameworks incorporate data handling practices by AI tools to
protect patient privacy and prohibit data/information exchange beyond minimum necessary, secondary
use of data without consent, or data handling practices that may enable unintended or unauthorized data
reidentification.

Safety and transparency. Transparency in the development of AI tools and what data are used in decision-
making, governance, and ongoing testing and maintenance plans is critical. AI developers should provide
clear, understandable information describing how the AI solution makes predictions, tailored in a way to
best suit the needs of end-users. Such transparency requirements should provide a conceptual model on
the importance of data used for AI tool inferences and how data is used. That transparency includes
ensuring patients and healthcare organizations can confidently trust companies will maintain
confidentiality of data. For trust to be maintained, AI developers should disclose how data from patients
and end-users is collected, stored, used, and shared.

Liability. The Alliance encourages HHS to ensure AI companies dealing with health data are held to the
same standards HIPAA-covered entities are held to today. HHS should determine the appropriate balance
of accountability that considers the role of AI developers in the creation, maintenance, and use of clinical
and non-clinical tools and the role of end-users in the use of these tools. HHS should also partner with the
Federal Trade Commission to ensure AI developer oversight includes unfair and deceptive trade practices.
Providers should not be held liable for an AI tool’s performance if they have completed good faith
evaluations and taken steps to mitigate quality or safety concerns. Policies must reflect that developers are
often best positioned to prevent harm due to their knowledge of the development, function, and intended
use of these tools.

Cybersecurity. As AI systems become deeply integrated into clinical and operational environments, their
exposure to sophisticated cyber threats presents an escalating risk to patient safety and organizational
integrity. End-users depend on AI tools that are secure by design, yet they often lack visibility into the
system’s vulnerabilities or control over its defenses. AI developers, not end-users, are best positioned to
mitigate and respond to cyberattacks given their direct access to system architecture, code, and
infrastructure. The Alliance recommends HHS strengthen regulatory guardrails to ensure that developers
bear primary responsibility for securing AI systems against intrusions, data exfiltration, and model
manipulation. Holding end-users liable for breaches or system failures they cannot control would
undermine trust and deter adoption. Robust, enforceable cybersecurity requirements for developers,
aligned with federal health data protections, are essential to protect patients, sustain trust in AI, and
accelerate responsible innovation in healthcare.

Real-world testing. Consistent with the Alliance’s Real-world Testing Consensus Statement, it is critical to
conduct real-world testing of AI tools across a variety of clinical settings to confirm these technologies are
operating as expected without adverse patient consequences. Real-world testing and documentation of
identified issues, including implicit biases, are of critical importance to end-users to advance AI adoption
in healthcare. End-users are often included at the end of the policymaking and technology development
process, not the beginning, which leads to technologies and policies that do not meet their intended goals.
It is crucial for HHS to prioritize policies that include end-users throughout the policymaking and
technology development processes to ensure regulatory and technology innovation goals, as well as the
needs of end-users, are met.

The Alliance is actively working with the National Institute of Standards and Technology (NIST) on its AI
Standards Zero Drafts Pilot Project, an initiative seeking to broaden stakeholder participation in the
creation of AI standards. We applaud NIST’s commitment to working with stakeholders on this effort to
implement directives within the Trump Administration’s AI Action Plan. Thus far, the Alliance has provided
feedback on the four initial topic areas identified by NIST for draft standards documents and the first
outline on testing, evaluation, verification, and validation. We have also held a listening session with NIST
to provide more detailed feedback. We encourage HHS to coordinate with NIST to advance nationwide AI
standards and ensure any AI policy frameworks are harmonized with such standards.

The Health IT End-Users Alliance thanks HHS for the opportunity to provide input on this request for
information. We are committed to being a partner in identifying unnecessary barriers to AI innovation,
development, and implementation while ensuring patient safety and improving the healthcare experience
for all stakeholders involved.

Submitted electronically to www.regulations.gov

RE: Notice of Request for Information; Regulatory Reform on Artificial Intelligence; OSTP-TECH-2025-0067

Dear Director Kratsios:

The Health IT End-Users Alliance (the Alliance) appreciates the opportunity to provide input on the Office of Science and Technology Policy (OSTP) Regulatory Reform on Artificial Intelligence Request for Information (OSTP-TECH-2025-0067), as published in the September 26, 2025, Federal Register.

The Alliance brings together health information professionals, physicians, hospitals, and other front-line healthcare providers and organizations that use health information technology (IT) to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use such technologies today. By working collaboratively, the Alliance is focused on priorities for how technology can best support clinical care and operations.

The Alliance appreciates OSTP’s interest in creating a policy and regulatory environment that is optimal to realizing the benefits of AI. AI is rapidly transforming healthcare, with developments and innovations producing promising non-clinical and clinical benefits in various settings and specialties. Concerns remain related to how AI models are developed, trained, used, and monitored, and the significant impact AI has on healthcare operations, patient care delivery, and health outcomes. Due to those concerns, widespread adoption of AI in healthcare requires thoughtful oversight and governance frameworks to minimize risks and ensure the appropriate, safe, and ethical use of AI to ensure patient safety continues to be prioritized. Health IT end-users are at the forefront of AI use in healthcare and are well-equipped to collaborate on common principles to ensure the proper balance between innovation and use of AI with appropriate guardrails.

The Alliance published a consensus statement in April 2025 reflecting on the current state of AI in healthcare including principles intended to guide policymakers as they work to ensure appropriate oversight of AI tools without hampering innovation. As end-users are often brought into the development cycle for technology during implementation, the Alliance’s consensus statement highlights the need for end-users to be engaged throughout the entire AI development lifecycle. The Alliance recommends OSTP prioritize policymaking activities that bring developers, policymakers, and end-users to the table from a project’s outset to ensure AI tools are built in ways that fit organizations’ unique needs, integrate with existing workflows, promote trust and confidence in the use of such tools, and inform organization-specific governance frameworks for end-users.

As OSTP reviews policies and regulations to determine the appropriate methods and areas for AI governance and oversight, the Alliance urges OSTP to consider preserving policies that assist the industry in fostering innovation in AI while maintaining the protection of health data, lowering administrative burden, and improving health outcomes. Recommendations on specific areas of policy and regulatory oversight include:

Regulation and oversight of AI. AI tools should augment, not replace, end-users’ expertise, and policy must ensure these tools supplement cognitive and administrative tasks while preserving human judgment. Healthcare AI requires a risk-based approach to oversight where the level of scrutiny and validation should be proportionally accounted for in policymaking to minimize the disparate harm and consequences the AI tool might introduce. The Alliance urges OSTP to partner with the US Department of Health and Human Services (HHS), including the Office for Civil Rights (OCR), Centers for Medicare and Medicaid Services (CMS), the Assistant Secretary for Technology Policy/ Office of the National Coordinator for Health IT (ASTP/ONC), and other relevant agencies to ensure policies are aligned across the federal government to avoid competing and confusing standards that could lead to non-compliance and increased burden. This includes ensuring HIPAA regulatory frameworks incorporate data handling practices by AI tools to protect patient privacy and prohibit data/information exchange beyond minimum necessary, secondary use of data without consent, or data handling practices that may enable unintended or unauthorized data reidentification.

Safety and transparency. Transparency on the development of AI tools and what data are used in decision-making, governance, and ongoing testing and maintenance plans is critical. AI developers should provide clear, understandable information describing how the AI solution makes predictions, tailored in a way to best suit the needs of end-users. Such transparency requirements should provide a conceptual model on the importance of data used for AI tool inferences and how data is used. That transparency includes ensuring patients and healthcare organizations can confidently trust companies will maintain confidentiality of data.

For trust to be maintained, AI developers should disclose how data from patients and end-users is collected, stored, used, and shared. To address these needs, the Alliance urges OSTP to preserve the decision support intervention (DSI) transparency requirements within the Health IT Certification Program. Any reduction in DSI transparency requirements of AI technologies reduces end-user trustworthiness in the algorithms used and the technologies themselves and is counterintuitive to the aim of increasing AI adoption.

Liability. The Alliance encourages OSTP to work with HHS to ensure AI companies dealing with health data are held to the same standards as HIPAA-covered entities are today. OSTP and HHS should work together to determine the appropriate balance of accountability that considers the role of AI developers in the creation, maintenance, and use of clinical and non-clinical tools and the role of end-users in the use of these tools. Providers should not be held liable for an AI tool’s performance if they have completed good faith evaluations and taken steps to mitigate quality or safety concerns. Policies must reflect that developers are often best positioned to prevent harm due to their knowledge of the development, function, and intended use of these tools.

Cybersecurity. As AI systems become deeply integrated into clinical and operational environments, their exposure to sophisticated cyber threats presents an escalating risk to patient safety and organizational integrity. End-users depend on AI tools that are secure by design, yet they often lack visibility into the system’s vulnerabilities or control over its defenses. AI developers, not end-users, are best positioned to mitigate and respond to cyberattacks given their direct access to system architecture, code, and infrastructure. The Alliance recommends OSTP strengthen, not weaken, regulatory guardrails to ensure that developers bear primary responsibility for securing AI systems against intrusions, data exfiltration, and model manipulation. Holding end-users liable for breaches or system failures they cannot control would undermine trust and deter adoption. Robust, enforceable cybersecurity requirements for developers, aligned with federal health data protections, are essential to protect patients, sustain trust in AI, and accelerate responsible innovation in healthcare.

Real-world testing. Consistent with the Alliance’s Real-world Testing Consensus Statement, it is critical to conduct real-world testing of AI tools across a variety of clinical settings to confirm these technologies are operating as expected without adverse patient consequences. Real-world testing and documentation of any identified issues, including implicit biases, are of critical importance to end-users in order to achieve the goal of increasing AI adoption in healthcare. End-users are often included at the end of the policymaking and technology development process, not the beginning, which leads to technologies and policies that do not meet their intended goals. It is crucial for OSTP to prioritize policies that include end-users throughout the policymaking and technology development processes to ensure regulatory and technology innovation goals, as well as the needs of end-users, are met.

The Alliance is actively working with the National Institute of Standards and Technology (NIST) on its AI Standards Zero Drafts Pilot Project, an initiative to broaden stakeholder participation in the creation of AI standards. We applaud NIST’s commitment to working with stakeholders on this effort to implement directives within the Trump Administration AI Action Plan. Thus far, the Alliance has provided feedback on the four initial topic areas identified by NIST for draft standards documents and the first outline on testing, evaluation, verification, and validation, and we have also held a listening session with NIST to provide more
detailed feedback. We encourage OSTP to coordinate with NIST as the appropriate entity to create nationwide AI standards and ensure any AI policy frameworks interact well and are harmonized with such standards.

The Health IT End-Users Alliance thanks OSTP for the opportunity to provide input on this request for information. We are committed to being a partner in identifying unnecessary barriers to AI innovation, development, and implementation while ensuring patient safety and improving the healthcare experience for all stakeholders involved.

Submitted electronically to ai-standards@nist.gov

RE: NIST AI Standards “Zero Drafts” Pilot Project: TEVV Outline: Invitation for Input

Dear Acting Director Burkhardt:

The Health IT End-Users Alliance (the Alliance) appreciates the opportunity to provide input on the National Institute of Standards and Technology (NIST) AI Standards “Zero Drafts” Pilot Project: testing, evaluation, verification, and validation (TEVV) outline.

The Alliance brings together health information professionals, physicians, hospitals, and other front-line healthcare providers and organizations that use health information technology (IT) to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use such technologies today. By working collaboratively, the Alliance is focused on priorities for how technology can best support clinical care and operations.
The Alliance was pleased to provide feedback to NIST on the initial topics and scopes for the development of AI standards. The end-user community is well-equipped and has the expertise needed to ensure appropriate design, development, implementation, and post-market surveillance of AI tools, and TEVV methods should include methods to gather feedback from end-users on their experiences. Ensuring there are avenues for end-users to provide feedback will ensure TEVV assessments are well-rounded, accurate, and reflective of an AI tool’s true performance and the impact to patient care and administrative workflows.

As previously shared, the Alliance published a consensus statement in April 2025 reflecting on the current state of AI in healthcare including principles intended to guide policymakers as they work to ensure appropriate oversight of AI tools without hampering innovation. We continue to urge NIST to prioritize end-users and the recommendations put forward within the consensus statement as NIST and other agencies contemplate how to ensure proper oversight of AI while fostering innovation. Having a diverse set of stakeholders, including end-users, at the table will assist NIST in guiding the development of standards that accomplish the AI policy goals of the Trump Administration.

Terminology, Key Terms, and Concepts (Clauses 3 and 4). In defining terminology, end-users must be present to assist in determining comprehensive definitions of the components that are involved with and influence TEVV. Those directly involved with the implementation, use, and monitoring of AI, such as end-users, are best positioned to assist NIST in identifying the components of how AI should be evaluated and what specific considerations should be included in appropriate TEVV. End-users can also assist in identifying what adequate validity and reliability looks like and raise considerations of variances across patient populations to ensure terminology, and in turn TEVV methods, are comprehensive and appropriate for the healthcare settings in which AI tools are used. End-users can provide feedback on how TEVV methods may need to be tailored to specific healthcare organizations and patient populations, as well as the potential limitations of TEVV and how to account for those. In Table 1, end-users can help amplify the current understanding associated with each term and build out examples allowing the eventual development of the standards draft to be comprehensive and reflective of how this information is applicable to the various capabilities of different healthcare organizations. It is also important to ensure end-user perspectives and expertise are incorporated in the TEVV methods in Figure 1, as the different considerations within these methods will vary across both AI tools and healthcare organizations.

Governance (Clause 5). End-user input throughout the development of governance policies and frameworks and monitoring of these policies is paramount. It is critical to ensure that different types of end-users in a healthcare organization and their workstreams are reflected, as organizations have varying resource levels. These differences in resource levels result in a wide range of diverse approaches that organizations use to implement governance strategies, manage personnel, determine internal workstreams, and make resource decisions. We appreciate NIST acknowledging these concerns and other limitations involved with governance and assessment of AI in the outline. As indicated in Figure 2, regulatory and legal requirements and rules can both be drivers and constraints on the use of AI and TEVV. End-users best know the important intersections between quality, safety, reporting, and payment requirements and regulations, and the associated workstreams. By leveraging the roles of end-users and bringing their expertise forward, NIST can ensure healthcare organizations are well informed of how to leverage resources and manage limitations, and the best ways to incorporate TEVV activities into existing workstreams and/or create new workflows in a meaningful, yet minimally burdensome manner.

The Health IT End-Users Alliance thanks NIST for the opportunity to provide input on the TEVV outline and throughout the standards development process. We are committed to being a partner in this effort to create standards that meet the end-user community’s needs and ensure AI is developed, implemented, and used in optimal ways. The Alliance looks forward to hosting NIST for a virtual listening session to expand on our AI principles and recommendations and provide more detailed feedback and substantive comments on the TEVV outline and draft from our member organizations.

Honorable Robert F. Kennedy Jr.
Secretary
US Department of Health and Human Services
200 Independence Ave. SW
Washington, DC 20407

RE: RIN 0938-AV68, Request for Information; Health Technology Ecosystem

Dear Secretary Kennedy:

The Health IT End-Users Alliance (the Alliance) applauds the US Department of Health and Human Services (HHS), the Centers for Medicare & Medicaid Services (CMS), and the Assistant Secretary for Technology Policy/ Office of the National Coordinator for Health Information Technology (ASTP/ONC) for seeking public input on how CMS and ASTP/ONC can best lead infrastructure progress to cultivate the health IT market, increase patient access to digital health tools, and expand the data available for all healthcare stakeholders.

The Alliance brings together health information professionals, physicians, hospitals, and other front-line healthcare providers and organizations that use health IT in the provision of care to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use technology tools for care. By working collaboratively across settings of care, the Alliance is focused on advancing end-user perspectives in health IT policy and standards development to support clinical care and operations.

In response to your health technology ecosystem RFI, we have attached our consensus statements on real-world testing and artificial intelligence, as well as our 2024-2027 collaborative roadmap. These documents illustrate the challenges end-users face in the health IT ecosystem and include a series of policy recommendations on how best to improve existing and future health IT policies to realize the opportunities associated with deploying technologies and standards that will minimize administrative burden, reduce costs, create efficiencies, and result in better outcomes for patients. The locations of the documents submitted for response consideration can be found at the links contained within the appendix of this letter and attached.

We thank you again for the opportunity to provide our input on this important set of issues and welcome the opportunity to share more about the Alliance and discuss our work further. We look forward to collaborating with CMS and ASTP/ONC on issues relating to the health technology ecosystem throughout your tenure as Secretary as we seek to end the nation’s chronic disease crisis by leveraging health technology tools and related technical standards.

Appendix:
• Document 1: Real-world Testing Consensus Statement
• Document 2: Artificial Intelligence Consensus Statement
• Document 3: Health IT End-users Alliance 2024-2027 Collaborative Roadmap

National Institute of Standards and Technology
100 Bureau Drive
Gaithersburg, MD 20899

Submitted electronically to ai-standards@nist.gov

RE: NIST AI Standards “Zero Drafts” Pilot Project: Invitation for Input

To whom it may concern:

The Health IT End-Users Alliance (the Alliance) appreciates the opportunity to provide input on the National Institutes of Standards and Technology (NIST) Artificial Intelligence (AI) Standards Zero Drafts Pilot Project.

The Alliance brings together health information professionals, physicians, hospitals, and other front-line healthcare providers and organizations that use health information technology (IT) to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use such technologies today. By working collaboratively, the Alliance is focused on priorities for how technology can best support clinical care and operations.
The Alliance appreciates NIST’s commitment to broadening participation in the development of standards for the AI community. We agree that the need for AI standards is urgent to keep up with the advancement of AI. The development of such standards requires a wide range of expertise and perspectives to ensure the standards function properly and benefit end-users and their workflows.

AI is rapidly transforming healthcare, with developments and innovations producing promising non-clinical and clinical benefits in various settings and specialties. Still, there are concerns about how AI models are developed, trained, used, and monitored, and the significant impact AI has on healthcare operations and health outcomes. The widespread adoption of AI in healthcare requires thoughtful oversight and governance frameworks to minimize risks and ensure the appropriate, safe, and ethical use of AI. Health IT end-users are at the forefront of AI use in healthcare and are well-equipped to collaborate on common principles to ensure the proper balance between innovation and use of AI with appropriate guardrails.

As healthcare organizations become more comfortable with AI and increase adoption and implementation of such tools, developers and policymakers can work with end-users to address resource constraints and hesitancy in deploying AI tools. By incorporating end-users in the design, testing, implementation, and monitoring of AI tools, AI can be developed and tailored to organizations in ways that fit their unique needs, integrate with existing workflows, promote trust and confidence in the use of such tools, and inform organization-specific governance frameworks for end-users.

The Alliance published a consensus statement in April 2025 reflecting on the current state of AI in healthcare including principles intended to guide policymakers as they work to ensure appropriate oversight of AI tools without hampering innovation. As end-users are often brought into the development cycle for technology during implementation, the Alliance’s consensus statement highlights the need for end-users to be engaged throughout the entire AI development lifecycle. We urge NIST to prioritize end-users and the recommendations put forward within the consensus statement as NIST and other agencies contemplate how to ensure proper oversight of AI while fostering innovation.
A copy of the consensus statement is attached for your convenience. Alliance feedback on the initial topics and scopes proposed by NIST can be found below.

Documentation about system and data characteristics for transparency among AI actors
Potential Scopes: Contents of model, data, and/or system cards; standardized mechanisms and practices for formatting, presenting, sharing, and/or accessing documentation; application of existing documentation practices for securing information technology supply chains to AI systems

Transparency from AI developers on the contents of the model, data used to train the model itself, performance measurements, evaluation data, and intended use are all critical components that can help end-users understand the tools they are working with. AMA surveys of more than 1,000 physicians have found a significant increase in calls for transparency between 2023 and 2024. (https://www.ama-assn.org/system/files/physician-ai-sentiment-report.pdf) The Alliance supports NIST’s inclusion of these components, as well as standardized mechanisms and practices for formatting, sharing, and accessing this documentation. Developers, in addition to policymakers, payers, and healthcare organizations, have a shared responsibility to prioritize transparency and promote end-user trust and confidence that AI tools are safe and effective to use over time and will not result in unintended bias.

Any documentation about system and data characteristics should be tailored in a way to best suit the needs of the end-users. We encourage NIST to collaborate with end-users to determine which transparency information is most helpful to better understand the unique needs of end-users, the settings in which they work, and how to best tailor this information. NIST should also collaborate with other federal agencies to ensure transparency requirements are aligned.

With this information provided in understandable ways and at appropriate levels, end-users can better consider their disclosure and consent processes and determine the best approaches to inform patients how AI is used in their care. Additionally, transparency can serve as a mechanism to clarify liability so that potential issues related to use of AI-enabled technologies can be identified and accountability apportioned appropriately.

Methods and metrics for AI testing, evaluation, verification, and validation (TEVV)
Potential Scopes: Application of well-established TEVV methods to TEVV for generative AI; approaches for translating heterogeneous benchmark scores into meaningful scores or rankings for a given use case; methods for preventing training data from becoming “contaminated” with canonical test outputs and detecting when contamination has occurred

Developing and applying established methods to evaluate AI tools throughout development, implementation, use, and post-market surveillance is critical to ensuring a tool is successful and works as intended. It is important to ensure that AI TEVV methods also include methods to gather qualitative feedback from end-users on their experiences using the tool, in addition to numerical data, to ensure scores and rankings are well-rounded, accurate, and reflective of an AI tool’s true performance and the impact to patient care and administrative workflows. The design, development, implementation, and ongoing surveillance of AI tools, as well as TEVV methods, must include efforts to identify and address biases that surface. Throughout these processes, the end-user community must be engaged in the design, development, implementation, and post-market surveillance of AI tools, including establishing appropriate feedback mechanisms.

Maps of concepts and terminology regarding AI system designs, architectures, processes, and actors
Potential Scopes: Clarification of the “AI stack”; reference architectures or design patterns for AI systems to establish shared understanding of AI system components and their relationships

The Alliance supports the idea of developing and clarifying an “AI stack” for AI applications, meaning the layers of technology and resources used to build AI applications, including the roles, responsibilities, and processes involved in each layer throughout the AI lifecycle. Including end-users and having an understanding of their roles and workstreams in each layer of the AI stack will be crucial to ensuring an accurate full picture of AI systems and will contribute to a greater shared understanding of system components and their relationships. End-users’ roles, responsibilities, and processes are directly impacted by the introduction of any new technology or AI tool and are among the individuals most knowledgeable about how both clinical and non-clinical processes and workflows may change as a result. AI must be designed to be an integrated component of existing workflows and should avoid exacerbating administrative burden by incorporating end-user experiences and feedback into AI stacks, concept maps, and implementation plans.

Technical measures for reducing risks posed by synthetic content
Potential Scopes: A taxonomy of approaches and terms to refer to these approaches; methods and metrics for evaluating and reporting the effectiveness of such measures

Creating a taxonomy of approaches for reducing risks and terms to refer to such approaches is helpful for developers, policymakers, payers, and end-users to fully understand the risk involved with AI tools. Additionally, establishing methods and metrics to evaluate these measures is critical to ensure risk is accurately measured and accounted for. Healthcare AI requires a risk-based approach to oversight, where the level of scrutiny and validation should be proportionally accounted to minimize the disparate harm and consequences the AI tool might introduce. Any method to determine risk and measures to reduce risk should consider and reflect the unique levels of risk associated with various AI tools.

The Health IT End-Users Alliance thanks NIST for the opportunity to provide input on the process and topics chosen for consideration in the pursuit of the creation of standards that meet the end-user community’s needs, and we are committed to being a partner in this effort.

“AI is revolutionizing healthcare delivery and management, demanding a paradigm shift in how we approach technology implementation,” stated Lauren Riplinger, JD, American Health Information Management Association (AHIMA) Chief Public Policy & Impact Officer. “Health IT end-users, whether they be HI professionals or providers, are on the front lines of deploying AI tools and therefore are acutely aware of the opportunities and risks associated with deploying such technologies. AI solutions must be developed and deployed with a focus on safety, transparency, and the experience. The Alliance’s consensus statement is representative of the end-user community coming together to develop a set of common principles to inform ongoing policy discussions within the Administration and the US Congress.

The HITEU Alliance AI Consensus Statement identifies critical areas requiring immediate attention including:

• Clear Regulation & Oversight: Emphasizing that AI should augment, not replace, human expertise, the statement calls for policymakers to establish risk-based regulations that ensure AI tools are rigorously validated, safe, and equitable.
• Transparency & Trust: Advocating for transparency in AI systems, the statement underscores the need for patients and providers to understand how AI operates, how decisions are made, and how data is utilized.
• End-User Participation & Leadership: Highlighting the importance of engaging the health IT end-user community, the statement calls for engaging end-users throughout the entire lifecycle of AI tool development, use, implementation, and post-market surveillance.
• Privacy & Security: Prioritizing data protection and cybersecurity, the statement calls for AI tools to adhere to stringent privacy standards.
• Reducing Administrative Burden: Aiming to enhance efficiency, the statement advocates for AI solutions that simplify workflows rather than adding unnecessary complexity.

“The family medicine experience is based on a deeply personal physician-patient interaction that requires support from technology, including AI,” said Jen Brull, MD, FAAFP, President of the American Academy of Family Physicians. “As a result, family physicians know the importance of responsible and secure AI development, training implementation and monitoring in our health care system. We’ve seen how AI can reduce administrative burdens, allowing more focus on patient care, but also recognize that it cannot replace a physician or our relationships with patients. We continue to partner with policymakers and health care leaders to ensure AI preserves and enhances primary care.”

Representing the voices of health information professionals, physicians, hospitals, and other front-line health care providers and organizations, the Health IT End-Users Alliance, remains steadfast in its commitment to improving healthcare policies and technology standards that reflect the real-world end-user experience.

#####

About the HITEU Alliance

The Health IT End-Users Alliance seeks to advance end-user perspectives in health IT policy and standards development and provides a collective voice to move beyond end-user involvement to end-user leadership. Members include, the American Academy of Family Physicians (AAFP), American College of Physicians (ACP), American Health Information Management Association (AHIMA), , Medical Group Management Association (MGMA), OCHIN and Premier, Inc.

RE: RIN 0955–AA06, Health Data, Technology, and Interoperability: Patient Engagement, Information Sharing and Public Health Interoperability (HTI-2) Proposed Rule

Dear Dr. Tripathi:

The Health IT End-Users (HITEU) Alliance appreciates the opportunity to provide the Assistant Secretary for Technology Policy/Office of the National Coordinator for Health Information Technology (ASTP) with feedback on the Health Data, Technology and Interoperability: Patient Engagement, Information Sharing and Public Health Interoperability (HTI-2) Proposed Rule (RIN 0955-AA06), as published in the August 5 Federal Register.

The HITEU Alliance brings together health information professionals, physicians, hospitals, and other front-line health care providers and organizations that use health IT in the provision of care to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use technology tools for care. By working collaboratively across settings of care, the Health IT End-Users Alliance is focused on priorities for how technology can best support clinical care and operations.^[1]

Our comments are grounded in the HITEU Alliance’s Consensus Statements on Data to Support Equity[2] and Real-World Testing, [3] as well as the Alliance Roadmap[4] that highlights the growing set of federal mandates faced by health IT end-users. Our comments focus on:

• Proposed changes to the information blocking exceptions
• Proposed prior authorization application programming interfaces (APIs)
• Pace of change in regulatory requirements and need for greater end-user input/education

Information Blocking

The HITEU Alliance appreciates and supports ASTP’s proposals to ensure access to care and accommodate requestor preferences. However, we believe that certified health IT should include functionality to support providers in complying with information blocking rules.

Protecting Care Access. The HITEU Alliance supports the proposed addition of a Protecting Care Access Exception, with a request to simplify the conditions under the exception. The exception would allow practices that would otherwise be considered information blocking, based on the actor’s good faith belief that sharing electronic health information (EHI) indicating that any person(s) sought, received, provided, or facilitated the provision or receipt of reproductive health care that was lawful under the circumstances in which it was provided, could result in a risk of potential exposure to legal action for those persons.

This exception would provide needed clarity that physicians and other actors may withhold access, exchange, or use of EHI to protect patients, providers, and others in accordance with the Office of Civil Rights’ (OCR) policies on protecting reproductive health information under the Health Insurance Portability and Accountability Act (HIPAA).

Physicians and other providers are committed to protecting their patients from all forms of harm, yet the lack of clear federal policies on the intersection of reproductive health information and information blocking requirements has placed them in serious ethical dilemmas and at risk of federal penalties.

The HITEU Alliance also requests that ASTP limit the burden of using the exception when acting in good faith. As currently constructed, actors must work through a range of conditions in order to satisfy the exception’s requirements, creating uncertainty and documentation burden.

Requestor Preferences. The HITEU Alliance supports the proposed addition of a Requestor Preferences Exception. This exception is appropriate for circumstances when a patient or other requestor asks for only certain information or asks for a delay in receiving information. For example, physicians and other health care professionals have expressed concerns that some patients do not want to receive life-changing diagnoses or lab results in advance of a clinical encounter, and experience real harm when they do not have appropriate guidance or context for understanding clinical information.

The HITEU Alliance urges ASTP to modify the proposed requirement that the preferences of a patient or other requestor be provided in writing. This requirement is overly restrictive and does not recognize that patient preferences, in particular, may be expressed verbally, such as during a clinical visit. We recommend that ASTP instead require that the preferences be documented by the actor.

Support from Certified Electronic Health Record (EHR) Technology. While the newly proposed exceptions provide additional flexibility for clinicians to protect their patients and other providers from inappropriate legal actions and to respect the preferences of patients and other requestors, we caution that the information blocking rules are generally overly complex, leading to confusion among end-users and conservative behavior by providers and vendors. Furthermore, ASTP has not included certification criteria that would result in EHR functionality that supports the use of these exceptions or limit the need for them by advancing technical tools for segmenting data.

To ease the burden of regulatory compliance through technology, the HITEU Alliance encourages ONC to accelerate efforts to advance digital tools that support data segmentation for privacy. This would allow  physicians and other providers to have more confidence that they can engage in information sharing without revealing sensitive information, including reproductive health data, that could negatively impact patients or lead to noncompliance with local, state, or federal privacy laws. We also recommend that ASTP require certified EHR functionality that supports health care providers in recording patient-level information that is needed for compliance with information blocking, such as patient preferences for the release of data to be withheld or delayed, or specific concerns about preventing harm or protecting privacy.

Prior Authorization APIs

ASTP proposes to adopt standards and certification criteria for Prior Authorization API functionality, which would deploy Fast Healthcare Interoperability Resources (FHIR)-based exchange using a set of implementation guides (IGs) developed by the Health Level Seven International (HL7) Da Vinci Project. These APIs are designed to allow providers to request coverage requirements, submit needed documentation for authorizations, and receive notification back from the payer on authorization status. ASTP proposes to include these functions in the Base EHR definition, which means that eligible hospitals and clinicians must implement them in order to comply with the Medicare Promoting Interoperability Program and Merit-Based Incentive Payment System (MIPS) requirements to use certified EHR technology (CEHRT), with no additional regulatory action by the [5]

The HITEU Alliance is concerned that there is no similar requirement on payers and health plans to adopt certified technology. In fact, CMS does not even require regulated plans use the Da Vinci Project IGs for the prior authorization APIs, but only recommends their use.[6] We urge ASTP and CMS to only require providers to use certified, standards-based APIs for prior authorization if payers are required to also do so, and on the same timeline. Having both parties use conformant, certified technology will increase the likelihood of success.

The HITEU Alliance generally supports solutions to the prior authorization challenges that currently face clinicians and are hopeful that technology can play a key role. In addition to the growing administrative burden of managing a very complex and often manual prior authorization process, providers and others have noted that prior authorization can delay access to care, result in patients abandoning a recommended treatment, and lead to higher out-of-pocket costs.[7]

However, we are concerned that the Da Vinci Project APIs may not have undergone sufficient real-world testing to date. Consistent with our Real-World Testing Consensus Statement, the IGs must undergo timely, real-world testing that provides transparent information to assess maturity and support adoption. The testing should create an understanding of whether the IGs will:

• Be implementable by health care organizations without significant effort beyond the value incurred by adoption;
• Be effective at achieving the desired goal;
• Encompass a complete solution to achieve the desired goal;
• Not result in unintended consequences that would harm individuals (caregivers, patients, physicians, and other clinicians);
• Respect and accommodate the privacy needs of individual patients;
• Not add extraneous work to the care team;
• Ensure sufficient return on investment to justify the health IT spend; and
• Not disparately impact providers who care for communities that are underserved/ marginalized.

It is encouraging that CMS approved a HIPAA exceptions process for voluntary use of the Da Vinci Project prior authorization APIs that included requirements for reporting on the outcome. We are concerned, however, that ASTP is moving ahead with these certification proposals before the findings of the exceptions process have been released to the public, or even referenced in this rulemaking. CMS specifically stated in its approval letter that “[t]he goal of the test is to determine whether FHIR based standards will reduce the cost, complexity, and reduce the burden of the prior authorization transaction” and required a report with findings to be provided to CMS by mid-July 2024.[8]

Pace of change in regulatory requirements and need for greater end-user input and education

As noted in the HITEU Alliance Roadmap, those who use technology tools for care – health IT end-users – face a growing set of federal mandates over the next four years. While each rule addresses important objectives, understanding the complete regulatory landscape highlights implementation challenges, areas of overlap, and even potential conflict.

The HTI-2 proposed rule includes a vast array of regulatory proposals that must be considered within the full context of other regulatory requirements. We applaud ASTP for addressing some areas of regulatory challenge or conflict through policies such as the proposed Protecting Care Access Exception to information blocking. However, many of the proposed new policies will add to existing requirements, including those first introduced in the HTI-1 Final Rule, which was published on January 9, 2024 – barely six months before ASTP released HTI-2 (July 10). In fact, many, if not most, of the provisions in HTI-1 have yet to be implemented (such as adoption of United States Core Data for Interoperability (USCDI) v3 or the Decision Support Interventions, among others).

It is challenging for end-users, and particularly those on the front lines of care delivery, to adequately absorb and assess the proposals in HTI-2 given the many other demands on their time and other regulatory proposals yet to be implemented (HTI-1) or under consideration (such as those contained in CMS payment rules). However, the ASTP rules have a significant impact on the tools available to end-users, their workflows, and their technology costs, thus meriting substantive review. This impact is even more direct given recent CMS rulemaking that leads to automatic alignment of ASTP’s definition of the Base EHR with Medicare’s definition of CEHRT that eligible hospitals and clinicians must use to avoid penalties under the Promoting Interoperability Program (PIP) and meet performance thresholds for MIPS.

The HITEU Alliance recommends that ASTP take a step back and reconsider how the proposals in the HTI-1 and HTI-2 rules fit together, where it makes sense to learn from implementation of HTI-1 provisions before advancing new requirements, and what bandwidth end-users have to update their systems to accommodate these priorities. For example, ASTP has proposed to include in the definition of the Base EHR for providers both a set of prior authorization APIs and a public health API even before payers and public health agencies are required to use certified health IT to communicate with providers.

The HITEU Alliance would be pleased to be a resource and engage in discussions of priorities, timelines, and the need for sub-regulatory guidance and education for physicians and other health care providers that must buy, deploy, and use certified systems to care for patients.

Conclusion

The HITEU Alliance applauds ONC for taking steps to improve the standardization and sharing of health information in support of clinical care. We stand ready to work with ONC to ensure that the end-user perspective is taken into account as the public and private sectors collaborate to make further progress.

[1] https://hitenduser.org

[2] Data to support equity consensus statement (hitenduser.org)

[3] Real-world testing consensus statement (hitenduser.org)

[4] EndUsersAllianceRoadmap-Digital-FINAL.pdf (hitenduser.org)

[5] Calendar Year 2024 Program Requirements | CMS

[6] https://www.cms.gov/priorities/key-initiatives/burden-reduction/interoperability/policies-and-regulations/cms-interoperability-and-prior-authorization-final-rule-cms-0057-f

[7] Consumer Problems with Prior Authorization: Evidence from KFF Survey | KFF and https://www.ama-assn.org/system/files/prior-authorization-survey.pdf.

[8] Da Vinci HIPAA Exception – Da Vinci – Confluence (hl7.org)