From Black-Box Clinical AI to Algorithmovigilance: A Minimum-Viable Governance Protocol for Safety, Accountability, and Prevention of Algorithmic Harm

Artificial intelligence (AI) is increasingly embedded in routine care through imaging platforms, EHR-integrated prediction tools, triage systems, and clinical decision support. In many deployments, clinical AI remains effectively black-box: its internal reasoning is opaque (because of model complexity, proprietary restrictions, or both), and its outputs can influence care pathways in ways that are difficult to anticipate or reconstruct without deliberate logging and oversight. A clear operational gap persists: health systems often adopt AI as if it were a static product, whereas safety infrastructures are built for technologies whose performance is more stable and whose failure modes are more familiar. Accountability for post-deployment surveillance is frequently fragmented across IT, clinical leadership, vendors, and risk management. Governance work on accountability and safety has emphasized the need to define responsibilities across this socio-technical stack. Moreover, black-box AI can contribute to what some authors and communities describe as “algorithmic violence”: systematic, preventable harms disproportionately borne by already underserved groups, arising from biased proxies, inequitable data, or workflow-mediated feedback loops. Empirical studies show that clinical algorithms and models can encode and amplify inequities, for example by using cost as a proxy for need or by selectively underdiagnosing underserved populations. For health systems, the key governance question is not whether bias exists in principle, but whether inequitable impact can be detected early, investigated rigorously, and corrected reliably. Figure 1 summarizes this lifecycle governance pathway, highlighting the points at which risk qualification, proportional safeguards, and continuous algorithmovigilance can prevent diffuse accountability and mitigate algorithmic harm. [...]