Karsten Wenzlaff, Advisor
August 26th, 2025
Karsten Wenzlaff is a Crowdfunding and Fintech Expert from Germany. He is Secretary-General of Digit[...]
Inside the Feedback Loops Driving AI Failure
AI | Nov 13, 2025
How AI Harm Happens and Why Governance Keeps Missing It
In October 2025, the Center for Security and Emerging Technology (CSET) released a report called, "The Mechanisms of AI Harm", that breaks down how artificial intelligence can cause harm through predictable pathways. The study found that most failures stem not from malicious use but from system design, biased data, and weak oversight.
See: AI Psychosis Threatens Trust in Innovation
CSET warns that as AI systems spread through banking, healthcare, and infrastructure, the harm becomes self-reinforcing. Each model learns from the last, compounding flaws over time. The problem is not bad actors but feedback loops that quietly scale risk. Governance is lagging behind by years, while the pace of model deployment grows monthly.
When Innovation Outruns Accountability
In Detroit, a police facial recognition system wrongly identified Robert Williams, leading to a false arrest that eroded public trust in the technology meant to improve safety.
An MIT article found that error rates for darker-skinned women reached 34% compared with less than 1 % for lighter-skinned men. These disparities reveal structural bias baked into training data and feedback systems.
Across Europe, the EU AI Act now treats such failures as foreseeable harms requiring traceability and documentation.
In the United States, federal regulators have begun enforcement actions against companies using opaque algorithmic decisions.
See: ShadowLeak Shows Zero Click AI Agent Risk
Canada’s proposed Artificial Intelligence and Data Act (AIDA) remains under parliamentary review and was impacted by prorogation in January 2025, leaving gaps between good intentions and enforceability, read the death of AIDA. Policymakers now are debating how AI accountability should evolve.
Many Canadian financial institutions already use machine learning to underwrite credit, detect fraud, and screen clients. Yet few publicly document how models are tested for bias or explainability. The Office of the Superintendent of Financial Institutions (OSFI) Guideline E-23 on Model Risk Management (2027) outlines how financial institutions should govern advanced analytics and AI models, including validation, bias testing, and accountability expectations, but without enforcement the framework still relies on internal discretion.
Regulators acknowledge the need for governance but lack consistent mechanisms to test, benchmark, or verify models across industries. The result is a patchwork system where AI tools can operate with little external visibility. Each institution assumes its safeguards are sufficient, even though no one has a full view of the risk landscape.
CSET’s model shows harm accumulates when oversight is fragmented. A 2024 OSFI-FCAC Risk Report on AI Uses and Risks at Federally Regulated Financial Institutions found that many financial institutions rely on third-party AI systems without full audit access or validation rights, which means models can change without regulators or even clients knowing. These dependencies create silent risk channels within the financial ecosystem.
And in Europe, a Europol “Facing reality? Law enforcement and the challenge of deepfakes” report cautions that deep-fake technology is proliferating rapidly and posing new risks for fraud, misinformation, and market manipulation. Each failure feeds the next, so it creates a feedback loop of risk spreading across jurisdictions and markets.
Why It Matters
Regulators and financial institutions have a limited window to close the governance gap before public trust erodes. The real challenge is not whether to slow innovation, but how to steer it responsibly. CSET’s findings suggest that effective AI governance must protect stakeholders while allowing systems to improve and scale safely. Overregulation can push innovation into unregulated spaces, but weak oversight leaves markets exposed to preventable harm.
See: Take Part in the Global AI in Finance 2030 Survey
CSET’s framework shows that AI harm grows from feedback loops where technical design, human behaviour, and oversight interact. Preventing harm means building governance that evolves at the same pace as the systems it regulates. The question is not whether AI will cause harm, but whether leaders will move fast enough to limit it. Accountability cannot be automated. It must be led.
The National Crowdfunding & Fintech Association (NCFA Canada) is a financial innovation ecosystem that provides education, market intelligence, industry stewardship, networking and funding opportunities and services to thousands of community members and works closely with industry, government, partners and affiliates to create a vibrant and innovative fintech and funding industry in Canada. Decentralized and distributed, NCFA is engaged with global stakeholders and helps incubate projects and investment in fintech, alternative finance, crowdfunding, peer-to-peer finance, payments, digital assets and tokens, artificial intelligence, blockchain, cryptocurrency, regtech, and insurtech sectors. Join Canada's Fintech & Funding Community today FREE! Or become a contributing member and get perks. For more information, please visit: www.ncfacanada.org
 |  |  |
|---|---|---|
 |  |  |
Leave a Reply