Who Regulates the Regulator’s Algorithm?
As supervisors start using AI to watch the firms they regulate, the hardest questions are about the watchers, not the watched.
AI is rapidly changing the financial sector, indeed changing our lives. We see this every day in hallway conversations, in meetings, in conferences, online, everywhere. AI is approving loans, drafting disclosures, and flagging suspicious transactions inside regulated firms. The rules and supervisory practices meant to govern it are still being debated. Writing those governance rules and developing best practices right now is a little like painting the center line on a highway before the concrete has been poured.
The Covid era boosted technology in the financial sector as businesses and consumers embraced everything digital. AI is not putting that rapid transformation into overdrive for the financial sector, creating gaps in performance expecations. These gaps are not hypothetical.
Supervisors across the globe are actively building AI oversight frameworks, and every few months a new capability or failure mode surfaces that the current drafts never imagined. Firms in the middle are being asked to comply with expectations that are, in places, still being written. Supervisors need to focus on governance and outcomes as any attempt to put rules around technologies will result in running in a loop with no end in sight. Further, supervisors will need to use their own AI to deal with the industry’s AI capabilities.
This brings us to a part of the story which is getting too little attention. Who will regulate the regulators as they start using their own AI models?
The watchers reach for the same tools
The reason is simple arithmetic. The volume of model-driven decisions inside a modern bank, the speed at which decisions are made, and the opacity of systems making them have already outrun what supervisors can review by hand. A team of examiners cannot read a million automated lending decisions. Software can. So the direction of travel is clear: regulators will increasingly use AI to supervise AI, and several already are, quietly.
On its face this is sensible, even overdue. Machine review is well suited to exactly the work that defeats human reviewers: sifting enormous volumes of model output for bias or drift, checking algorithmic systems against a control standard, spotting the suspicious pattern that hides across thousands of unremarkable individual decisions. Done well, it could make supervision more consistent, more efficient, and more scalable than any human-only regime ever managed.
The moment a regulator deploys a model to judge a firm, the regulator inherits every problem it spent the last decade telling firms to fix.
A supervisory model carries its own assumptions, its own training data, its own biases, and its own blind spots. Train it mostly on historical enforcement cases from one sector, one geography, or one technological era, and it will faithfully reproduce the priorities of that past. It will look hardest where regulators have already looked, and it may be quietest precisely where the next problem is forming. In most software that is a tolerable limitation. In regulation, where the model’s output can trigger an investigation or a fine, it stops being a technical footnote and becomes a question of legitimacy.
Enforcement needs an explanation
Regulated firms have rights that ordinary software users do not. A firm is entitled to understand the basis of a finding against it, to challenge that finding, and to ask for it to be reviewed. Those rights assume something can be explained.
Picture a supervisory model that flags a lender for bias in its credit decisions. The lender asks the obvious question: on what basis? If the honest answer is that the supervisory tool is itself a black box, the enforcement action is suddenly standing on thin ice. You cannot hold a firm to account on the strength of a conclusion you cannot account for, and explaining their own outputs is the one thing many AI systems are worst at. The regulator would be demanding from the firm a standard of transparency it could not meet itself.
Four questions no one has answered
These are not edge cases to be tidied up later. They sit at the centre of any serious move toward automated supervision, and no jurisdiction has answered them convincingly yet.
Who regulates the regulator’s own models? There is no established independent mechanism for reviewing the supervisory tools that authorities use or are planning to use. Firms are inspected; the inspector’s software, so far, inspects itself.
What is the remedy when the supervisory AI is wrong? Firms already struggle to contest algorithmic judgments made about them. Adding a second AI layer into the chain of oversight makes that harder, not easier, and pushes the burden of disproving a machine onto the supervised.
What happens when the model inherits the bias in its data? Historical enforcement data is not neutral. It records which firms were investigated, which sectors were treated as priorities, and which harms happened to be visible enough to be written down. A model trained on it learns those choices as if they were the shape of risk itself.
And who is liable when an automated supervisory decision causes harm: to a firm, to consumers, to a market? AI liability is unsettled even in ordinary commercial life. In the regulatory context it is almost entirely untouched.
Symmetry is the whole point
None of this is an argument against AI-assisted regulation. That direction is already set, and if used carefully, machine supervision may prove fairer and more thorough than the overstretched human version it replaces. The argument is narrower and harder to dispute: a regulator’s own tools should be held to at least the standard those regulators impose on the firms they supervise. A bank that deployed an unexplainable model to make decisions about customers would be told, correctly, that it had a governance failure. The same logic has to apply when the model belongs to the supervisor and the longer that gap persists, the more it quietly erodes confidence in the whole architecture of oversight, which works only if its judgments can be trusted and tested.
What this means for firms, now
Uncertainty about the rules is not the same as their absence, and it is not an excuse to wait. The direction of travel is clear enough to act on, and tracking it across jurisdictions in real time is precisely the gap we built Sherlocq to close. A few things are worth doing before anyone is forced to.
Know what you are running. Inventory the AI and machine-learning systems in use, where they sit and what decisions they touch, with a named owner for each. If no one can say who owns a model, that is already the failure.
Document as you deploy. Records of data sources, testing, monitoring, and human oversight are becoming the baseline expectation. Evidence that you weighed bias, explainability, and customer impact will matter before any rule formally demands it.
Assume the rules will diverge. Designing for the lowest common denominator and hoping it travels is a bet against the evidence. Build for a common core with regional variations you can switch on as each jurisdiction firms up.
Treat AI readiness like inspection readiness. Firms already rehearse for exams on financial crime, conduct, and financial stability. AI-focused reviews are coming, and they will ask about models, data, governance, and outcomes. Better to have the answers ready than to assemble them under questioning.
This is not work for the technology function alone. Compliance and risk professionals need not become data scientists, but they can no longer stand outside the conversation about how models are built, tested, and challenged. Compliance should understand a model well enough to ask the sharp question, and treat “the output can’t be explained” as a red flag rather than a detail for engineering. Risk should carry model drift, data quality, bias, and explainability failure on the risk register as the operational exposures they are. Boards need plain dashboards and honest narratives about how AI is used and what it produces. Oversight here demands not advanced mathematics but the right questions and truthful answers.
“I don’t understand this model” has stopped being a defensible position in a senior seat. Regulators know it. In time, they will test it.
The regulation of AI will always be a work in progress, because the technology keeps moving faster than the law chasing it. The firms that come through well will not be the ones that waited for perfect rules. They will be the ones that built sound judgment into their governance early, stayed curious about what regulators in different regions were signalling, and treated every new model as a question of accountability rather than just a product decision. The rules are still being drafted. The choices firms make now are what they will be defending later.