Why Canadian Financial Institutions Are Moving AI Workloads Off Hyperscale Cloud
Banks, insurers, and investment managers that moved aggressively to public cloud are now confronting the compliance, cost, and data sovereignty limitations of running AI at scale on hyperscale infrastructure. A growing number are rearchitecting their approach around certified Canadian infrastructure.
The cloud migration wave that swept through Canadian financial services between 2018 and 2023 was, in hindsight, a largely infrastructure migration — a move from on-premises hardware to hyperscale cloud compute and storage that delivered real operational benefits in flexibility, scalability, and cost predictability for general-purpose workloads.
AI is a different kind of workload. And the assumptions that made hyperscale cloud attractive for general enterprise IT do not translate cleanly to AI deployments that process sensitive financial data at the scale that modern use cases require. A growing number of Canadian financial institutions are confronting that mismatch — and a meaningful subset are responding by moving AI workloads off hyperscale cloud and onto certified, domestically-operated infrastructure.
The Compliance Friction Here
The Office of the Superintendent of Financial Institutions (OSFI) has been clear that federally regulated financial institutions remain responsible for the protection of client data regardless of where it is processed. OSFI’s technology and cyber risk guidance, and the cloud-specific guidance that followed, established that third-party cloud arrangements do not transfer an institution’s risk management obligations.
For AI workloads that process client financial data — fraud detection systems, credit risk models, client intelligence platforms, regulatory reporting automation — the compliance review required before deployment on US-based hyperscale infrastructure is substantial. Legal review, privacy impact assessments, vendor security assessments, cross-border data transfer agreements, and ongoing audit obligations add time and cost to deployments that, on domestic infrastructure, face a materially simpler compliance path.
IBM maintains a strong enterprise presence across Canada’s major financial institutions, managing hybrid environments for several of the Big Five banks — a data point that illustrates both the appetite for AI in financial services and the preference for infrastructure arrangements that keep accountability within a Canadian governance framework.
"The cloud economics argument for AI looks different once you account for the compliance overhead. On domestic certified infrastructure, that overhead drops substantially — and the per-token cost model disappears entirely."
The Cost Argument
Compliance friction is one driver. The economics of AI at scale are another. Financial institutions running production AI workloads — particularly those involving large language models, retrieval-augmented generation pipelines, or agentic AI systems that generate multiple LLM calls per task — are encountering per-token billing costs that scale non-linearly with usage.
At pilot scale, per-token costs are manageable and often justified by the speed to deployment that commercial API services enable. At production scale — where an AI-assisted client intelligence platform might generate millions of inference calls per day, or an agentic compliance monitoring system runs continuous analysis across large document sets — the cost profile changes materially.
On privately operated GPU infrastructure, the economics invert. The capital or operational cost of the infrastructure is fixed; the marginal cost of additional inference is, beyond power and cooling, effectively zero. For high-volume AI workloads, the break-even point between hyperscale API costs and owned infrastructure costs is arriving faster than most financial technology leaders anticipated when they began their AI programmes.
What the Transition Looks Like in Practice
The transition away from hyperscale cloud for AI workloads is not, in most cases, a wholesale cloud exit. Financial institutions are retaining hyperscale cloud for general-purpose workloads where it delivers genuine value — development environments, non-sensitive analytics, collaboration tooling, and workloads where data sensitivity does not trigger the compliance overhead that makes hyperscale AI deployment complicated.
What is changing is the infrastructure layer for production AI workloads that process sensitive client data. Those workloads are moving to certified, domestically operated GPU infrastructure — either purpose-built managed AI infrastructure from providers that offer the full stack from colocation through to managed AI operations, or hybrid arrangements where the compute layer is domestic and the AI deployment and operations layer is provided by a specialist managed services partner.
The market for certified Canadian AI infrastructure capable of meeting the requirements of regulated financial institutions is still developing. But the demand signal from the sector is clear — and the providers positioning themselves to serve it are doing so now, before the demand-supply balance tightens further.