Smaller, specialised models are cheaper to run and easier to host on your own infrastructure. For enterprises moving AI from pilot to production, that changes the maths.
Earlier this month, the World Bank urged developing economies to pursue what it calls “small AI”: specialised, affordable systems that work with limited computing power and patchy connectivity. According to The Guardian, the Bank sees real potential in AI built for local languages, infrastructure and needs, rather than relying only on expensive general-purpose technology. The advice was aimed at governments. It applies just as well to a CIO paying per-token rates for a model that spends most of its day sorting invoices. For most of the AI boom, progress has been measured by scale: bigger models, more compute, and the investment and headlines that follow. But as boards start asking what their AI spend has returned, a more practical question is surfacing. How much of that capability do everyday business tasks actually need?
What counts as “small”
Enterprises have run small models for years: fraud scoring, demand forecasting, recommendation engines. What’s new is that small models can now handle language. The term “small AI” covers lightweight predictive models, task-specific applications and small language models (SLMs), and SLMs are the ones getting attention. They have fewer parameters than large language models, need far less compute, and can be fine-tuned for a specific job, so a business pays only for the capability it uses.
Microsoft’s Phi family is a good example. Phi-4-Mini, released in 2025, has 3.8 billion parameters, and Microsoft’s technical report found it matched models roughly twice its size on selected maths reasoning and coding benchmarks. Better training data and model design are closing part of the gap that scale used to cover. IBM is moving in the same direction. Its Granite 4.1 release in April 2026 paired smaller language models with specialised models for speech, vision, retrieval and AI safety, each built for a different part of the enterprise workflow. Size, in other words, is a poor shortcut for choosing a model. What matters is whether it does the job reliably and what it costs to run.
Where the savings are, and where they aren’t
The cost argument gets stronger as AI moves out of pilots. A generative AI tool used now and then by a small team is cheap to run. Embed the same capability in customer service, internal platforms and thousands of daily transactions, and the bill looks very different. Every request uses compute, whether it is charged per token, through dedicated infrastructure or on hardware the enterprise runs itself. At volume, the gap between a large general-purpose model and a smaller one adds up quickly. Many of those requests are routine. A bank processing thousands of standard documents does not need frontier-level reasoning to identify a document type, pull out standard fields or route an application for review. An IT department can use a specialised model to sort support tickets and flag common requests, and save the more capable system for difficult troubleshooting. Cheaper compute isn’t the whole bill, though. Integration, security and ongoing monitoring all cost money, as does the work of getting accuracy to an acceptable level. A cheap model that regularly gets answers wrong will end up costing more than it saves.
Smaller models also give enterprises more say over where AI runs. Some can run on local servers, private infrastructure or edge devices without constant calls to an external cloud. Manufacturers can put inspection and monitoring systems right next to production equipment. Banks can keep certain sensitive workloads in-house. Businesses spread across many sites can keep applications running when connectivity drops. This is where the World Bank’s argument comes closest to enterprise concerns. Its World Development Report 2026 argues that AI pays off when it is adapted to existing infrastructure and local requirements. For multinationals operating in markets with uneven digital infrastructure, a system built around constant connectivity and heavy cloud use will not suit every location. Running models locally still demands proper cybersecurity, data governance and compliance, but it gives organisations real control over where their data is processed.
One model, or several
Early enterprise adoption usually started with a powerful foundation model and a search for things to use it on. As deployments mature, the better starting point is the workload. A routine information request, a complex financial analysis and a high-risk compliance decision have very different requirements, and running all three through the same model is convenient but rarely efficient. That is pushing more organisations towards hybrid, multi-model setups. A smaller model handles straightforward customer queries, retrieves information or classifies incoming requests, and anything harder goes to a larger model or a human specialist. Done well, this keeps response times and running costs down. It can also reduce dependence on a single model provider, provided the systems are built to work together. For CIOs and CTOs, this turns model selection into an infrastructure decision. How a model scores on its own matters less than how it connects to enterprise data, existing applications, security controls and monitoring.
Testing on your data, not benchmarks
Standard benchmarks are useful for comparison, but they can’t show whether a model will hold up inside a particular business. A small model that scores well on public tests may struggle with company-specific terminology, complex instructions or exceptions in a process. It needs to be tested on representative business scenarios, with clear thresholds for when it hands over to a human. In regulated industries, where a wrong output can carry legal or financial consequences, that step is essential. Small models have real limits. They are weaker at complex, multi-step reasoning and unfamiliar subject matter, and their performance drops outside the domains they were tuned for. So the aim is to combine systems based on what each task measurably needs, rather than to swap every large model for a small one.
Choosing the right size
Frontier models still matter. Complex research and advanced reasoning will keep benefiting from scale. But most of the systems that keep a business running have more modest needs. For enterprises, the hard part has moved on from proving AI can do a task to proving it can do that task reliably, at a cost the business can sustain. Smaller models make that easier to achieve across specific functions and environments. For digital leaders, the work now is choosing the right mix of models, measuring how each one performs and fitting them into existing operations. A useful first step: pull the logs for your highest-volume AI workload and count how many of those requests genuinely needed a frontier model.
Source Intelligence Layer: 1 | 2 | 3 | 4
Follow the SPIN IDG WhatsApp Channel for updates across the Smart Pakistan Insights Network covering all of Pakistan’s technology ecosystem.
