The Economic Case for Foundation Model Regulation

The EU AI Act’s foundation model regulation faces criticism motivated by economic concerns around the future of foundation model development in Europe. A deeper look at the EU AI ecosystem and resulting market dynamics reveals that such concerns might be highly misguided - a strong regulatory focus on foundation models would be highly economically beneficial. 

I collaborated with Dominik Hermle on this text.

We were happy to see our thoughts shared and developed e.g. in the EU AI Act Newsletter, in Euractiv, by Yoshua Bengio, and an open letter of leading AI experts to the German government.

Introduction

Approaches to AI regulation are torn between an upstream focus on the providers of powerful foundation models and a downstream focus on the practical deployment of these models’ capacities. Recent discussion around the EU’s AI Act has seen the proposal of an article 28b stipulating extensive regulatory requirements for providers of foundation models. This proposal was met with economic concerns regarding the EU’s global position in the AI space. We argue that these economic objections are misguided, and make two core claims: (a) that the EU AI ecosystem will most likely not feature globally competitive foundation model providers, but will continue to consist mostly of downstream deployers of AI; and that (b) foundation-model focused regulation leads to dramatically fewer regulatory burdens on AI EU players and enables a less constrained and more efficient market. 

EU AI Regulation At a Crossroads

In April 2021, the EU Commission published its proposal for comprehensive regulation of artificial intelligence in Europe. This proposal, the AI Act, seeks to ensure safe and beneficial artificial intelligence in Europe by preventing harms from misuse and unreliability of AI systems while harnessing their economic and social potential. Following extensive negotiation, the EU trilogue is set to finalize the AI Act shortly. 

Regulating Foundation Models

One of the few remaining controversies surrounding the AI Act concerns the regulation of so-called foundation models. Soon after the AI Act was originally suggested in 2021, public and political awareness of recent advances in AI research skyrocketed. This specifically motivated a stronger focus on the cutting edge of AI capabilities, which is driven by foundation models - particularly powerful AI models with a wide range of applications. For instance, ChatGPT is based on OpenAI’s foundational language models GPT-3.5 and GPT-4. Strong expert consensus warned of risks of misuse, or worse, of unreliability and loss of control, from foundation models. In consequence, the European Parliament suggested the addition of an article 28b to the AI Act, introducing requirements and liabilities for foundation models.

The specific details of 28b have since been the meandering subject of negotiations - the common element relevant for our point is that 28b envisions that some of the burden of ensuring safe and legal AI outputs is born by the providers of foundation models, and not only by the deployers ultimately bringing the AI to the customer. For instance, models might have to go pre-publication screening for resistance to misuse or reliability; model providers might be liable for the harms caused by blatantly exploitable security gaps; or providers might be obligated to ensure model outputs cannot violate privacy rights.

The Economic Objection 

The parliament’s suggestion of article 28b has faced strong resistance on economic grounds, led by the national governments of France and Germany. Their predominant concern is that regulating foundation models might endanger nascent European foundation model development, such as by French MistralAI and German Aleph Alpha. Facing contentions that EU foundation models are not yet cutting-edge, EU policy-makers point to growing interest and investment - and hope that EU providers will become globally competitive and strategically important. For instance, Aleph Alpha recently made headlines by securing a $500M funding pledge from a range of German businesses. The objection to article 28b then holds that burdening local AI providers with regulation at a crucial moment of their expansion could prevent the EU from catching up to the US and China. In contrast, established AI players like OpenAI, Google and Meta might benefit from a higher regulatory burden further entrenching their position as market leaders. Concerns are exacerbated by gloomy economic forecasts motivating a renewed focus on drivers of economic growth. Beyond the strictly economic, the EU has, in recent years and following geopolitical crises, strengthened its focus on strategic autonomy - a notion that might drive this most recent push to secure homegrown capabilities in key technologies.  

We believe the argument against foundation model regulation from economic concerns is misguided and threatens to squander the potential of a comprehensive and effective EU AI Act: Even abandoning foundation model regulation entirely would be unlikely to create global cutting-edge foundation model development in the EU. However, adopting foundation model regulation offers significant, but mostly overlooked economic upsides - while also safeguarding against manifold risks. 

EU Foundation Model Aspirations: A Reality Check

Foundation models from EU providers currently are not in a leading position in the international market. Even Europe’s foremost providers and their models - Germany’s Aleph Alpha, France’s Mistral, or, on a broader definition, French-American HuggingFace - lag behind in performance and applications for end-users built on top of them. While GPT-3.5 and GPT-4 record over 100M users per week through ChatGPT alone, it is difficult to find substantial consumer-facing products based on Aleph Alpha’s or Mistral’s models. In the comprehensive HELM evaluation, which considers a range of benchmarks from question answering to text summarization, European models are far behind the state of the art. In its own performance benchmarks, Aleph Alpha only compares their current best public model, Luminous Supreme, to the original release version of GPT-3 from 2020, placing the EU three years behind the curve. Optimists will point out that Mistral’s most recent model compares favourably to Meta’s LLaMa-2; but even that only relates to LLaMa-2’s weakest, 13B parameter version. Already today, EU foundation model development is mostly on the backfoot. And we believe the gap will widen.

To cash in on the economic and strategic potential of AI motivating the opposition to article 28b, a European foundation model provider would have to become a globally competitive large player in their own right. This aspiration is gated by three factors: compute — specialized computational hardware powering model training and continuous operation; data — a large training corpus of text and images that capture a broad range of subjects, tasks, and aspects of life, and talent — capable experts in machine learning and engineering to develop the models.

Lacking Computational Resources and Funding

Firstly, in the modern era of AI development, computational resources prove to be a prohibitive bottleneck for innovation. While some initial performance breakthroughs in deep learning have come from universities, this trend started to dramatically reverse once larger amounts of highly parallel and high throughput compute (like GPUs or TPUs) were needed to achieve better performance. With an ever growing focus on compute, lack of computational resources in the EU could stifle foundation model development. This bitter lesson on the essentiality of compute plays into the hands of the established players that already command large amounts of compute and infrastructure: Google DeepMind; OpenAI and Anthropic and their respective collaborators Microsoft, Alphabet and Amazon; and Meta each have access to some of the most impressive concentration of compute on the planet. From this concentration, running costs are comparatively low, further bolstering established players. 

Catching up would be dramatically costly for European foundation model providers. The recent news of Aleph Alpha securing €500M of investment pledges was touted to be a game changer - but it falls short of comparable investments made in the US by at least an order of magnitude. Operating expenses for a compute cluster to train a model like GPT-4 are estimated above $1B, with models under current development likely exceeding such costs substantially. OpenAI received $10B in funding from Microsoft earlier this year - on top of the already massive compute that enabled OpenAI to develop GPT-4, which to this day outcompetes many European alternatives. Even if the EU and member states sought to bridge this gulf with state subsidies, costs would be enormous and success doubtful: Their ambitious IPCEI Microelectronics II encompasses €8.1B in subsidies - distributed across 56 (!) companies. Even if Aleph Alpha was to receive an entire IPCEI worth of funding on top of its recent investment round, this would still barely match OpenAI’s single latest funding influx. There is little indication that sufficient funds for EU providers to catch up to the global frontier in compute exist. 

Lacking Data and Talent

Similarly, data bottlenecks favor incumbent providers with connections to big tech corporations. To train larger models efficiently, increasing amounts of data are needed. Incumbent tech companies own huge amounts of text data that can be used for training: Meta, for example, trains their models on public Facebook and Instagram posts. StackOverflow, X/Twitter and Reddit all made drastic changes to their API this year, in large part to prevent the free use of their user’s data for training AI models. No European provider-to-be - and even worse, no European stakeholder at all - has access to a comparable wealth of exclusive data. Moreover, having a large user base creates a positive feedback loop where users generate new data, which can then be used to improve the current model, leading to more users. Incumbent providers have such user bases, European providers do not. If constraints and advantages from data continue to matter more and more, the EU hopefuls might thus face yet another substantial obstacle.

Lastly, in comparison, the EU lacks talent. General concerns on the attractiveness of EU countries to exceptional international talent in computer science, engineering etc. apply to the exceedingly demanding challenge of building foundation models in full force. A recent evaluation of AI researchers in Germany clearly demonstrates this trend’s application to AI - even where European countries manage to foster home-grown AI talent, the best and brightest are quick to leave for global hotspots. And when talent remains in the EU, international, non-native tech companies like Amazon or Meta are top employers. This talent gap grows - because the most qualified individuals are motivated to work with qualified experts, on the most exciting and cutting-edge projects. As state of the art models are mostly developed in North America and the UK, EU countries lose even more home-grown talent to these countries.

Can EU Providers Find a Niche?

Faced with these challenges, many voices have instead suggested that European providers’ business case would be to occupy market niches via comparative cost advantages or specialisation on safety and EU-specific (e.g. GDPR) compliance. We believe this to be a realistic path forward for European providers. However, even with specialized models, European providers face obstacles: economies of scale surrounding compute, data and talent as well as the broad, non-specialized capabilities of foundation models make it exceedingly difficult to outcompete the largest providers. Hence, specialized foundation models offered by smaller providers might be unlikely to capture significant market share - case-specific applications or specialized versions of frontier general-purpose models could be more attractive in many cases. The same effect applies to safety and compliance: Due to expertise and scale, large providers are likely to be able to design more comprehensive and reliable guardrails. Gated by these effects, the EU’s foundation model development would be limited to a small suite of somewhat technologically outdated models with potentially higher operating costs. 

While this niche is certainly valuable for some use cases, it is not obvious that it is valuable enough to motivate the elimination of article 28b: Firstly, if EU providers are unlikely to be at the global frontier of performance, it is less obvious how EU foundation models would constitute a strategic technology justifying decisive political intervention on the level of e.g. semiconductors - especially given the substantial costs of 28b’s elimination discussed later. Secondly, it is unclear why such a compliance-focused niche market would even have any reason to be concerned about compliance-focused provider regulation to begin with - instead, article 28b would even seem conducive to the realistic future for EU providers by providing regulatory support for their purported advantages over generalist models. And thirdly, in what is referred to as a ‘tiered approach’, 28b might only apply to larger and more general foundation models; in this case, the European champions and their future business cases would not even be affected by its stipulations whatsoever. At any rate, the niche market argument fails to justify abandoning provider regulation. 

In summary, it is unlikely that European providers can become major, globally competitive players. For that, they lack the talent, the compute, and the data, with no apparent way to close this threefold gap - whether article 28b is eliminated or not. But if they are set to become providers of niche applications instead, they don’t require the protectionist response of eliminating 28b.

The Economic Case for Foundation Model Regulation

But while the EU might not be set to become a major nexus of foundation model development, AI is still set to be an ever-growing major part of the economy. Businesses will have their internal processes powered by AIs, existing software developers will build user-facing applications drawing on the power of foundation models, and start-ups will explore novel ways to harness AI for specific purposes - the common market is set to be rich in downstream deployers of AI. The deployer space is set to be much less consolidated around big players - just as with non-AI software today, a multitude of downstream applications for artificial intelligence exists, many of them specific to countries, languages or even single businesses, and none of them as demanding in terms of talent or compute. No obvious countervailing concetration mechanism as powerful as in foundation model provision comes to mind. So just as there are many European apps, but few European tech giants, there will surely be many downstream European deployers, but few providers. 

Foregoing comprehensive foundation model regulation would pose a grave economic risk for this future of AI use and deployment in Europe. This is because, in a world without foundation model regulation, downstream deployers are set to carry the brunt of the regulatory burden. For many regulatory requirements that article 28b would impose on upstream providers, its removal would instead shoulder the downstream deployers with even more burdensome provisions. Instead of the providers ensuring their model is not trained on protected data, deployers will have to ensure privacy-compliant outputs; instead of providers training models on safe and ethical guidelines, deployers will have to prevent unsafe or harmful output. Legally, this burden shift might play out in two ways: Firstly, the final version of the AI act might explicitly impose responsibilities for preventing the display of potentially harmful or illegal foundation model outputs on the ultimate deployers. Or the deployers, as the user-facing entity, would simply be entirely liable for their applications’ output - including the potential security risks, privacy breaches, etc. passed on from the foundation model they employ. Either way: Deregulation of upstream foundation models increases the de-facto regulatory pressures on downstream deployers massively. Threefold economic potential lies in regulating foundation models and their provders instead: 

Lightening Regulatory Burdens 

Firstly and most obviously, vulnerable deployers face economic peril at the extent of regulatory burdens. As it stands, businesses in Europe already struggle with bureaucratic burdens - the software start-up ecosystem in particular struggles with the resulting hostile environment. Ensuring compliance of an application based on an unregulated foundation model is a large task. At minimum, it will require substantial expertise in navigating a range of requirements, extensive development and stress-testing of filters, and similar. Realistically, it might also require enlisting expensive, sought-after and otherwise unnecessary expertise in machine learning to modify employed foundation models to ensure compliance. Such requirements to set up a comprehensive safety division place a heavy burden on the vulnerable early stages of deployment and threaten to stifle a budding downstream ecosystem in the EU. Serious foundation model providers, even the smallest of which have to command a baseline of funding and relevant expertise far beyond many fledgling deployers, are surely more resilient to such burdens. Plus, recall that the EU is rich in deployers and poor in providers - so even if shifting regulatory burden to providers was a zero-sum game, this would favor the EU’s economy. 

Lowering Compliance Costs

However, secondly, shifting the burden from providers to deployers will likely increase the overall burden, cost and even the possible extent of regulatory compliance. In ensuring safe and compliant model outputs, providers have a range of technical advantages: Some non-compliant output, especially relating to data protection and privacy, can best be addressed when selecting the initial training data - something only providers can do. Furthermore, many promising approaches to ensure safe outputs apply at the AI model’s initial training stage, with further modification of an already-trained model being a distant, less reliable and more costly second choice. And thirdly, foundation model providers are likely to be much more aware of their models’ specific shortcomings, providing them with further advantages in ensuring compliance. Furthermore, deployer-level regulation begets redundancy: If one foundation model services 100 deployers, provider-level regulation requires one central safety effort, while deployer-level regulation requires 100 separate designs and implementation loops for safety measures. Hence, the system-wide burden of compliance is lower in a provider-focused framework - making this focus not only advantageous to the EU, but overall positive-sum. 

Ensuring a Free Marketplace

Thirdly, foundation-model level regulation offers a much more efficient and dynamic AI marketplace. If the responsibility to comply rests with deployers, their solutions for compliance will likely have to be specific to the foundation model they employ, and its respective risks. Since risks and failure modes can vary drastically between different foundation models, it stands to reason that such compliance solutions would not enable ‘plug-and-play’ between different foundation models. And even if they were, plugging and playing might still come with high execution costs, such as when conducting fine-tuning via reinforcement learning. As a result, switching between foundation models would be highly costly for deployers. However, with every element of security compliance already homogeneously built into each foundation model, bespoke compliance solutions shrink and the ensuing costs of switching fall. As a result, the market for foundation models becomes less and less contestable, leading to much more difficult conditions for newcomers and worse terms for deployers and ultimately consumers. This is particularly concerning given already salient concerns of market concentration in foundation models. 

Looking ahead, AI in the EU is likely to be heavy on deployers and light on providers. In that environment, foundation model regulation decreases both local and global regulatory burden and ensures fair and functional market mechanisms. 

Alternatives to Foundation Model Regulation?

Defenders of deployer-focused regulation argue that regulatory burdens on deployers will lead providers to adapt, incentivizing them to provide compliant and safe models. We doubt this claim for two reasons: Firstly, the market might not be responsive enough to make such incentives stick: Path dependencies in choices of foundation models might make switches costly and unlikely, with further lock-in on the horizon as described above. And secondly, safety is likely at odds with maximum functionality - as AI models get more capable, they also become more difficult to safely align with legal desiderata. Whereas the hard requirements of regulation would ensure the prioritization of safety, a soft market incentive might still lead providers to forego compliance in favor of further capability gains. Empirically, it stands that, due to the unreliability of their products, foundation model providers have not been able - or willing - to make reliable assurances about safety and reliability. Existing market concentration of foundation model development with only a handful of providers exacerbates the issue. Hence, market incentives seem unlikely to reduce economic harms from burdens on deployers. 

Lastly, recent debate has seen the proposal of a ‘tiered approach’, in which regulation applies only to the largest foundation models - likely aimed to protect smaller foundation model providers. Overall, such an approach would accommodate many of the concerns above. However, we remain somewhat skeptical: If large, regulated foundation models and small, unregulated foundation models coexist, one of two issues might arise. Either employing an unregulated foundation model places additional regulatory burden on the deployers - in that case, deployers would suffer drastically increased costs from employing the smaller models. It is hard to imagine how these smaller models would then remain competitive, rendering the protection the tiered approach was supposed to afford them ineffective. If employing an unregulated foundation model comes with no additional deployer burdens, however, the AI Act leaves open a conceptual gap - applications built on the smaller models could produce harmful or illegal output with little legal recourse available. A tiered approach is much better than no foundation model regulation, but remains either somewhat inefficient or somewhat unsafe.

Conclusion

Opponents of article 28b’s provider-focused regulation perceive a tension between caution of risks on the side of provider focus and economic potential on the side of deployer focus. However, eliminating article 28b has little economic upside: EU foundation model development is too unlikely to catch up to the global frontier to warrant its protection via deregulation. Instead, the overall European AI ecosystem and its many AI deployers greatly benefit from foundation model regulation: by virtue of less local and global regulatory burden and a stronger and more efficient market. A strong economic case supports regulatory focus on foundation models and their providers.