AI Infrastructure

Summary

The following is a summary of a video recording and may contain errors in spelling or grammar. Although IBKR has edited for clarity no material changes have been made.

This is Frank Downing, Research Director at ARK, focused on AI cloud computing and software. And today we’re going to go over our AI infrastructure section of our Big Ideas 2026 report.

So, jumping right into what we see in the current state of the AI market is a continued explosion in demand. When we look at for example the tokens inferenced on open router over the last year since December of 2024 they’ve grown over 25-fold and what’s really driving this is a reduction in cost and an increase in performance of the leading models, combined with models finally being built into products that we use in our everyday life both on our time spent as a consumer using AI in our day‑to‑day and AI getting rolled into the workplace both through kind of general subscriptions like ChatGPT for enterprise or Anthropic’s cloud product and through kind of vertical specific applications and we talk through all of that in detail in our AI productivity section.

Focusing on cost declines for a second, this is something that’s really important and foundational in our research at ARK and it helps us understand when a technology is ready for prime time and will grow to meet more and more use cases over time. We use this for example in studying electric vehicles in the past to know that the cost declines in electric batteries would allow electric vehicles not only to be possible but profitable at scale. And we’re seeing these same dynamics play out in the AI space. There’s a lot of benchmarks out there. If you look at artificial analysis which does an index of many of the leading benchmarks, the cost to achieve a certain level of intelligence on that benchmark has fallen by 99% over the last year.

And we see this across many different benchmarks., which again we speak to in in in other sections of this deck, but this kind of evokes this concept of Javens Paradox, where a decline in cost actually increases the market size and increases the demand because you’re unlocking new use cases. We see that in domains like software development for example whereas coding models have become more intelligent relative to the price you’re paying, software developers are leaning in harder and harder and using and consuming more tokens than they were in prior years.

What is all that doing? It’s driving a huge investment in the underlying infrastructure required to run generative AI. We’ve seen the long‑term trend of data center system spending which was growing at a 5% annual rate over the last 10 years prior to the launch of ChatGPT. It was growing slowly from $150 billion to about $200 billion per year. That has inflected upwards and since the launch of ChatGPT accelerated to grow at a 29% annual rate hitting 500 billion nearly $500 billion in 2025 and the current market estimates are for that to grow to nearly $600 billion in 2026. If you’re wondering what data center systems are, these are the compute servers, networking to connect them and storage that’s attached to them that are put into data centers. So really the core IT equipment driving these workloads, it doesn’t include which is a whole another big expense, the data center actual facility and power that gets connected and enabled to power all those IT systems.

I think this is where we spent a good amount of time in our research this year which is adding historical perspective on where we are in the cycle. There’s a lot of fears of being in an AI bubble in the market today and we wanted to shed some light on that. Looking at the left side of this chart, we looked at how big is the scale of this investment relative to other times in history. The last time we saw a Capex cycle like this was the tech and telecom bubble of the late 90s and early 2000s, and those numbers I mentioned on the past slide are part of the reason why this ratio of the capex relative to GDP for technology companies has been growing over time and is now reaching levels that haven’t been seen since that late ’90s period. Actually, I think what’s interesting about this is if you look since the previous times this chart bottomed both after the dotcom bubble burst in around 2002, and then after the great financial crisis in 2008 – 2009, we’ve actually seen a consistent rise of tech capex as a percentage of GDP over time.

This actually intuitively makes sense if you think of compared to the year the iPhone was launched versus now, how much more technology is represented in our day‑to‑day lives and how much bigger these companies just are large cap tech today relative to their size a decade and a half ago.

On the right-side chart, I think this is helpful on the market cycle context of answering that question of where are valuations and are we in a bubble? Is the market multiple higher than it was post covid for example when the market bottomed? Yes. The S&P 500 has gone from a PE multiple of around 20 to around 30. So, it’s definitely higher though off what I consider an overly pessimistic base judging by how the market has performed since then.  If you look at large cap technology companies specifically, we’ve highlighted where the current Mag 6 are, as of the beginning of 2026, which is around a market multiple of 40 and comparing that to what large cap tech the Cisco, Oracles the IBMs, Microsoft is the only company that’s in both of these lists where they were through the late 90s period they actually peaked at over a hundred times earnings. So, if you looked at where they hit 40, that was in about 1997. So, multiples are elevated but partially because these companies are so profitable already and are funding this buildout with a larger percentage of free cash flow than companies in the 1990s and because the actual returns on AI are real. There is real revenue flowing into the cloud divisions of these businesses in particular. We see multiples healthier, and this would imply there’s a long way to go before we reach the same euphoria that was seen in 1999 and 2000 for example.

So, in the next slide here we look at what’s evolving in the heart of AI compute.  These are the chip designers that are that are making the hardware that makes it possible to run generative AI at scale. Of course, what we’ve seen since well since they started building data center chips between 2012 and 2014 but really since launch of ChatGPT, NVIDIA has been the star of the show.

Now three years into the generative AI revolution we’re seeing the beginnings of a broadening out of the market for compute providers, and a really compelling story is AMD who’s coming to market with more competitive chips having competed against Nvidia in the gaming space on the consumer side and has competed against Intel very successfully in the data center side. AMD went from almost 0% market share in 2017 to 40% today in data center CPUs. And we think there’s a similar share gain story possible for AMD in the data center space with regards to GPUs. They’ve won customers like OpenAI and Meta for example and are looking to extend that list as they have new chips that come to market and are even more competitive relative to Nvidia.

On the research side we looked at performance using a variety of semi‑analysis benchmarks of AMD versus Nvidia on small models versus large models. What you can see here on small models relative to the cost of the chips, AMD has already caught up and is actually more performant on a performance per dollar basis or the amount of tokens you could generate per dollar for the same model than Nvidia is. On the large model side, however, you could see where Nvidia still has this huge advantage, their rack-scale solution Grace Blackwell can deliver 15.5 million tokens per dollar of total cost of ownership compared to the current stats for AMD’s top‑of‑the‑line which is about 2.9 million tokens.

So, this rack scale offering really is an advantage for Nvidia in the market today. But what we’re going to see through the rest of 2026, which is important for forecasting where we’re going to go, is that AMD is coming to market with their own rack scale solution called Helios that they expect to go toe‑to‑toe with Nvidia’s next generation Vera Rubin. So, we’re excited to see when these products both come out in the second half of this year to see where these performance numbers go, but what we can see from the customer orders is early indications are looking like it’s going to be a compelling product.

Not in the chart because the benchmark data hasn’t come out for it yet but in the table is Google’s TPU which is another important part of what’s happening in the AI compute market which is the maturing and the scaling of custom silicon projects within the hyperscalers. So, all of Nvidia’s biggest customers actually have different efforts to build their own AI chip. Google’s TPU, Amazon’s Tranium and Microsoft’s Maya series. Microsoft has really not gotten a good shot on goal yet. Trainium from Amazon would be the next most developed and Google having worked on the TPU for over 10 years is definitely the most mature in the market. They’re running all of their internal Gemini workloads on TPU and trained their latest model Gemini 3 on the TPU and it is one of the most capable frontier models out there.

So, we’re seeing increased investment by Google on their TPU stack and that’s benefiting companies like Broadcom that are the back‑end design provider or basically the silicon partner for Google and really like the go‑between between Google and TSMC to help them scale that product.

So, if we take all that into context and we look out through the rest of this decade, we are again at around 500 billion in annual data center system spending now. We think with the scaling of AI applications and the continued cost declines we’re seeing that total investment could nearly triple to 1.4 trillion by 2030.

And again a large part of that being spent on compute and the composition of that compute market will change pretty significantly where we have already the change in market share from traditional CPU driven computing to accelerated computing powered by GPUs and these custom silicon projects or ASICs application specific chips developed by the hyperscalers they’re already taking a majority share of incremental compute spend. We think within that custom silicon and could grow to a third or higher of spend as some of these projects like the TPU and Tranium continue to scale.

So, all this is setting up for what we think is a sustainable infrastructure buildout. There’s a chance that we go too crazy, and the market overheats between now and then, but all of our research on how AI is being used. When you think about it, it is really early in the deployment cycle relative to how we integrated we think it can be into our daily lives especially at work where enterprises have paused and then now are scrambling to try and figure out how can I best use AI. We think every business just like they became common users of the internet will become common users of AI and the businesses that become power users will be the ones that lead, really become the next leadership in the market. So, with that that’s a brief rundown of our AI infrastructure section of Big Ideas 2026. We’re really excited about the research this year and the insight it’s giving us into the market and we’re excited to keep on learning and following innovation.