CUDA, Nvidia’s Real Moat

Nov 14, 2026 · @Michele Laurelli

Introduction

Nvidia closed fiscal year 2026 with $215.9 billion in revenue, $193.7 billion of it from the data center, a gross margin of 71.1% and a market capitalization that in mid-November still hovers around $4.4 trillion, despite the turbulence of recent weeks. Estimates of its share of the artificial intelligence accelerator market range between 75 and 80% of revenue, depending on how you count the silicon that hyperscalers design for internal use.

data_center_nvidia

Numbers of this magnitude are usually attributed to the quality of the chips. It’s an understandable reading, but an incomplete one. AMD, Google and a dozen startups have spent years producing silicon that looked competitive on paper without managing to dent that share, and the explanation lies in a software platform that Nvidia introduced in 2006 and called CUDA. Over twenty years CUDA has grown from an extension of the C language into an ecosystem of compilers, runtimes, more than a thousand libraries, profiling tools, academic communities and professional habits. The thesis of this article is that Nvidia’s market power comes from there far more than from the transistors.

The pages that follow explain how CUDA works on a technical level, how a bet that lost money for a long time became a standard, and which economic mechanisms made it so hard to replace. The last part tackles the question of these weeks. Anthropic told investors that its Mythos model got the inference of one of its own models running on AMD GPUs in about three days instead of the usual six to twelve months, and it did so in the same days in which Jensen Huang was stepping down from the helm of the company after thirty-three years. To understand how much the moat is really at risk, we need to distinguish carefully between what AI agents make easy and what remains hard. The conclusion, which I’ll anticipate here, is that CUDA is losing its function as a barrier before it loses its function as a standard.

What CUDA Is, From a Technical Point of View

CUDA (Compute Unified Device Architecture) is, all at once, a programming model, a set of extensions to the C and C++ languages, a compiler, a runtime, a driver and a hardware architecture designed to be programmed in that way. This overlap is the first thing to understand, because it is precisely the coherence between the layers that makes CUDA something more than an API.

anatomia_cuda

The figure works as a map for the rest of the article. The language, which is the layer everyone thinks of, is also the one that protects the least, while the advantage is concentrated in the libraries that sit on top and in the system integration that sits underneath.

The SIMT Model and the Execution Hierarchy

The programmer writes a function, called a kernel, that describes the work of a single thread, and then launches it on a grid of thousands or millions of threads. Threads are organized into blocks, and blocks form the grid. Each thread knows its own coordinates (threadIdx, blockIdx, blockDim) and uses them to decide which portion of the data to work on. A minimal example, the SAXPY operation, gives the idea.

__global__ void saxpy(int n, float a, const float* x, float* y) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < n) y[i] = a * x[i] + y[i];
}

// lancio: blocchi da 256 thread, quanti ne servono per coprire n elementi
saxpy<<<(n + 255) / 256, 256>>>(n, 2.0f, d_x, d_y);

In hardware, each block is assigned to a Streaming Multiprocessor (SM), and inside the SM threads run in groups of 32 called warps. All the threads in a warp execute the same instruction in the same cycle, on different data, and this is what Nvidia calls SIMT, Single Instruction Multiple Threads. When an if sends some threads of the warp down one branch and others down the other, the two branches run in sequence with the uninvolved threads masked off. This phenomenon, known as divergence, halves efficiency or worse. The number 32 will come back into play when we talk about porting to AMD, where the equivalent unit, the wavefront, is historically 64 threads on the CDNA architectures for data centers.

The Memory Hierarchy

The real difficulty of GPU programming has to do with memory more than with computation. Each thread has private registers, the threads of a block share a programmable on-chip shared memory with very low latency, above that sit the L1 and L2 caches, and finally global memory, today HBM with bandwidth on the order of terabytes per second but latency of hundreds of cycles. A fast kernel is almost always one that reads global memory in a coalesced way (contiguous threads reading contiguous addresses, so that the controller merges the requests into a few wide transactions), that reuses the data loaded into shared memory for as many operations as possible, and that keeps enough warps active to hide latency while others wait for data. Occupancy, the ratio between active warps and the maximum warps per SM, depends on the kernel’s register and shared memory usage, and balancing it is an art that requires knowledge of the specific chip.

Tensor Cores and Specialized Instructions

Since 2017, with the Volta architecture, Nvidia GPUs have included Tensor Cores, units that perform a multiply-accumulate on small reduced-precision matrices in a single step. Since then almost all the useful compute for AI has gone through them, and the way to access them has changed with every generation: from the wmma APIs and the mma.sync instructions of Volta and Ampere, to the asynchronous wgmma of Hopper with the Tensor Memory Accelerator (TMA) for tile copies, up to the tcgen05 instructions and the dedicated Tensor Memory of Blackwell. Each jump requires rewritten kernels to reach peak performance, and this is where Nvidia’s libraries, updated ahead of the chip launches, make the difference.

The Compilation Chain and the Role of PTX

The nvcc compiler splits the host code, which goes through the regular C++ compiler, from the device code, which is lowered to NVVM (an LLVM dialect) and translated into PTX, a publicly documented virtual instruction set. PTX is then assembled by ptxas into the real machine code of each specific architecture, called SASS, which is not documented. Executables contain a fatbinary with SASS for the known architectures and PTX as a fallback, and the driver can compile that PTX just-in-time for GPUs released after the program. This mechanism, together with the concept of compute capability, which numbers the features of each generation, has guaranteed for twenty years that code written for a 2007 GeForce keeps running on later chips. Nvidia claims it now runs a million times faster on Vera Rubin. Backward and forward compatibility is one of the least visible and most valuable assets of the whole ecosystem.

The tools around the language

Around the core there are streams and events to overlap computation and transfers, CUDA Graphs to cut launch overhead by capturing sequences of kernels, unified memory, direct GPU-to-GPU communication, and a suite of tools (Nsight Systems, Nsight Compute, compute-sanitizer, cuda-gdb) that lets you see hardware counters, warp stalls, shared memory bank conflicts and Tensor Core utilization. Anyone who has optimized a kernel knows that without these tools you are working blind, and for years their maturity was one of the sharpest gaps between Nvidia and its competitors.

The origins and the economic bet (2006-2012)

In the early 2000s, some researchers had realized that GPUs, built to compute the color of millions of pixels in parallel, could be bent toward scientific computing. But they did it by disguising data as textures and calculations as graphics shaders in OpenGL or DirectX, an acrobatic and fragile job that came to be called GPGPU. At Stanford, in Pat Hanrahan’s group, Ian Buck developed Brook, a stream language that hid part of this acrobatics. In 2004 Buck joined Nvidia, where he worked with John Nickolls on what would become CUDA.

The launch came in November 2006, together with the GeForce 8800 GTX, based on the G80 chip, the first Nvidia architecture with unified shaders, meaning general-purpose compute units instead of units specialized for vertices and pixels. The public SDK arrived in February 2007, followed by the first Tesla card dedicated to computing. The coincidence of hardware and software is essential, because the G80 was designed with CUDA in mind too, with a programmer-addressable on-chip shared memory that earlier graphics architectures did not offer.

A platform on every card sold

The strategic decision that sets Nvidia apart from every later attempt was to make CUDA available on every GPU, including gaming cards costing a few hundred dollars, instead of reserving it for an expensive professional line. This meant adding silicon area, control logic and software engineering work to products sold to an audience that, for the most part, would never use those features. For a few years the investment weighed on margins with no visible revenue, and the 2008 crisis pushed Nvidia’s market capitalization down to a few billion dollars, with analysts and investors seeing general-purpose computing as a distraction from the core graphics business.

The result of that choice, however, was a huge and pervasive installed base. Every PhD student with a gaming PC had a CUDA machine at home, and every university lab could buy a consumer card to experiment without going through a purchase order for specialized hardware. This lowering of the barrier to entry is the first ingredient of the dominance that followed.

Cultivating the community before the market

In those same years Nvidia invested systematically in education. CUDA Centers of Excellence were set up in universities, starting with the University of Illinois. The book by David Kirk and Wen-mei Hwu on massively parallel programming became the reference text for courses. The company handed out free GPUs to research groups, funded online courses and, from 2009, organized the GPU Technology Conference. With the Fermi architecture of 2010, which introduced ECC memory, a cache hierarchy, efficient double precision and more complete C++ support, Nvidia entered high-performance computing in earnest, and that same year the Chinese supercomputer Tianhe-1A reached first place on the TOP500 thanks to Tesla GPUs.

At that time scientific computing (molecular dynamics, fluid dynamics, quantitative finance, seismic imaging) was a niche market that did not justify the expense. But it helped build what would be needed later: thousands of people who knew how to program GPUs, a scientific literature that took CUDA for granted, and a set of numerical libraries that were already mature when deep learning arrived.

AlexNet and the birth of the deep learning stack

In 2012 Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton won the ImageNet competition with a convolutional network trained on two 3 GB GeForce GTX 580 cards, with a top-5 error of 15.3% against 26.2% for the runner-up. The code, cuda-convnet, was written by hand in CUDA by Krizhevsky. That victory is commonly cited as the start of the deep learning era, and it is worth noting that it happened on gaming cards, exactly the hardware that the 2006 bet had made programmable and widespread.

Nvidia quickly understood what was happening and made the move that cemented its lead. In 2014 it released cuDNN, a library of optimized primitives for neural networks (convolutions, pooling, normalizations, activations, later recurrent cells and attention), which the frameworks of the time, such as Caffe, Theano and Torch, adopted almost immediately. The economic meaning of cuDNN runs deep, because it moved the cost of writing and maintaining the most critical kernels from every single research group to Nvidia. A researcher who wanted to try a new architecture no longer had to become an expert in shared memory and coalescing. Calling the library was enough, and the library ran at its best only on Nvidia hardware.

When Google published TensorFlow in 2015 and Meta (then Facebook) released PyTorch in 2016, both were built with CUDA and cuDNN as their primary backend. This has even made its way into developers’ everyday language: in PyTorch the accelerator device is literally called "cuda", so much so that AMD GPUs, when used through ROCm, are exposed under the same name so as not to break existing code. A commercial brand has become the generic name of the category, something that happens only with true standards.

The following years saw a tight co-evolution between research and hardware. In 2016 Huang personally delivered the first DGX-1 to OpenAI. In 2017 Volta introduced Tensor Cores, the same year the Transformer paper came out, and mixed-precision training became standard practice precisely because Tensor Cores made it worthwhile. Every algorithmic innovation was prototyped on Nvidia GPUs, optimized on Nvidia GPUs and published with code that assumed Nvidia GPUs, and every new generation of chips was designed with an eye to the workloads that research produced. FlashAttention, in 2022, is the clearest example of this cycle: an algorithm whose gain comes entirely from knowing the memory hierarchy of Nvidia GPUs, and which was written first for CUDA and only later, with effort, adapted elsewhere.

The CUDA-X stack, where the value really lives

When people talk about the “CUDA moat” they often think of the language, meaning the syntax of kernels. But syntax is the easiest part to imitate, and in fact AMD copied it almost line by line with HIP. The value is concentrated in the layers above and below the language. At GTC in March 2026, celebrating the platform’s twenty years, Nvidia spoke of more than a thousand CUDA-X libraries, a community of between 4 and 6 million developers depending on the company source, and a couple of hundred thousand public projects that depend on CUDA. Huang called the libraries the company’s “crown jewels,” and it is a technically accurate description.

Library	Function	Why it matters
cuBLAS / cuBLASLt	Dense matrix multiplications (GEMM)	GEMM is almost all of the computation in a Transformer; every percentage point of efficiency is worth millions of dollars on a cluster
cuDNN	Neural network primitives, fused attention	Shifted the cost of critical kernels onto Nvidia for all of research
CUTLASS / CuTe	Open source C++ templates for building custom GEMMs and convolutions	The foundation on which many FlashAttention kernels and inference stacks are written
NCCL	Collective communications between GPUs (all-reduce, all-gather) over NVLink and InfiniBand	Without efficient collectives, distributed training on thousands of GPUs does not exist
TensorRT / TensorRT-LLM	Inference compilation and optimization	Quantization, kernel fusion, batching in production
CUB / Thrust	Parallel primitives (scan, sort, reduce)	Basic building blocks for anyone writing their own kernels
cuFFT, cuSPARSE, cuSOLVER, RAPIDS	Scientific computing, sparse algebra, data science	The HPC legacy that brings in users from outside AI

The work hidden inside these libraries is enormous. An optimal GEMM depends on the shape of the matrices, the data type (FP32, BF16, FP8, FP4 and block formats such as MXFP), the architecture, the tile size, the number of pipeline stages and the scheduling strategy, and cuBLAS contains heuristics and specialized kernels for thousands of combinations. This work is redone with every generation, often before the chip is on the market, because Nvidia gets access to the silicon and to the internal documentation of the real ISA (SASS) months in advance. Many of these libraries are distributed only as binaries, so their contents cannot simply be recompiled for other hardware.

PTX as a long-term contract

The second, less visible pillar is stability. PTX works as a contract between Nvidia and developers, because it guarantees that code written today will also run on future GPUs through the driver’s JIT compilation. For a company that has to amortize years of software development, knowing that the investment will outlive the hardware turnover is a very powerful commercial argument, and it explains why the perceived cost of staying on CUDA is so low compared with the perceived cost of leaving it. Competing platforms, which for years changed APIs, supported only a few cards at a time and dropped support for recent hardware, sent the opposite message.

Why it became the standard: a socioeconomic view

The technical story explains why CUDA was good, but it isn’t enough to explain why it has become almost impossible to dislodge. For that we need a few concepts from the economics of platforms and innovation, which in the case of CUDA all operate together and reinforce one another.

Indirect network effects and the flywheel

CUDA is a two-sided platform: on one side are the people who buy hardware, on the other the people who write software. The more Nvidia GPUs exist, the more it pays for developers to write for CUDA; the more CUDA software exists, the more it pays to buy Nvidia GPUs. Huang has presented this mechanism for years as the “CUDA flywheel”: an installed base attracts developers, developers produce algorithms and discoveries, discoveries open markets, and markets widen the installed base. In economics it’s called an indirect network effect, and it’s the same one that made Windows dominant in the 1990s, with one difference: Nvidia also controls the hardware, so it captures value on both sides.

Path dependence

Paul David and Brian Arthur described how an early advantage, even an accidental one, can lock in a standard for decades when returns are increasing. CUDA benefited from almost perfect timing: it was the only platform that was mature, documented and available on cheap hardware at the exact moment (2012-2016) when deep learning exploded. The choices a few dozen research groups made back then spread to everything that came after, from frameworks to university courses, from model architectures to numerical formats.

Specific human capital

An engineer who has spent five years optimizing CUDA kernels has skills that are only partly transferable. They know how to read Nsight reports, they know the quirks of every GPU generation, they remember which version of cuDNN introduced which regression. Companies hire people with these skills because their stack is CUDA, and people learn CUDA because job postings ask for it, so the labor market itself becomes a lock-in mechanism. The same is true of diffuse knowledge: forum answers, resolved GitHub issues, blog posts and post-mortems. Together they make up an enormous collective asset, and it is almost entirely specific to CUDA.

Switching costs and risk aversion

Shapiro and Varian showed that lock-in depends less on the cost of the alternative than on the total cost of switching, which includes code migration, retraining, the risk of regressions and lost time. In frontier AI, time is worth more than almost anything else. A lab training a model that costs hundreds of millions of dollars won’t accept the risk that a bug in an alternative vendor’s compiler ruins weeks of compute, and a 20-30% discount on hardware loses its meaning if it means falling a quarter behind competitors. It’s the contemporary version of the old adage that nobody ever got fired for buying IBM.

Complementary assets and liquidity

David Teece explained that whoever controls the assets complementary to an innovation captures its value. In Nvidia’s case the complementary assets are countless, starting with availability in every public cloud, where an Nvidia GPU is the standard product that everyone knows how to use. High-end Nvidia GPUs have become almost a currency, with a liquid secondary market and an estimable residual value, so much so that they have been used as collateral for billion-dollar financing by operators such as CoreWeave. An alternative accelerator, even a cheaper one, has an uncertain resale value and thinner demand, and that weighs on investment decisions.

A proprietary standard that moves faster than consortia

CUDA established itself as a de facto standard, without any formal recognition. Competing open standards, such as OpenCL, were run by consortia in which vendors with diverging interests had to agree on every extension, while Nvidia could add support for Tensor Cores, new numerical formats or new synchronization primitives on the very day a chip launched. The speed of a single owner beat the neutrality of a committee. Nvidia then defended this advantage on the contractual front too, adding to the toolkit license a ban on using translation layers to run CUDA output on non-Nvidia hardware, a clause that became a matter of public debate in 2024.

The system moat: hardware, networking and supply chain

Over the last five years Nvidia’s advantage has shifted from the single chip to the complete system, and this shift is decisive for understanding how much the moat really depends on software. Frontier models are trained on tens or hundreds of thousands of GPUs that must exchange gradients and activations with minimal latency, and at that scale the network matters as much as the compute.

In 2019 Nvidia announced the acquisition of Mellanox for about $6.9 billion, bringing in-house InfiniBand, which was already the standard for interconnects in supercomputers. To this were added NVLink and NVSwitch, to connect GPUs within a node and then across an entire rack, up to the NVL72 systems of the Blackwell generation, in which 72 GPUs form a single NVLink domain and behave, as far as the software is concerned, almost like one giant GPU. Networking has become a business of its own, with revenue of $10.98 billion in the fourth quarter of fiscal 2026 alone, up 263% year over year.

Software is what holds this system together. NCCL knows the topology of NVLink and InfiniBand and chooses communication algorithms accordingly, parallelism frameworks such as Megatron-LM take those primitives for granted, and distributed inference stacks such as Dynamo orchestrate the separation of prefill and decode across multiple nodes. Anyone who wants to replace Nvidia therefore has to replicate the chip, the interconnect, the communication library and the orchestration software, and do it with enough reliability to withstand weeks of continuous training.

The supply chain as a barrier

Finally, there is one barrier that software doesn’t touch at all. Production of high-end accelerators depends on TSMC’s CoWoS advanced packaging capacity and on the availability of HBM memory from SK Hynix, Samsung and Micron. Both are scarce resources, allocated years in advance. Thanks to its volumes and advance payments, Nvidia has secured a very large share of this capacity, and a competitor with an excellent chip can still find itself limited by how many units it can get manufactured. On top of this comes the web of financial agreements with the customers themselves, including investments in OpenAI, in Anthropic and in specialized cloud providers. Some analysts look at these with concern because of the circularity of the money flows, but in practice they tie the biggest buyers to the supplier.

Twenty Years of Attempts to Challenge CUDA

The story of CUDA’s competitors is useful because it shows which strategies failed and which worked at least in part, and it sets the stage for understanding why AI agents are a different kind of threat.

OpenCL, the Open Standard That Stopped Halfway

OpenCL was born in 2008-2009 on Apple’s initiative and was handed to the Khronos consortium, with the ambition of offering a single language for CPUs, GPUs and accelerators from any vendor. It guaranteed code portability but not performance portability, because a kernel optimized for one architecture ran poorly on another. Nvidia supported it only lukewarmly, staying on version 1.2 for years. Apple itself deprecated it in 2018 in favor of Metal, and its library ecosystem never reached the depth of cuBLAS and cuDNN. The major deep learning frameworks never adopted it as a first-class backend.

ROCm and HIP, the Declared Imitation

AMD launched ROCm around 2016, with HIP as a language deliberately almost identical to CUDA and the hipify tool to translate source code automatically. At the syntactic level the translation is largely mechanical, which is why AMD could present migration as simple. The problems lay elsewhere: limited support for consumer cards, breaking changes between versions, less polished documentation, incomplete coverage of equivalent libraries (rocBLAS, MIOpen, RCCL), and above all the performance gap you get without specific optimization work. Independent analyses published in late 2024 on the MI300X described hardware with theoretical numbers higher than the H100’s, but software that made it impossible to reach them outside AMD’s curated configurations. Since then the pace has changed, with ROCm 7 in 2025, version 10.0 in August 2026 and a six-week release cadence announced in July.

A concrete example shows where mechanical translation ends. Warp-level reduction is one of the most common idioms in optimized kernels, and in CUDA it is written like this.

// CUDA: somma i valori dei 32 thread di un warp
__device__ float warp_sum(float v) {
    for (int off = 16; off > 0; off >>= 1)
        v += __shfl_xor_sync(0xffffffff, v, off);
    return v;
}

The code contains two implicit assumptions, the initial offset of 16 and the 32-bit mask, both valid only for a warp of 32 threads. On Instinct GPUs the wavefront has 64, and a syntactic conversion that renames the functions while leaving the constants untouched produces a kernel that compiles, runs and sums only half of the wavefront, without reporting any error. The portable version derives everything from the actual size.

// HIP: corretto su AMD (wavefront da 64) e su Nvidia (warp da 32)
__device__ float wave_sum(float v) {
    for (int off = warpSize / 2; off > 0; off >>= 1)
        v += __shfl_xor(v, off);
    return v;
}

Differences of the same kind affect vote masks, which become 64-bit, shared memory banks, inline PTX assembly, which has no automatic translation, and calls to Tensor Cores, which on AMD go through different libraries. HIP’s documentation has always warned that after conversion there is still platform-specific tuning work to do. That work requires understanding the intent of the code, and it is the kind of task where a language model outperforms a syntactic rewriting tool.

oneAPI, ZLUDA, SCALE and the Compatibility Layers

Intel bet on oneAPI and on the SYCL language, a Khronos standard based on modern C++, with respectable technical results but limited adoption, partly because of the company’s difficulties on the accelerator front. Another path was binary or source compatibility layers. ZLUDA, developed by Andrzej Janik and funded first by Intel and then by AMD until 2024, made it possible to run unmodified CUDA binaries on GPUs from other manufacturers, while Spectral Compute’s SCALE compiles CUDA sources directly for AMD hardware. These approaches run into two limits: Nvidia’s license clause on translation layers, and the fact that replicating CUDA’s exact behavior isn’t enough to match the performance of code written natively for the target hardware.

Triton, JAX and TPUs: Moving Up a Level Instead of Imitating

The strategies that worked best are the ones that avoided a direct confrontation, raising the level of abstraction above CUDA. Triton, the language introduced by OpenAI in 2021 and designed by Philippe Tillet, lets you write kernels in Python by reasoning in blocks of data instead of individual threads, leaving shared memory, coalescing and Tensor Cores to the compiler. Since 2023 the PyTorch 2 compiler has generated Triton code automatically, and Triton has backends for AMD as well. This means that a growing share of the kernels used in production is no longer written directly in CUDA.

On Google’s side, JAX and the XLA compiler have made TPUs a complete, self-contained platform, on which the Gemini models are trained and on which Anthropic has also committed large volumes of compute, alongside AWS’s Trainium chips. Initiatives like MLIR and Modular’s Mojo go in the same direction. The lesson from these cases is that the moat erodes when someone introduces an intermediate layer controlled by others, because at that point CUDA becomes one of several possible backends instead of the interface developers think in.

The hyperscalers’ custom silicon

So far, the strongest pressure on Nvidia’s share has come from the chips that the big cloud operators design for themselves, with Broadcom and Marvell as design partners that together cover about 95% of this market. TrendForce estimates that in 2026 ASIC-based servers will reach 27.8% of AI server shipments, with shipments growing 44.6% against 16.1% for commercial GPUs. Google is on its seventh generation of TPUs, Amazon has put more than a million Trainium chips into service, and Meta and Microsoft are developing MTIA and Maia.

These chips never needed CUDA, because they are born together with their own software stack and serve internal workloads that their owner knows in depth. Their limit, for anyone looking for an alternative, is that you can’t buy them. You can only rent them in the cloud of the company that designed them. So they erode Nvidia’s revenue among its largest customers without offering the rest of the market an ecosystem to migrate to.

China, an ecosystem without CUDA out of necessity

The second front is geopolitical. US export restrictions have effectively closed the Chinese market for advanced chips to Nvidia, so much so that the 2026 guidance includes no data center revenue from China, and they have pushed an entire country to build an alternative stack. In 2025 Huawei open-sourced CANN, the CUDA equivalent for its Ascend chips, and in March 2026 it accompanied the Ascend 950PR with a version designed to offer near-direct substitutes for CUDA interfaces. On September 30, DeepSeek and Huawei published shared tools, including a high-level language for kernels and a matrix multiplication library optimized for Ascend.

The road is still long. According to a researcher quoted by the South China Morning Post, porting a training pipeline built for CUDA to Ascend adds at least 50% in time and cost, and Chinese labs keep training on Nvidia hardware when they can get it. But the episode shows that an alternative ecosystem emerges when access to the dominant one is denied, and that the strength of CUDA also depends on potential competitors being able to keep using it.

2026, the AI agents that bring CUDA to AMD

During 2026 the discussion about the moat changed in nature, because for the first time the cost of migration began to fall without anyone having to convince Nvidia or impose a new standard. The new factor is models that can read, understand and rewrite low-level system code, checking the result on their own.

Date	Event	Real significance
Nov 12, 2026	Jensen Huang steps down as CEO and from Nvidia’s board; Colette Kress becomes interim CEO	The person who personally managed customers, suppliers and governments leaves the stage
Nov 10-11, 2026	Anthropic IPO roadshow; Mythos reportedly got a model’s prompt processing running on AMD MI355X in about three days	Company statement, limited to one model and one phase of inference; no technical document
Sep 30, 2026	DeepSeek and Huawei publish open source tools for Ascend chips	A stack without CUDA grows where CUDA no longer reaches
Jul 22-23, 2026	AMD and Anthropic announce up to 2 GW of Instinct MI450 series GPUs in Helios racks starting in the first half of 2027, with AMD investing up to $5 billion; ROCm.ai debuts	First frontier lab committed to AMD at gigawatt scale
Jan 22, 2026	A developer uses Claude Code to port a CUDA backend to ROCm in about 30 minutes	Open source project, tested on a single consumer GPU

January, the demonstration on Reddit

The first signal came from a developer known as johnnytshi, who used Claude Code to write a complete ROCm backend for a chess engine based on neural networks with attention, with FP32 and FP16 variants and automatic detection, relying on MIOpen as the equivalent of cuDNN and without going through hipify. The only serious obstacle reported was the difference in data layout between the two platforms. The news got an enormous response, even though the more careful commenters immediately noted the limits of the experiment: a small codebase, tests on a single consumer AMD architecture, and no claim of parity with hand-optimized versions.

July, a weekend on an MI355X rack

At the Advancing AI event in July, AMD and Anthropic announced a multi-year collaboration that includes using Claude to accelerate ROCm development and adopting Claude in AMD’s engineering teams. On that occasion the episode that would end up in the roadshow in November was told for the first time. AMD had delivered a rack of MI355X to Anthropic, an engineer had handed the porting over to the model and let it work on its own for the weekend, and on Monday the team had in front of them the performance curves of their flagship model on that hardware, climbing steadily.

At the same event AMD presented ROCm.ai, which gives coding agents (Claude, Codex, Cursor, Gemini) dedicated skills, a deterministic and scriptable command line, and a console with telemetry and profiling, with the stated goal of turning agents into expert ROCm users. AMD explained that its engineers had been using AI-assisted coding tools for a year to generate kernels, optimize code and debug, with better results than expected. The choice is significant, because AMD has stopped chasing Nvidia in training human developers and has started making its platform legible to agents.

November, Mythos’s three days

The strongest claim came in the investor materials tied to Anthropic’s IPO. Co-founder and Chief Compute Officer Tom Brown explained that the company trains and serves Claude on all four major chip families (Trainium, TPUs, Nvidia GPUs and AMD GPUs) and treats them as a single, interchangeable pool of compute. For the AMD debut, the work was handed to Mythos, which according to Brown wrote or ported almost 100% of the kernels with “basically no human intervention,” in about three days, against the six to twelve months and dozens of engineers the traditional approach would take.

The caveats are in the footnote of the slide, which limits the three days to the prompt processing of a single Anthropic model on the MI355X, brought to a level of performance described as useful. Token generation, training and any numerical comparison with Nvidia hardware are left out. The figure comes from an investor presentation, a setting in which the company has every interest in showing efficiency in the use of capital, and as of mid-November no technical document describing the port has been published.

Finally, we should remember how tangled the interests are. Nvidia remains both a supplier to Anthropic and an investor, with a commitment of up to $10 billion made in November 2025 and a stake in the private placement described in the S-1/A filing of November 10, the same one AMD takes part in. The filing also ties Anthropic to AMD through a supply of MI450s that, if bought in full, would exceed $20 billion in 2027. Whoever tells the story of how easy the port was therefore has a significant commercial relationship with both parties involved.

Still, the episode fits into a practice that is now widespread. Nvidia itself publishes examples on its blog of agents translating CUDA code between languages, and the 2026 hackathons on AMD compute produced dozens of multi-agent pipelines that classify kernels, translate them into HIP, compile them on the MI300X and compare the results with the original. In less than a year, model-assisted translation has gone from a curiosity to an ordinary working tool.

How much CUDA is really at risk

The risk is real and selective. Agents hit with precision the mechanism that for twenty years made leaving CUDA expensive, namely the specialized human work needed to rewrite and optimize code, and they leave almost untouched the layers of the moat that don’t depend on software. But the part that gets hit coincides with the customers who generate the largest share of revenue, and for this reason the potential effect goes far beyond the technical scope of any single demonstration.

A new kind of threat

All previous attempts asked developers to change habits, language or trusted vendor. Agents work on existing CUDA code and produce native code for the other platform, knowing both ecosystems better than the average engineer. In economic terms, they reduce switching costs and the specificity of human capital at the same time, two of the pillars described above. A company that once had to hire a team of ROCm specialists, rare and expensive, can hand the first translation to the agent and focus its experts on validation.

What has been shown and what is missing

Prompt processing is the most favorable case. It is dominated by large, regular matrix multiplications, limited by compute capacity, which map relatively naturally onto AMD’s GEMM libraries. Token-by-token generation, by contrast, is limited by memory bandwidth and depends on attention kernels with paged cache, speculative decoding, Mixture of Experts routing and communication between GPUs, all areas where architectural differences weigh the most. The distinction is so sharp that Nvidia itself uses it in its own racks, where it has started to host third-party chips specialized in the decode phase while leaving prefill to the GPUs.

Training adds backward passes, numerical stability at low precision, collective communication across thousands of GPUs and fault tolerance over runs lasting weeks. None of these areas has been the subject of a public demonstration comparable to the one on prefill, and the table summarizes where things stand, layer by layer.

Moat layer	Difficulty before agents	With capable agents	Residual moat
Language syntax	Low, hipify was enough	Negligible	None
Inference kernels, prefill	High, months of work	Days, according to Anthropic	Low for those who have model and hardware
Inference kernels, decode and MoE	Very high	Declining, not publicly demonstrated	Medium
Large-scale distributed training	Very high	To be demonstrated	High
Libraries and tuning for new generations	Very high, with a time advantage for Nvidia	Declining for competitors too	Medium
Interconnect (NVLink, InfiniBand, NCCL)	Hardware and software together	Software can be translated, hardware remains	High
Supply chain (CoWoS, HBM)	Outside of software	Unchanged	High
Long tail of companies and research	Depends on frameworks	Helps, but expertise is needed to validate	Medium-high

Inference shifts the center of gravity

Inference is taking up a bigger share of total compute every quarter, and it combines three traits that work against Nvidia. It is the easiest workload to move, because for each model the kernels are few and stable. It is the workload where alternative silicon claims its biggest cost advantages, up to 65% of total cost of ownership according to the most favorable estimates reported in the trade press. And it is the workload a lab fully controls, since it serves its own models on its own stack. Taken together, these factors explain why the first documented breach involves the inference of a frontier lab, and suggest that the next ones will come from the same direction.

Verification as a Condition

An agent that translates kernels is useful only inside a closed loop, where it can compile, run, compare the results against a numerical reference and measure performance with a profiler. That is why ROCm.ai is investing in scriptable tools and in telemetry exposed to agents, and why the best results come from organizations that have a working reference implementation, direct access to the hardware and extensive test suites. Frontier labs have all three, while an average manufacturing company or a university group rarely has more than one. So the erosion is concentrated at the top, where the multibillion-dollar buyers are, and moves much more slowly in the long tail.

Nvidia’s Countermoves

Nvidia saw the transition coming and is responding on two fronts. The first is software. With CUDA 13.1, in December 2025, it introduced CUDA Tile, a tile-based programming model with its own intermediate representation (Tile IR) and the cuTile Python language. CUDA 13.3 extended it to C++, along with CUDA Python 1.0 and the CompileIQ autotuning, and Nvidia also wrote a Tile IR backend for Triton. The goal is for the high level where developers and agents do their thinking, until now contested by Triton and by framework compilers, to stay inside the Nvidia ecosystem and lead to the proprietary optimizations of its compiler.

The second front is the system. With NVLink Fusion, introduced in 2025, Nvidia licenses its interconnect to those who design custom silicon, on the condition that every platform contains at least one of its products. In 2026 it invested $2 billion in Marvell, which designs the chips for AWS and Microsoft. In September the startup d-Matrix announced that its accelerator would be integrated into Nvidia racks following the scheme described above, with prefill on the GPUs and decode on the specialized chip. The logic is that of a highway toll booth. If part of the compute is destined to migrate to someone else’s silicon, Nvidia prefers that it happen inside a rack, a network and a management software that remain its own.

It’s worth adding that agents work in both directions. They lower the cost of porting a model from Nvidia to AMD, and in the same way they lower Nvidia’s cost of producing optimized libraries for every new architecture, as well as a customer’s cost of going back if the alternative disappoints. In a market where software translates easily, the winner is whoever offers the best balance of performance, total cost and availability in that quarter. It is a competition in which Nvidia starts out strong, but with far less of a positional rent.

Margins and Bargaining Power

The most plausible short-term effect concerns prices. In fiscal 2026 Nvidia’s gross margin was 71.1%, down from 75.0% the year before, for reasons the company attributes to the transition to Blackwell systems and to inventory charges. Margins at this level hold as long as the large customers have no credible alternative for critical workloads, and the customer base is highly concentrated: in the same fiscal year, two direct customers accounted for 22% and 14% of total revenue.

A lab that can show it can move inference to AMD, TPUs or Trainium in days instead of months sits at the table with a credible threat, even if it keeps buying Nvidia. Anthropic’s statement should also be read in this light, as a signal aimed at markets and suppliers just as the company prepares to go public.

A Context That Narrows the Margin for Error

The technical erosion arrives at a delicate moment. Huang resigned on November 12, after admitting to a relationship with a woman whom US intelligence believes to be a Chinese spy and while the board was conducting an independent investigation. With him goes the person who, for 33 years, directly managed relationships with customers, suppliers and governments. The restrictions on China have led to guidance with no Chinese data center revenue, and in early November there were estimates that Chinese developers would receive about 1% of the advanced chips ordered. On November 5 the stock lost 9.26% in a day of widespread selling in semiconductors.

None of these factors concerns CUDA itself, but all of them reduce the room Nvidia has to absorb a weakening of its software advantage. The quarterly report of November 17, the first signed by an interim CEO, will tell us something about the numbers and probably little about the long-term dynamics.

Three Scenarios and the Signals to Watch

The assessments that follow are mine and should be taken as working hypotheses, ordered from most to least likely over a two-to-three-year horizon.

Scenario	What happens	Signals to watch
Margin erosion	Market share stays dominant, inference at the large labs becomes multi-platform, and the biggest customers get better terms	Gross margin consistently below 70%, announcements of production inference across several chip families
The moat shifts to the system	CUDA becomes one of several possible backends, and Nvidia defends its value with racks, networking and manufacturing capacity	Adoption of NVLink Fusion, growth in networking revenue, Tile IR adopted by frameworks
Commoditization	Frontier training is documented on non-Nvidia hardware with comparable performance	A verifiable technical paper, the actual delivery of Helios’s 2 GW in 2027, AMD above 10-15% of industry revenue

The first two scenarios are compatible with each other and already partly under way. The third requires agents to handle distributed training and the optimization of new chip generations as quickly as they did with prefill, and today there is no public evidence that they can.

Conclusions

Nvidia became what it is because in 2006 it decided to make every GPU it sold programmable, bore that cost for years with no visible return, cultivated an academic community before a market existed, and found itself with the only platform ready when deep learning exploded. From then on, the advantage fed itself through network effects, path dependence, specialized human capital, libraries that shifted the cost of optimization onto Nvidia, and long-term compatibility that made every software investment safe. In recent years, control of networking and privileged access to manufacturing capacity have been added to all this.

Coding agents challenge the part of this edifice that rested on the scarcity of expert labor. It is the part that, more than any other, kept AMD from turning good hardware into market share, and its erosion is already visible in the labs that have their own models, hardware and tests. The physical parts of the moat remain solid, custom silicon and China press from other directions, and Nvidia is working to bring back inside its own perimeter both the abstraction layer on which developers and agents will reason and the racks in which competitors’ chips will run.

So CUDA is losing its function as a barrier before it loses its position as a standard, and the difference between the two will show up in margins more than in market share. The signal to follow is the appearance of a documented, verifiable port that includes distributed training or complete inference in production, because from that moment competition among accelerators will be played out on hardware, networking and manufacturing capacity, as in any other semiconductor sector.

Sources

Anthropic, AMD and assisted porting

Nvidia, numbers and context

Competitors and market

The historical references (Brook, G80, AlexNet, cuDNN, OpenCL, Mellanox, Triton) and the concepts from platform economics come from the well-known literature, not from these pages.