Nvidia Let a Korean Rival Into Its Rack. It Gets the Second Half of Every Answer

Korea keeps showing up on this blog. Yesterday it was Nvidia's $1 billion Naver check, which is about money. Today's Korea story is about the plumbing inside the rack, and I think it's the more interesting one.

On November 8, Nvidia announced that Rebellions, a Korean AI chip startup, will plug its next chip, the R200 (no relation to Nvidia's own Rubin GPU, which SemiAnalysis also calls R200; naming is hard), into Nvidia's rack platform using NVLink Fusion, and that SK Telecom plans to run those systems in its AI data center. The three signed a memorandum of understanding the next day at Nvidia's AI Day event at COEX in Seoul (Seoul Economic Daily; AI Affairs, citing Yonhap).

TL;DR

• Nvidia is letting a would-be competitor's chip sit inside its rack, on its high-speed wiring. Nvidia keeps owning the rack and the wiring.
• The plan splits each AI answer in two: Nvidia GPUs do "prefill" (reading your prompt) and Rebellions chips do "decode" (writing the reply). That split is a real, well-studied trick for serving AI more cheaply.
• It's a plan, not a purchase order. No dollar figures, no R200 specs, and systems aren't expected until 2028.
• For Rebellions, valued at about $2.34 billion, the timing is useful: the same day, Korean press reported its IPO filing is likely slipping to next year or later.

Quick jargon decoder

• GPU vs. NPU. A GPU is Nvidia's general-purpose AI chip. An NPU ("neural processing unit") is a chip built only for running AI models, usually trading flexibility for efficiency. Rebellions makes NPUs.
• Prefill and decode. When you ask a chatbot something, it first reads your whole prompt in one big gulp (prefill), then writes the answer one word-piece, or "token," at a time (decode).
• KV cache. The model's notes on your prompt. Prefill creates them; decode keeps re-reading them to write each new token. In a split system, those notes have to be handed from one chip to the other, fast.
• NVLink / NVLink Fusion. NVLink is Nvidia's super-fast chip-to-chip wiring inside a rack. NVLink Fusion is the program that lets other companies' chips use it.
• MGX. Nvidia's standard rack blueprint (size, power delivery, cooling), so outside chips can slot into a familiar box built by an existing supplier base.
• MoU. A memorandum of understanding: a signed "we intend to work together," not a contract to buy anything.

What was actually announced

Per Nvidia's release:

• Rebellions will connect R200 NPUs to each other within a rack over NVLink, using NVLink Fusion, and build the systems on Nvidia's MGX rack architecture.
• The companies "plan to develop an approach to disaggregated inference," with "NVIDIA GPUs for prefill and Rebellions NPUs for decode."
• SK Telecom plans to deploy the systems and connect them to its own models, including A.X K2, and to public services under Korea's "AI for All" project.
• Korea's Ministry of Science and ICT is backing it, and the deputy prime minister and science minister, Bae Kyung-hoon, said it lets Korean NPUs "enter the computing infrastructure market alongside NVIDIA."
• Timing: "Rebellions' R200 systems are expected to be ready for deployment in 2028."

Seoul Economic Daily adds that this is the first time Nvidia has formalized this kind of tie-up with a Korean NPU company, following Marvell and MediaTek, and that the ministry wants to extend it to other Korean NPU makers. AI Affairs is careful to note every step is "part of the companies' proposed deployment rather than a reported operating installation." Fair.

Why split one answer across two chips?

This is the part worth understanding, because it's where AI serving is heading generally.

Prefill and decode stress a chip in opposite ways. The DistServe paper (from researchers at Peking University, StepFun and UC San Diego) lays it out plainly: prefill "tends to be compute-bound," meaning it's limited by how fast the chip can do math, while decode is "constrained by the GPU's memory bandwidth," meaning it's limited by how fast the chip can pull data out of memory. Each decode step only produces one token, but it still has to re-read the model and the KV cache.

Everyday analogy: prefill is a speed-reader skimming a 50-page brief. Decode is someone writing a reply one word at a time, flipping back through the brief before every single word. Different jobs, different ideal workers. When one chip does both, they get in each other's way.

The paper's simple worked example (a 13-billion-parameter model on Nvidia A100s) shows the payoff: one GPU doing both jobs handled about 1.6 requests per second within its latency targets. Splitting the work across two prefill GPUs and one decode GPU handled 10 requests per second, or about 3.3 per GPU, roughly 2.1x better per chip.

That's one academic setup from 2024, not a promise for R200. But it's why everyone, Nvidia included, is building for the split.

The twist: Nvidia's own roadmap splits it the other way

Here's what caught my eye. When Nvidia built its own specialized chip for this, the Rubin CPX, it aimed the cheaper chip at prefill: a monolithic die with 128GB of GDDR7 memory (the cheaper, slower kind), working alongside regular Rubin GPUs, with availability expected at the end of 2026. SemiAnalysis's write-up makes the logic explicit: putting prefill on a chip full of expensive high-bandwidth memory "is a waste," because prefill barely uses that bandwidth.

In the Rebellions plan it's reversed. Nvidia keeps prefill, and the outside chip takes decode, the memory-hungry half. So R200 only makes sense if it delivers lots of memory bandwidth per dollar and per watt. Neither Nvidia's release nor Rebellions' latest funding release gives R200's memory specs; Nvidia's release only says it's "optimized for low-latency and energy-efficient inference" on frontier models, including "multitrillion-parameter mixture-of-experts models." That's the number I most want to see.

Why Nvidia would do this at all

My read: NVLink Fusion is Nvidia's way to stay the landlord even when it isn't the only tenant. Nvidia unveiled the program in May 2025, with MediaTek, Marvell, Alchip, Astera Labs, Synopsys and Cadence as early silicon partners and Fujitsu and Qualcomm planning CPUs. If countries want their "own" chips for sovereignty reasons, Nvidia would rather those chips live inside an Nvidia rack, on Nvidia wiring, next to Nvidia GPUs, than in a rival system built from scratch.

And Nvidia keeps the prefill half, the part that needs big raw compute, which is exactly what its GPUs are best at. Some would call it giving up share. I'd call it renting the other half of the house while owning the building. Galaxy-brained, honestly.

The money side: a credibility boost at a useful moment

Rebellions isn't small. It raised $400 million in a pre-IPO round in March, led by Mirae Asset and the Korea National Growth Fund, after a $250 million Series C in September 2025. That's $850 million raised in total, at a valuation of about $2.34 billion (CNBC has the same figures). Its backers include Samsung, SK Hynix, SK Telecom, Arm and Aramco.

The IPO picture is shakier. In July, CEO Park Sunghyun told Reuters Rebellions aimed to list in Korea in the first half of 2027, with a possible U.S. listing later. But on November 9, Seoul Economic Daily reported, citing investment-banking sources, that Rebellions and Upstage had planned to file for preliminary review this month and that filing next year or later "has rapidly emerged as the likely option," as returns on AI stocks have dropped. The same report says these unicorns, Megazone Cloud aside, were loss-making, and that bankers see Anthropic's IPO, slated for late this month, as the swing factor.

So an Nvidia logo next to yours, in the same week your IPO slips, is not a bad look. To be clear, that's my interpretation, not anything the companies said.

Side note: Nvidia also wrote checks

Same trip, same day: at a startup forum it co-hosted with Korea's science ministry, Nvidia announced an investment in Point2, which builds radio-frequency interconnects that move data between chips and racks, plus planned investments in LLM startups Motif Technologies and Upstage and in robotics-model developer RLWRLD. No amounts were given. Seoul Economic Daily adds that Point2 had already taken money from NVentures, Nvidia's venture arm, in the first half of this year, and that the other three deals aren't finalized.

What I think

• Real signal, small near-term dollars. No contract value, no unit counts, nothing shipping until 2028. Don't model revenue off this.
• The split is the story. The industry is moving from "one big GPU does everything" toward specialized chips per phase of an answer. Nvidia wants to own the wiring that connects them, whoever makes the chips.
• Decode is the hard half to win. Rebellions picked, or was given, the memory-bound job. Without public R200 memory specs, nobody outside these companies can judge whether it wins on cost per token.
• Korea's playbook is now clear. Naver gets Nvidia equity and Brookfield money; SK Telecom gets Nvidia racks with Korean chips inside. Either way, Nvidia sits in the middle.

What I'm watching

• CoreWeave reports tomorrow, November 10, and Nebius on November 12, as I flagged in the Naver post. Funding costs and delivery dates still drive this whole market.
• R200 specs: memory type, bandwidth and power. That decides whether the decode bet works.
• Which Korean NPU maker is next. The ministry says it wants more than Rebellions; FuriosaAI is the obvious name to watch, but nothing has been announced.
• Rebellions' filing date, and how Anthropic's IPO later this month trades. Korean bankers are openly using it as the bellwether.