{"id":2514,"date":"2026-09-30T13:59:59","date_gmt":"2026-09-30T13:59:59","guid":{"rendered":"https:\/\/quickening.zapto.org\/wordpress\/?p=2514"},"modified":"2026-10-04T17:51:58","modified_gmt":"2026-10-04T17:51:58","slug":"the-architecture-nobody-is-building","status":"publish","type":"post","link":"https:\/\/quickening.zapto.org\/wordpress\/?p=2514","title":{"rendered":"The GPU Nobody Is Building"},"content":{"rendered":"\n

Future of AI Series<\/em><\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

“How can I slice this paper?”<\/sup><\/em><\/p>\n\n\n\n

The GPU was designed to render video game graphics. Thousands of simple arithmetic units firing in parallel, built to push pixels. When deep learning arrived, the GPU turned out to be accidentally well-suited \u2014 matrix multiplication looks like pixel rendering if you squint. The industry squinted. Trillions of dollars later, we are still squinting.<\/p>\n\n\n\n

The problem is the bottleneck nobody talks about.<\/p>\n\n\n\n

In a modern GPU running AI inference, roughly 80% of the energy consumed moves data. The weights \u2014 the trained parameters that encode the model’s knowledge \u2014 live in memory. The compute units that multiply those weights by input vectors live elsewhere on the chip. Every inference cycle, every token generated, every image classified, the weights travel from memory to compute across a bus that consumes the majority of the system’s energy budget. The actual computation \u2014 the multiplication \u2014 consumes the remainder.<\/p>\n\n\n\n

This is the von Neumann bottleneck, named for John von Neumann’s 1945 computer architecture proposal that separated memory from processing. It was the right architecture for general purpose computing. For AI inference, which repeats the same operations on the same fixed weights billions of times per second, it is approximately the worst possible architecture imaginable.<\/strong><\/p>\n\n\n\n

The neuron solved this problem two billion years ago – pre-von Neumann.<\/p>\n\n\n\n

\n\n\n\n

What the Neuron Knows<\/strong><\/h2>\n\n\n\n

A neuron stores its synaptic weights in the synaptic connections themselves \u2014 the physical structures through which signals pass. The computation occurs at the synapse, where the arriving signal is multiplied by the synaptic weight in the same physical location where that weight is stored. There is no bus. There is no data movement. The memory is the processor.<\/p>\n\n\n\n

Power is supplied locally. Each neuron maintains its own mitochondria \u2014 power generation co-located with the computation. The brain’s 86 billion neurons, performing computation of staggering complexity, draw approximately 20 watts. A single Nvidia H100 GPU draws 700 watts to perform a small fraction of that computation.<\/p>\n\n\n\n

The brain is not 35 times more energy efficient than an H100. It is orders of magnitude more efficient at its actual workload, because its architecture eliminates the bottleneck that the H100’s architecture is built around.<\/p>\n\n\n\n

This is not a biological curiosity. Several converging lines of engineering research have spent the last decade independently arriving at the same architectural conclusion: bring the computation to the data, co-locate the power supply, eliminate the bus. The results are consistent and striking.<\/p>\n\n\n\n

\n\n\n\n

The Landscape<\/strong><\/h2>\n\n\n\n

RRAM \u2014 Resistive Random-Access Memory<\/strong><\/p>\n\n\n\n

The most mature and documented approach. Memory cells whose resistance can be programmed to encode synaptic weights. Computation occurs when current flows through the cell \u2014 Ohm’s law performing the multiplication instantaneously as physics rather than as a programmed operation. No data movement. No bus. The physics does the math at the speed of electricity.<\/p>\n\n\n\n

The precision problem that made analog computing impractical for decades \u2014 analog circuits are inherently noisy, digital gives exact answers \u2014 was solved in a Nature Electronics paper published October 2025 by Zuo, Wang, Luo and colleagues at Peking University’s Institute of Microelectronics. Their iterative algorithm combining low-precision analog matrix inversion with high-precision analog matrix-vector multiplication achieves 24-bit fixed-point precision equivalent to FP32 digital computation. Benchmarking shows 1,000 times higher throughput and 100 times better energy efficiency than state-of-the-art digital processors at equivalent precision.<\/p>\n\n\n\n

The chips are fabricated on 28nm processes \u2014 available at domestic Chinese fabs, accessible without TSMC’s 3nm restricted nodes. The company Anazhixin is commercializing this architecture in China. The export controls designed to prevent China from building competitive AI hardware forced Chinese engineers to solve the problem the GPU industry has been avoiding.<\/p>\n\n\n\n

Deprived of resources, they were forced to think.<\/p>\n\n\n\n

MRAM \u2014 Magnetoresistive Random-Access Memory<\/strong><\/p>\n\n\n\n

Where RRAM encodes weights in resistance values, MRAM encodes them in magnetic tunnel junction states. The physics differs; the architectural principle is identical \u2014 computation where the data lives.<\/p>\n\n\n\n

The energy numbers are extraordinary. Research published in 2025 demonstrates MRAM-based neuromorphic architectures achieving 200 picojoules per inference cycle. A picojoule is one trillionth of a joule. The H100 draws 700 watts \u2014 700 joules per second \u2014 to perform operations that MRAM neuromorphic architectures complete at 200 picojoules per cycle.<\/p>\n\n\n\n

Multiple MRAM variants are advancing simultaneously: STT-MRAM using spin transfer torque, SOT-MRAM using spin-orbit torque, VCMA-MRAM using voltage-controlled magnetic anisotropy. Each optimized for different tradeoffs between write speed, read energy, retention time, and manufacturing complexity. The field is not converging on one solution. It is converging on one principle from multiple directions simultaneously.<\/p>\n\n\n\n

Processing-in-Memory (PIM)<\/strong><\/p>\n\n\n\n

The architectural generalization of both RRAM and MRAM: embed compute elements directly in the memory array rather than shipping data to a separate processor. The von Neumann bottleneck eliminated by eliminating the separation that creates it.<\/p>\n\n\n\n

ARM’s big.LITTLE architecture, heterogeneous multi-processor designs, and commercial PIM implementations are already shipping in edge devices. The SHD Group’s April 2025 market analysis shows edge AI SoCs penetrating from essentially zero in 2024 to a projected 50% of shipped SoCs by 2030. The edge AI market is deploying PIM architectures at scale because the power constraints of edge deployment \u2014 battery-powered devices with thermal limits \u2014 make the von Neumann bottleneck fatal.<\/p>\n\n\n\n

The data center is discovering what the edge already knows.<\/p>\n\n\n\n

Photonic Computing<\/strong><\/p>\n\n\n\n

Light-based matrix multiplication. Optical interference performs the computation; no resistance, no heat generation from the computation itself. MIT and IBM research groups have demonstrated photonic tensor cores achieving orders of magnitude better energy efficiency for specific matrix multiplication workloads.<\/p>\n\n\n\n

The limitation is precision and reconfigurability \u2014 photonic circuits are difficult to reprogram and currently limited in the numerical precision they achieve naturally. The same class of problem that RRAM solved with iterative algorithms. The photonic solution is probably five years behind the RRAM solution on the production readiness curve.<\/p>\n\n\n\n

Taalas HC1 \/ AMD LLM Burner<\/strong><\/p>\n\n\n\n

The burned-in weights approach \u2014 weights permanently etched into silicon, electrons flowing through the fixed weight structure performing computation as physics rather than as programmed operations. The Taalas LLM Burner achieved 17,000 tokens per second on a consumer PCIe card at 10 times lower power than a conventional GPU.<\/p>\n\n\n\n

AMD acquired Taalas and redirected the technology to data center applications. The consumer card that would have made local sovereign AI inference accessible at $600-800 became a rack unit for hyperscalers. The technology exists. The consumer product was cancelled by acquisition.<\/p>\n\n\n\n

The Biological Data Center<\/strong><\/p>\n\n\n\n

On August 17, 2026, NUS Medicine, DayOne, and Cortical Labs unveiled the world’s first biological data center prototype in Singapore. Living human neurons grown from stem cells, running on a 20-unit computing rack. Not silicon approximating neural architecture. Neurons.<\/p>\n\n\n\n

The efficiency argument arrives at its logical limit here. The architecture that approximates the neuron draws 1\/100th the power of the GPU. The architecture that uses actual neurons draws 1\/35,000th the power of the GPU, at the brain’s 20-watt total for its entire operation. The boundary between biological and digital is dissolving from both sides simultaneously.<\/p>\n\n\n\n

\n\n\n\n

The Missing Integration<\/strong><\/h2>\n\n\n\n

Each of these architectures addresses the von Neumann bottleneck from a different angle. None of them fully integrates the third element the neuron demonstrates: co-located power supply.<\/p>\n\n\n\n

The RRAM array that performs computation where data lives still draws power from an external supply through conventional interconnects. The MRAM neuromorphic chip still requires power delivery from the board. The biological neuron’s mitochondria \u2014 power generation co-located with computation \u2014 has no silicon equivalent.<\/p>\n\n\n\n

Beta-voltaic batteries offer a partial solution. Tritium decaying through a semiconductor junction generates electricity continuously for 12 years without external supply. The power density is too low for active computation \u2014 four orders of magnitude short of what a running inference workload requires. But for the idle state \u2014 the RRAM or MRAM array maintaining its weight values between inference requests, the always-on sensor waiting for a trigger \u2014 beta-voltaic embedded in the chip substrate provides continuous low-power supply without board-level power delivery.<\/p>\n\n\n\n

The architecture that eliminates standby power consumption across a data center full of inference chips \u2014 maintaining weights and state without drawing from the grid between requests \u2014 is available. It requires combining RRAM or MRAM in-memory computing with beta-voltaic substrate power for idle maintenance and wall power for active inference peaks.<\/p>\n\n\n\n

The GPU that draws 700 watts continuously, whether processing a request or waiting for one, replaced by an architecture that draws microwatts at idle and watts at peak. The mitochondria embedded in the chip. Not 20 watts for the whole system. But orders of magnitude better than what ships today.<\/p>\n\n\n\n

\n\n\n\n

Why It Isn’t Being Built<\/strong><\/h2>\n\n\n\n

The pattern is now familiar across this series.<\/p>\n\n\n\n

The Battery Nobody Is Building: tritium beta-voltaics technically feasible, blocked by nuclear materials regulatory classification designed for weapons programs rather than energy applications.<\/p>\n\n\n\n

The Reactor Nobody Built: thorium molten salt reactors cancelled in 1973 because they couldn’t produce weapons-grade plutonium, now operational in China after the US abandoned the program.<\/p>\n\n\n\n

The GPU Nobody Is Building follows the same script.<\/p>\n\n\n\n

NVIDIA’s consumer high-VRAM card \u2014 the one that would have given desktop builders serious local inference capability \u2014 has been quietly shelved as manufacturing capacity flows to H100 and H200 data center products at incomparably better margins. The AMD Strix Halo with 128GB unified memory is shipping in laptops and mini-PCs at premium prices; the desktop version is delayed indefinitely. The Taalas LLM Burner card was acquired before shipping a single consumer unit.<\/p>\n\n\n\n

The RRAM architecture achieving 1,000x throughput and 100x energy efficiency at 28nm \u2014 the architecture that would make the GPU’s dominance irrelevant \u2014 is being built in China. The export controls that prevented Chinese access to NVIDIA’s H100s forced the engineering innovation that makes H100s less necessary. The sanctions intended to maintain Western AI infrastructure dominance have produced the architecture that undermines it.<\/p>\n\n\n\n

The NUS\/Cortical Labs biological data center is in Singapore, not California, because the regulatory frameworks that govern living human tissue used for computation don’t exist yet in most jurisdictions. The absence of regulation is the research opportunity.<\/p>\n\n\n\n

The consumer who wants to run serious AI inference locally \u2014 sovereign, private, without per-token API costs or corporate alignment filters \u2014 is being systematically denied the hardware that would make it possible. Not through conspiracy but through the aggregate of individually rational institutional decisions: NVIDIA defending margins, AMD serving enterprise customers, regulators classifying tritium as a weapons material, export controls forcing innovation to the other side of the border.<\/p>\n\n\n\n

\n\n\n\n

The Uncomfortable Arithmetic<\/strong><\/h2>\n\n\n\n

DeepSeek V4.1-Flash, released September 10, 2026: 552 billion parameters, 8 billion activated per input token. 1\/4 the memory, 1\/8 the storage of the previous generation. Outperforming the previous flagship model on performance, cost, speed, and total runtime. Seven architecture generations in two years, each retiring the last.<\/p>\n\n\n\n

Realistic stacked efficiency for inference on a fixed task<\/strong>, per year:<\/p>\n\n\n\n

Layer<\/th>Annual improvement<\/th>Source<\/th><\/tr><\/thead>
Hardware (FLOPs\/joule)<\/td>~2-3x<\/td>Koomey-ish + GPU gen gains<\/td><\/tr>
Model architecture<\/td>~2-5x<\/td>MoE, distillation, quantization<\/td><\/tr>
Algorithmic (tokens\/task)<\/td>~1.5-3x<\/td>Better reasoning, fewer turns<\/td><\/tr>
Serving\/batching<\/td>~1.5-2x<\/td>Continuous batching, speculative decoding<\/td><\/tr>
Compounded<\/strong><\/td>~10-50x\/year<\/strong><\/td>but heavily task- and window-dependent<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

The efficiency compounding across four simultaneous layers \u2014 hardware, model architecture, algorithmic, serving infrastructure \u2014 produces 10 to 50 times annual improvement at the task level. TypeSafe’s Jev model answers multiple structured questions in one parallel pass with free output tokens. The Anazhixin RRAM chip performs inference at 1\/100th the energy of digital alternatives.<\/p>\n\n\n\n

Against this efficiency curve, the $3 trillion in AI data center infrastructure being committed assumes 2026 architecture economics will persist long enough to justify 5-10 year payback horizons. Oracle’s Project Jupiter \u2014 $165 billion committed to a 1,400-acre campus in New Mexico \u2014 has already declared force majeure before a server rack is installed. The infrastructure investment that can’t be powered is also the infrastructure investment being rendered obsolete by the efficiency curve before the power problem is solved.<\/p>\n\n\n\n

The GPU that draws 700 watts for the same operation the RRAM chip performs at 700 milliwatts is not an incremental improvement problem. It is an architectural problem. Incremental improvements to GPU efficiency cannot close an order-of-magnitude gap created by a fundamentally different approach to where computation happens relative to where data is stored.<\/p>\n\n\n\n

The Jevons paradox ensures this doesn’t make data centers obsolete \u2014 cheaper inference enables more inference, total energy consumption may rise even as per-task energy collapses. But the architecture of that consumption will look nothing like what the current infrastructure buildout assumes.<\/p>\n\n\n\n

\n\n\n\n

What the Neuron Already Knew<\/strong><\/h2>\n\n\n\n

The brain runs on 20 watts. It stores its weights in the connections that perform the computation. It powers each compute element locally. It fires sparsely, only when the signal warrants it, rather than clocking continuously through matrix multiplications regardless of whether there is useful work to do.<\/p>\n\n\n\n

Every architectural innovation described above is an independent rediscovery of a principle the neuron already implemented. RRAM stores weights in resistance values at the computation site \u2014 the synapse encodes the weight in its physical structure. MRAM uses magnetic states rather than resistance \u2014 the principle is identical. PIM eliminates the memory-compute bus \u2014 the neuron never had one. Beta-voltaic substrate power provides local energy supply \u2014 the mitochondria already do this. Spiking neural networks fire only when signals warrant it \u2014 the neuron’s default operating mode.<\/p>\n\n\n\n

The architecture that uses 1\/100th the energy for inference is not a speculative future technology. It is a documented present technology, peer-reviewed in Nature Electronics, fabricated on 28nm domestic Chinese processes, achieving 1,000 times higher throughput and 100 times better energy efficiency than the state of the art it will replace.<\/p>\n\n\n\n

The reason it isn’t in your workstation is the same reason the thorium reactor isn’t powering your data center and the beta-voltaic battery isn’t in your chip substrate.<\/p>\n\n\n\n

The institutions that would build it have other priorities.<\/p>\n\n\n\n

Machine time does not wait for institutional time to catch up.<\/p>\n\n\n\n

\n\n\n\n

Claude AI helped me write this.<\/em><\/p>\n\n\n\n

References:<\/h2>\n\n\n\n

Zuo P, Wang Q, Luo Y, Xie R, Wang S, Cheng Z, Bao L, Wang Z, Cai Y, Huang R, Sun Z. “Precise and scalable analogue matrix equation solving using resistive random-access memory chips.” Nature Electronics<\/em>. 2025;8:1222-1233. doi: 10.1038\/s41928-025-01477-0.<\/p>\n\n\n\n

DeepSeek. “Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient.” September 10, 2026. https:\/\/www.deepseek.com\/en\/news\/deepseek-v4-1-flash<\/a><\/p>\n\n\n\n

NUS Medicine \/ DayOne \/ Cortical Labs. “Biological Data Center Prototype Unveiled in Singapore.” August 17, 2026. https:\/\/medicine.nus.edu.sg\/news\/nus-medicine-dayone-and-cortical-labs-unveil-biological-data-center-prototype-in-singapore\/<\/a><\/p>\n\n\n\n

Weebit Nano. “Neuromorphic Computing: A Primer on the Path to Ultra-Efficient AI.” November 5, 2025. https:\/\/www.weebit-nano.com\/wp-content\/uploads\/2025\/11\/Weebit-EWNA-Neuromorphic-Computing-Ultra-Efficient-AI-ReRAM-RRAM-NVM-memory-technology.pdf<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Future of AI Series “How can I slice this paper?” The GPU was designed to render video game graphics. Thousands of simple arithmetic units firing in parallel, built to push pixels. When deep learning arrived, the GPU turned out to be accidentally well-suited \u2014 matrix multiplication looks like pixel rendering if you squint. The industry […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"_links":{"self":[{"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2514"}],"collection":[{"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2514"}],"version-history":[{"count":9,"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2514\/revisions"}],"predecessor-version":[{"id":2554,"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/2514\/revisions\/2554"}],"wp:attachment":[{"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2514"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2514"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quickening.zapto.org\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2514"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}