GPU cluster inference has hit a performance ceiling, and brain-inspired chips are stepping in to supercharge it.

The 2026 World Internet Conference ran from November 2 to 5 in Wuzhen, where the Light of the Internet Expo announced its ten flagship debut achievements. Among them was the "brain-inspired super node for FFN acceleration in high-performance large-model inference," jointly submitted by Beijing Lynxi Technologies, China Mobile (Suzhou) Software Technology, and the CETC Nanhu Research Institute.

What the system does is straightforward enough: pair GPUs with brain-inspired chips, leaving Attention computation to the GPUs while the brain-inspired chips handle inference for FFN and MoE expert layers.

Understanding why that is worth doing starts with seeing where GPUs actually waste effort during inference. The heterogeneous hybrid inference system, built on domestic GPUs plus brain-inspired chips, made its first public appearance back in September at the 2026 China Computing Power Conference.

The Cost Problem Is Mismatch, Not Compute

Why does large-model inference keep getting more expensive? The answer points to misallocated resources.

Take the Transformer architecture. Every layer contains two core modules, Attention and FFN. Attention handles context, repeatedly reading the KV Cache, the stored history of context already computed, which puts heavy demands on memory capacity and bandwidth. FFN loads model weights to run matrix computation and carries the bulk of the compute load.

The two jobs want different things from hardware, but today's mainstream GPUs have to cram both onto a single card. That creates two shortfalls.

First, the memory wall inherent to von Neumann architecture: compute capability far outstrips the rate at which data can be supplied, so utilization stays low and power is burned for nothing. Second, the GPU's dynamic scheduling mechanism, which enables parallel computation but also introduces latency uncertainty, pushing latency higher.

Under MoE architectures the problem is sharper. A model may have a large number of experts, but each token activates only a fraction of them, scattering compute across different experts. FFN cannot form large enough batches, and some GPU capacity simply sits idle.

From P/D Disaggregation to AFD

If one card cannot do two jobs, split them apart. That is exactly the path large-model inference architecture has taken.

First came P/D disaggregation, assigning prefill and decode to separate compute resources. The next step is AFD, or Attention-FFN Disaggregation, which separates the two modules at the layer level, deploying the Attention and FFN portions of each Transformer layer independently.

This has become a direction the global industry is exploring. In December 2025, NVIDIA brought Groq and its in-memory computing LPU technology into its fold; in March 2026 it released its first Rubin GPU and Groq LPU hybrid cluster design, expected to reach the broad market by the end of this year. Each Groq LP30 carries 500MB of compiler-managed on-chip SRAM, keeping weights and activations on-chip as much as possible.

In China, Lynxi Technologies has long worked on brain-inspired chip design and its application to large-model inference. Its KA200 uses an on-chip in-memory computing, event-driven architecture, a technical approach closely aligned with the Groq LPU. According to the report, Lynxi proposed and began development of it no later than NVIDIA did.

GPUs Handle Attention, Brain-Inspired Chips Handle Feed-Forward

The core shift in Lynxi's system is letting each type of chip do what it does best.

GPUs take on prefill, building and maintaining the KV Cache, and Attention computation, playing to their strengths in parallel compute, large HBM memory, and a mature software ecosystem. The brain-inspired super node handles inference for FFN feed-forward networks and sparse MoE expert layers, leaning on the KA200's in-memory computing, event-driven design, and high on-chip bandwidth.

The KA200 is built on a 12nm process and has three defining features.

The first is on-chip SRAM in-memory computing. Conventional GPUs shuttle data back and forth between memory and compute units; the KA200 parks weights right next to the compute units, so data does not have to travel far.

The second is the event-driven mechanism. Compute resources are scheduled according to input events, and with no event there is no work, which cuts wasted effort.

The third is static compilation. Instruction and data paths are planned in advance for zero runtime scheduling, which makes latency deterministic.

Wiring Two Chip Types Together Is Harder Than Dividing the Work

Dividing up the tasks is the easy part on paper. Joining two architectures in practice is considerably harder.

Lynxi's approach: three to six GPU servers paired with three brain-inspired super nodes form one system, with each super node built from thousands of interconnected KA200 chips. The FFN operators originally running on the GPU side are offloaded and deployed to the brain-inspired side; once computation finishes there, the output is written back to the GPU side to feed into the model's Attention-related computation. Both sides drive the inference pipeline together, with the brain-inspired side serving as an FFN service node.

Notably, the FFN portion uses a static compilation method. Once compilation is done, the on-chip SRAM deployment across each chip in the super node is fixed, making data pipelining and latency deterministic and progressively eliminating latency instability and random jitter.

Three in-house pieces underpin this coordination: a model compiler, a high-speed interconnect protocol, and a unified heterogeneous inference engine. The hardware is built from HP640 series brain-inspired compute boards and LynxLink high-speed switches, with software running a heterogeneous scheduling engine and unified memory address compilation technology.

The system natively supports mainstream development frameworks including PyTorch, DeepSpeed, and Megatron-LM, along with a range of inference engines, so models need no complicated rework. The FFN acceleration capability is packaged as an independently deliverable compute unit that is hardware-agnostic and model-agnostic, able to be networked directly alongside existing GPU servers and suited to the air cooling conditions of current AI data center racks.

Set against NVIDIA's Rubin GPU plus Groq LPU design, Lynxi is pursuing a domestically developed route to heterogeneous inference. It does not ask GPUs to leave the inference system; it adds dedicated brain-inspired compute nodes that act as a turbocharger bolted onto the GPU.

Tens of Thousands to 100,000 Tokens per Second, and Then What

Whether an architectural idea holds up comes down to what it actually produces.

The Lynxi team has run heterogeneous inference tests on the DeepSeek V4 Flash model. With context lengths of at least 32K, three to six A800 GPU servers paired with three brain-inspired super nodes delivered system output throughput ranging from tens of thousands to 100,000 tokens per second.

On a like-for-like investment basis, according to the project materials, the GPU plus brain-inspired super node setup more than doubled both inference throughput and energy efficiency, while cutting overall operating costs by more than 40%. The system has also completed commercial validation with mainstream large models at a leading telecom operator's AI data center.

There is still room to climb. DeepSeek V4 Flash uses an MoE architecture in which each token activates only 6 of 256 routed experts, and layering on mixture-of-experts optimization and model quantization should further reduce hardware requirements. With both optimizations stacked, the project team is targeting system output throughput in the hundreds of thousands of tokens per second.

The Case for Reusing Existing Hardware

From an industry standpoint, the design carries a value that is easy to overlook: making use of what is already installed.

Many AI data centers have already deployed large fleets of GPU servers, and some of those installed cards lag behind the latest high-end products in performance. Lynxi has already integrated with earlier-generation GPUs from Iluvatar CoreX, more than doubling both inference output and inference energy efficiency.

Looking ahead, the FFN brain-inspired super node could connect to different types of GPUs, forming more flexible heterogeneous computing clusters. For domestic GPU makers and AI data center operators, that opens a new path to expanding capacity while lowering costs.

From competition over single-chip process nodes to system-level coordination between GPUs and brain-inspired chips, China's computing power sector is hunting for new performance headroom. As large-model inference moves into large-scale deployment, competitiveness now hinges on hardware division of labor, data movement, cross-architecture scheduling, and system energy efficiency.

Lynxi's super node making the list of ten flagship debuts signals that China's heterogeneous inference route is moving from technical validation toward standardized products. If the idea of letting different architectures each play their own role pans out, token factories will depend somewhat less on pure GPU architectures and advanced HBM.