There were no executives from major tech companies on stage, but the room was packed with familiar faces from the open-source community.

A quick scan of the audience was enough to spot several star GitHub handles:

BBuf (Xiaoyu Zhang), a core developer of SGLang, one of today’s leading large-model inference frameworks;

Tang Zhengju, maintainer of TileLang, which is building the next-generation operator programming ecosystem;

Ma Teng, a core contributor to Mooncake, a tool for KVCache disaggregation and transfer;

Xiao Hang from the Beijing Academy of Artificial Intelligence, who has been deep in AI compiler work around Triton and FlagOS;

and hardcore developers such as R0CKSTAR, who is unusually active on GitHub.

What looked at first like an offline meetup for open-source geeks came with a striking twist:

the organizer was Moore Threads, one of China’s domestic GPU players.

That makes the whole thing worth watching.

In the past, when people talked about Chinese GPUs, the first keywords that came to mind were usually hardware specs, memory capacity, computing power metrics, ecosystem substitution, and whether models could run at all.

But the real question raised by the SGLang × MUSA Meetup was different:

How can Chinese GPUs truly enter the mainstream open-source engineering pipeline for large-model inference?

Put more directly, how can SGLang, Triton/FlagOS, TileLang, Mooncake, KVCache, P/D disaggregation, distributed communication, CI/CD, and upstream PRs all start revolving around Chinese GPUs?

After listening through the event, one impression was hard to miss.

Competition among Chinese GPUs is no longer just about chip specifications. It is moving toward a contest over ecosystem positioning.

Why? Let’s keep going.

Chinese GPUs Are Expanding Their Circle

Start with the meetup itself.

Its theme was clear: SGLang × MUSA.

SGLang is one of the most closely watched open-source frameworks in large-model inference serving today. Built for LLMs and multimodal models, it focuses on low latency and high throughput, and spans deployments from a single GPU to large-scale distributed clusters.

Frameworks like this matter because putting large models into use is no longer as simple as training a model and running it online.

Once a system enters production, it has to handle a full stack of complex problems.

For example: how to split prefill and decode, how to reuse KVCache, how to cut the cost of long context, how to reduce TTFT in multi-turn conversations, how to schedule large-scale clusters, how to provide day-0 support when new models are released, and how to trace a performance gap down to a specific kernel.

The first speaker was BBuf, a core developer of SGLang.

The open-source inference framework, which has 27,000 stars on GitHub, has become a top choice for developers worldwide deploying large models.

His SGLang 2026 Q2 roadmap hit several of the industry’s sore points:

full-stack optimization for DeepSeek V4, including W4A16 quantization, MegaMoE acceleration, and sparse attention support;

a full replacement of the traditional sgl-kernel with jit_kernel, using TVM-FFI to improve compilation speed by several times, eliminating the need to wait hours for wheel packages;

the full rollout of Vibe Coding, using AI agents to automatically analyze profilers, identify performance bottlenecks, and submit PRs, with more than 60 optimization tasks completed before May;

and a major upgrade to multimodal capabilities, supporting the latest models such as LTX2, Wan, and Hunyuan Video, with performance up to five times faster than other frameworks.

What stood out most was a set of data he showed.

Using a P/D disaggregated architecture, SGLang achieved 52.3k input tokens/s/node and 22.3k output tokens/s/node across 12 H100 nodes. That was five times cheaper than DeepSeek’s official API, and the result has already been reproduced by more than 10 teams globally.

Next up was Moore Threads contributor R0CKSTAR, who delivered the event’s most engineering-heavy talk.

He summed up the past six months of work in one sentence:

SGLang on MUSA has completed the full pipeline from environment setup to CI testing.

What does that mean?

Today, developers can clone the official SGLang repository, install sgl-kernel and sglang, and run almost all mainstream large models directly on Moore Threads MTT S5000 GPUs.

Popular models including DeepSeek, Qwen 3.5, GLM-4.5, FLUX, and Wan have all been deeply optimized.

He specifically highlighted MUSA’s three-layer CUDA compatibility stack.

In the past, adapting an inference framework meant changing thousands of lines of code. Now, adding a single line, import torchada, at the beginning allows 99% of CUDA code to run directly. That seemingly simple change addresses a major pain point in the Chinese GPU ecosystem.

According to available information, as of May 12, Moore Threads had submitted 47 PRs to the SGLang mainline, 41 of which had been merged, completing the full pipeline from environment setup to distributed inference.

Xiao Hang from BAAI then presented day-0 adaptation results for DeepSeek V4 on MUSA.

Through FlagOS’s Triton operator optimization and Moore Threads’ SQMMA tensor acceleration engine, they reduced DeepSeek V4’s first-token latency by 56.7% and increased throughput by 23%.

On this, Xiao Hang said:

We did not use any black magic. We simply optimized the two most critical operators to the extreme.

The FP8 matrix multiplication operator achieved an average 8.85x speedup, while the sparse attention operator achieved an average 6.01x speedup. Once the two operators that account for 80% of inference time were optimized, end-to-end performance naturally improved.

Tang Zhengju, maintainer of TileLang, gave attendees a look at the future of next-generation operator programming.

The project was open-sourced only in February 2025, but in just over a year it has gained 6,000 stars and 133 contributors. Even DeepSeek V4’s core kernels were written in TileLang. As Tang put it:

Writing FlashAttention in TileLang takes only 50 lines of Python code, with performance identical to expert-written CUDA.

And judging from the comparison chart he showed on site, the same GEMM operator reached CUTLASS-level performance with 15 lines of TileLang code, cutting code volume by 90%.

The final speaker, Alibaba Cloud’s Ma Teng, shared the latest progress on Mooncake.

The project, focused on KVCache disaggregation, has become a standard component in mainstream inference frameworks such as SGLang and vLLM.

He showed a particularly eye-catching set of numbers:

With RDMA P2P weight updates, synchronization time for the Kimi K2 1T model fell from 53 seconds to 7.2 seconds, a 7.37x speedup. The EPD three-tier disaggregated architecture reduced first-token latency for multimodal models by 6x to 8x. The HiCache + Mooncake backend pushed cache hit rates for multi-turn conversations above 90%.

At this point, the meetup’s full picture was essentially complete:

SGLang is the main inference framework pipeline; MUSA is the underlying Chinese GPU platform; FlagOS/Triton handles key operator optimization; TileLang lowers the barrier to high-performance kernel programming; and Mooncake fills in KVCache and production deployment.

Together, they form a relatively complete engineering pipeline.

Why Could Moore Threads Bring Them All Together?

The answer cannot be reduced to simply hosting an event.

The open-source community is practical. People showed up not because someone had a good story to tell, but because the work was genuinely relevant to the engineering problems they are solving.

Start with the original design intent behind MUSA.

Moore Threads CTO Zhang Yubo explained in the opening that MUSA stands for Meta-computing Unified System Architecture.

Meta-computing points to general-purpose computing. Moore Threads wants GPUs to embrace general-purpose computing as much as possible, rather than setting limits on future computable domains. Unified means Moore Threads wants its products to follow one unified standard, avoiding different instruction sets and architectures across product lines that prevent the software ecosystem from accumulating.

The more important point is this: MUSA does not want developers to relearn an entirely new stack just to use MUSA.

That sounds simple, but it goes straight to the pain point of China’s GPU ecosystem.

What do developers fear most?

Not new hardware itself, but having to learn a new set of APIs, rewrite piles of code, and still fail to get changes upstream, only to patch everything again when the community updates.

If a Chinese GPU ecosystem requires developers to relearn everything from scratch, it faces enormous migration resistance.

So MUSA’s route is to stay as close as possible to the GPU programming methods, APIs, and usage habits developers already know. The underlying implementation can differ, but the upper-layer experience should be as consistent as possible.

That is where the three-layer CUDA compatibility stack matters.

torch_musa connects PyTorch with MUSA’s basic capabilities; torchada keeps the CUDA-first ecosystem working; and mthreads-ml-py exposes device management, topology, memory, MTLink, P2P, and other information to upper-layer frameworks.

Put more plainly, Moore Threads is trying to extend the road developers already use to its own doorstep.

That directly affects whether open-source collaboration is feasible.

Upstream projects care most about low intrusion, maintainability, and reusability. If an adaptation requires large-scale changes to mainline code, every rebase becomes painful, and upstream maintainers are unlikely to accept it.

By contrast, if adaptation can be done in a more transparent way, PRs are easier to review and easier to keep aligned with community iteration.

That is the difference between maintaining your own branch and entering the mainline.

Then look at ecosystem integration.

SGLang × MUSA opens up the main inference pipeline.

Moore Threads began treating SGLang as a priority open-source project for integration and contribution last year. After more than half a year of work, the MUSA backend was recently merged into the SGLang mainline. Going forward, the company hopes not only to keep pace with features, but also to contribute more at the framework level.

The significance is that Chinese GPUs are no longer merely external adaptation targets for a framework. They are starting to become part of the mainline ecosystem.

FlagOS × MUSA is about key operators and new-model adaptation.

Performance competition in large-model inference is increasingly happening at the kernel, compiler, scheduling, low-precision, and communication layers. Work such as day-0 adaptation for DeepSeek V4 is essentially a test of response speed from model release to engineering deployment. Whether an ecosystem can run the model immediately, tune it quickly, and find better configurations on real shapes determines whether it can keep up.

Mooncake × MUSA is about inference disaggregation and production deployment.

The value of KVCache has grown further in the era of agents, multi-turn dialogue, and long context. The combination of Mooncake and MUSA is not just about running a cache backend on Chinese GPUs; it is exploring production-grade problems such as cross-instance KVCache sharing, elastic scaling, cache reuse, and in-place upgrades.

TileLang × MUSA is an early bet on the next-generation operator ecosystem.

If more future models and hardware require custom kernels, operator programming cannot remain forever in the hands of a small group of experts. The value of DSLs such as TileLang is that they turn high-performance kernel programming into an engineering tool more developers can actually use.

Taken together, these four lines explain why Moore Threads had the credibility to convene the room.

It has placed itself inside the real engineering network for large-model inference, spanning frameworks, operators, cache, communication, deployment, CI/CD, upstream work, and more.

And that is exactly the lesson China’s GPU ecosystem needs to learn.

Chinese GPUs Are Moving Toward a Collaborative Role

Seen from the broader perspective of computing power development, the value of this meetup may go far beyond the technical talks themselves.

Over the past few years, the ecosystem problem facing Chinese GPUs has been fairly obvious.

Many vendors were used to building behind closed doors: writing a deep learning framework from scratch, assembling their own operator library, and then finding few users because the result did not match mainstream developers’ habits.

Or some vendors would adapt a private fork and never submit code upstream, so whenever a mainstream framework updated, their adapted version became an unmaintained one-off.

Now, Moore Threads has offered a completely different answer:

fully integrate into the global open-source ecosystem and work with some of the smartest people in the world.

At the event, several terms came up again and again: day-0 support, upstream PR, and CI/CD.

That shows the role of Chinese GPUs in the ecosystem is changing in substance. Moore Threads is no longer satisfied with being a passive adapter. It wants to move proactively, become a contributor to core code, and even help co-build future architectures.

It is not merely throwing over isolated patches. It is deeply embedding a full engineering loop, including environment setup, PR submission, automated CI testing, release publishing, and documentation maintenance, into top-tier projects such as SGLang.

This sustainable upstream model is the real way to gain a voice in the ecosystem.

This open-source gathering also proved one thing: Chinese GPUs have now taken a seat at the public table of the open-source ecosystem for large-model inference.

At that table already sit the fast-rising SGLang, the Triton/FlagOS community pushing hard on low-level compilation, TileLang reshaping the operator ecosystem, and Mooncake leading disaggregated architecture.

Now, Chinese GPUs can also pull out a chair, sit down calmly, and play the most important hand of the large-model era alongside these star players.