On September 15, Infinigence AI, Tsinghua University and Shanghai Jiao Tong University formally open-sourced APXInf, an on-device inference engine for embodied AI aimed at model inference, performance tuning and engineering deployment on robots themselves.

APXInf launches with support for two embodied models, π0.5 and WALL-OSS, and is adapted to computing platforms including the RTX 4090, Jetson Orin and Jetson Thor, spanning model development, validation and deployment on the robot.

Compared with general-purpose inference frameworks built for cloud-based large models, APXInf targets the constraints of robotics: limited on-device computing power, tight power budgets, demanding real-time requirements and complex model adaptation.

Robots Need Lower Inference Latency On-Device

Embodied models can already interpret environments and tasks from visual and audio inputs and generate robot actions. But the full loop, from sensing the environment to model decision-making to physical execution, has to close within a short window.

For a cloud chatbot, a few hundred milliseconds of latency may only slow down a reply. For a robot that requires continuous perception and real-time control, the same delay can mean lagging movements, broken trajectories, or outright task failure.

Robots are also bound by local chip performance, memory capacity, battery life and thermal limits. An on-device inference system therefore has to deliver low latency, stable operation and efficient hardware utilization all at once, within a tight resource envelope.

π0.5 Inference Latency Cut to Under 26 Milliseconds

Infinigence AI says that when running the π0.5 FP8 model on Jetson Thor, APXInf reduces end-to-end inference latency from 278 milliseconds to under 26 milliseconds, reaching an FP8 inference rate of 38.46Hz.

The team argues that this level of performance meets the inference frequency and response latency needs of a range of real-time robot control scenarios. The figures come from the project team's own testing, however, and real-world results will vary with robot hardware configuration, model version, task type and operating environment.

APXInf is not a straight port of a cloud inference framework onto a robot. It systematically optimizes how models run for on-device conditions: small-batch computation, continuous inference and real-time interaction.

Connecting Model Training to On-Robot Inference

In September 2025, Infinigence AI and teams including Tsinghua University open-sourced RLinf, a large-scale reinforcement learning training framework for embodied AI used for post-training and evaluating embodied models.

APXInf extends that technical chain to on-device inference and robot deployment. Developers can handle model training, validation, inference optimization and on-robot operation within a single open-source ecosystem.

In effect, RLinf addresses how embodied models are trained and evaluated, while APXInf handles getting trained models running on a robot's local hardware. Together they form an engineering pipeline from cloud training to physical execution.

Domestic Chips and Robot Operating Systems Next

According to the public roadmap, APXInf will add support for model types including VLA, VLM and world models, with adaptation work on Qwen and GR00T currently underway.

The project team also plans to keep optimizing key open-source models, expand its performance optimization list, push forward NVFP4 compute performance tuning, and adapt to mainstream chips such as AMD's.

On the domestic front, APXInf plans to support Chinese compute chip backends and Chinese robot operating systems, widening the range of robot bodies and computing platforms it can run on.

Whether APXInf achieves adoption at scale as an open-source project will depend on how quickly it adapts to new models, how stably it runs across hardware, the developer community it builds, and its long-term performance on real robotic tasks.