On October 20, the Shanghai-based MemTensor team released and open-sourced MemT 1.0, a series of memory-native foundation models. The lineup includes MemT-1.0-Flash (288B parameters, 17B active, 336M dynamic memory) and MemT-1.0-Nano (36B parameters, 3B active, 10M dynamic memory). Weights and the technical report are available on GitHub, Hugging Face and ModelScope.

It is the first time a Chinese team has moved "memory" out of a bolted-on knowledge base and into the model's own computation. According to the team, the new models fold persistent, updatable internal memory into model computation, so that past information and task experience can be called on directly in later tasks without feeding the original history back in as text.

Why Today's Memory Is All Bolted On

An analysis published by Synced lays out the problem clearly: nearly every mainstream engineering approach to memory over the past two years has been external, leaving the model itself unchanged. Memory in large models today falls into roughly three categories.

Activation memory is the context window and the KV cache. It disappears when the conversation ends, and its cost grows linearly with history length. Parametric memory is baked into the model weights and can be invoked implicitly, but updating it means fine-tuning the whole model, and individual entries cannot be edited. External long-term memory lives in a knowledge base outside the model. It is editable and traceable, but the model has to retrieve from it to "remember" anything at all.

Read and write efficiency differs sharply across the three. That also explains why AI assistants today generally start every conversation from scratch: rules, methods and past mistakes sit in notes as text that the model can see at any time, but the weights never change, so each new round of inference begins from zero.

MemT 1.0 takes a different route. Instead of repeatedly injecting history into the context as text, it encodes that history as a persistent, updatable internal state that the model reads from and writes to during the generation of every token.

Parallel, Not Replacement: A Memory Path Alongside Attention

The architecture puts this inside the Transformer layer, running the existing attention mechanism in parallel with parametric memory. The current input travels the original path while a second path reads historical state through a memory query; the two results are fused before continuing through the rest of the computation. Information left behind by earlier interactions can therefore shape the generation of the current token.

Memory has two components: dynamic local memory, which holds compressed history, and hyper memory, which supplies projection and gating parameters. The key design choice is that new memory is written through forward computation, with no need to fine-tune the base model for each interaction — a new paradigm for updating a model's parametric memory.

As reported by The Paper and Xinmin Evening News, the architectural innovations fall into three areas. The first is a token-by-token stable update mechanism based on Gated DeltaNet-2, which merges new information into existing memory through erasure, writing and forgetting at the end of each interaction. The second is Mixture-of-Storage (MoS), which expands the memory module into multiple memory experts with dense routing, laying the groundwork for scaling memory capacity. The third is a two-stage training process of continued pre-training followed by post-training, which gives the model the ability to complete tasks on the basis of memory.

On training method, Synced reports that the team first prepared the base model with a three-track data pipeline, with the recipe settled after five full training runs and several hundred experiments. A teacher model with access to the complete context was aligned with a student model that could see only the memory state, using KL distillation and optimizing the memory parameters alone.

In deployment, the system binds memory state to a specific user or task trajectory, using serialization, saving and restoration to carry existing memory across sessions. External storage holds these states along with the source material, while the formation, updating and use of memory are handled by mechanisms inside the model.

Three Sets of Numbers, and the Caveats Attached to Each

On benchmarks, The Paper reports that across 11 memory benchmarks, MemT-1.0-Flash and Nano scored an average of 50.6 and 47.0 respectively, ranking first and second among parametric memory methods.

Worth noting: in the same analysis, Synced adds an important caveat. MemT 1.0's average score leads among parametric memory methods but trails MemOS, MemTensor's own text-based memory system. The first and second placings are rankings within the parametric memory subcategory, not an across-the-board lead over all memory approaches.

In coding agent tests, Xinmin Evening News reports that with the execution framework triggering experience writes at key checkpoints, Nano's average success rate across the 500 tasks in SWE-bench Verified rose from 66.07% to 69.73%, while the share of tasks solved on all three attempts climbed from 54.00% to 62.80%.

The gains are more visible in long-history scenarios. According to Xinmin Evening News, Nano delivered roughly 1.86x (128K history) and 3.99x (256K history) end-to-end inference speed compared with the full-context approach, with persistent memory state about 1/66 the size. The team also notes in its technical report that its advantage in time to first token (TTFT) widens as history grows.

What these three sets of numbers point to is a single conclusion: at a fixed configuration, continued interaction can keep updating the existing matrices, decoupling the size of a single session's persistent state from the growth of history length. The longer it runs, the less likely costs are to balloon.

Synced also walks through a concrete case. On the classic Django "parse_http_date" two-digit year parsing task, an agent without accumulated experience tends to locate the relevant code and start editing immediately, only adding reproduction and verification steps afterwards. In "memory-native" mode, the agent builds a reproduction test case before making any changes, and even revisits edge conditions after its first edit. The team takes this as evidence that the model's problem-solving sequence and verification habits have changed.