SenseTime open-sourced its experimental SenseNova-Loop-9B-Preview language model on Friday, introducing a recurrent architecture that reuses physical weights during inference. The release makes the model weights available under the permissive Apache 2.0 licence, allowing developers and researchers to download, modify and deploy the system for commercial and academic applications.
Key points
- SenseTime released SenseNova-Loop-9B-Preview under an open-source Apache 2.0 licence.
- The architecture reuses a 24-layer middle block twice to deliver 56 computation passes from 32 physical layers.
- A two-stream mHC design supports information flow across repeated layer executions.
- The release tests whether looped weight reuse can lower memory requirements while retaining model reasoning depth.
How looped execution works
Standard transformer language models process input tokens through a sequence of distinct layers. Each layer contains its own weights, meaning that depth requires a proportional increase in physical parameters and memory storage. SenseTime designed SenseNova-Loop-9B-Preview around a looped transformer mechanism that alters this progression.
The system contains 32 physical layers in total, but it routes tokens through 56 separate layer passes. It accomplishes this by taking an internal block of 24 layers and running inputs through that block a second time before passing activations to the final stages of the network. The initial four layers and final four layers execute once, while the central core executes twice.
This design allows the model to simulate the computation depth of a substantially larger model while storing only nine billion parameters in memory. By reusing the weights of the middle section, the network applies additional processing steps to complex representations without requiring the memory bandwidth needed to fetch fresh parameter matrices from high-bandwidth memory for every single layer pass.
Two-stream routing with mHC
Recurrent execution in deep networks often faces degradation problems. When the same layers run multiple times, signal representation can drift or suffer from vanishing gradients during training. SenseTime addressed this by incorporating a two-stream mechanism designated as mHC alongside the looped layers.
The two-stream approach separates distinct information pathways as tokens move through the repeated block. One stream maintains raw context and identity across iterations, while the second stream applies transformation steps. This separation prevents the repeated operations from distorting baseline context, stabilising token representations as they re-enter the 24-layer middle block for their second pass.
By splitting processing into complementary streams, the model retains features extracted during the first pass while refining intermediate conclusions on the second pass. The architecture aims to give the model iterative reasoning capabilities typically seen in larger parameters classes, without the operational footprint of maintaining distinct physical weights for all 56 steps.
Why parameter reuse matters for inference
Memory capacity and memory bandwidth remain the primary bottlenecks for serving artificial intelligence systems on modern hardware. Running an open-weight model locally or on modest server clusters requires sufficient video memory to hold all active weights. When weights must be transferred repeatedly to computing cores, performance drops.
A nine-billion-parameter model with recurrent middle layers changes the economic equation of deployment. Because only 32 physical layers reside in memory, the static footprint matches standard nine-billion-parameter configurations. Hardware needs only to cache those weights once, then feed activations through the central 24 layers a second time while keeping those same weights active in local caches.
For developers running inference on consumer graphics cards or cost-constrained cloud instances, this structure offers deeper computational processing without increasing parameter storage. The trade-off shifts from memory capacity to compute time, trading additional arithmetic clock cycles for reduced memory consumption.
Open licensing and next steps
SenseTime published the model weights under the Apache 2.0 licence. This decision allows commercial integration without proprietary licensing fees, reflecting a broader trend among Chinese developers releasing competitive open models to attract software builders and research contributions.
The preview status indicates that SenseTime is testing public reception and gathering community feedback on recurrent transformer performance. Independent benchmarks will determine how closely the 56-pass execution matches the output quality of native 56-layer models with unshared weights across coding, reasoning and general language tasks.
Engineers and researchers can inspect the model configuration directly to evaluate how the recurrent loop interacts with standard inference engines. SenseTime has not yet published fine-tuning recipes or announced when a non-preview release will follow.
