GLM-5.3-Flash: 320B Total, 18B Active, and a Clean Jump Past GLM-5.2

Z.ai released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series, with weights on Hugging Face at zai-org/GLM-5.3-Flash under MIT. The listed sizing is 320B total parameters with 18B active, and the model was tested anonymously as ox-alpha on OpenCode and OpenRouter before release, where Z.ai reports it became the most popular model of the week.

GLM-5.2 was already well liked. The move to 5.3-Flash is not incremental on the reported numbers — DeepSWE v1.1 46.2 → 63.4 and AutomationBench 26.2 → 48.8 are the eye-opening deltas, with consistent gains across coding and agentic suites. The release is framed as frontier intelligence at flash cost, and as a recipe being scaled to a larger GLM-5.3.

Finding

GLM-5.3-Flash is a 45-layer mixture-of-experts transformer with a hybrid attention core and a new residual design, trained on a 30T-token multimodal corpus.

The specification from config.json:

• 45 layers, hidden size 4096, vocab 154,880, context 1,048,576 — layer_types is 34× linear_attention and 11× deepseek_sparse_attention in a 3:1 pattern (three linear, one sparse, repeated), with the sparse layers at 3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43.
• Linear attention: 64 heads, head dimension 128, short convolution kernel 4. Captures local dependencies through state modeling at constant per-token cost.
• Sparse attention: DeepSeek-style sparse attention with a lightweight indexer. The indexer selects relevant context via 32 index heads, head dimension 128, index_topk 2048, with indexer_rope_interleave enabled.
• IndexPool: At 1M context the indexer itself becomes expensive. Four indexer key vectors are compressed into one through weighted pooling (index_kpool 4, index_kpool_compress: true, index_kpool_always_select_tail: true, index_share_for_mtp_iteration: true), which Z.ai reports reduces indexer latency and memory while preserving retrieval.
• MoE: 288 routed experts + 1 shared expert, 8 active per token (num_experts_per_tok: 8, moe_intermediate_size: 2048), topk_method: noaux_tc, routed_scaling_factor: 2.5, norm_topk_prob: true.
• Residual: Manifold-Constrained Hyper-Connections (mhc: true, hc_mult: 4, hc_sinkhorn_iters: 20, hc_eps: 1e-6) to improve scaling efficiency. This replaces a single residual stream with a constrained multi-path connection that is Sinkhorn-normalized.
• Other: hidden_act: silu, dtype: bfloat16, first_k_dense_replace: 3, MLA-style kv_lora_rank: 512 with mla_use_nope: true.

Compared with GLM-4.5, which had a similar total count (355B) but 32B active and 92 layers, GLM-5.3-Flash nearly halves both active parameters (18B) and depth (45 layers) while keeping total scale (320B). Z.ai reports that versus the larger GLM-5.3, the Flash attention design reduces per-token attention compute by 3.0× and average KV cache per layer (BF16) by 4.4×, and that it has the lowest attention compute among DeepSeek-V4-Flash and Kimi-K3 in their comparison. KV cache remains slightly larger than those two, leaving headroom.

Training and data: Z.ai attributes part of the gain to a 30T-token multimodal pre-training corpus and to visual-coding data synthesis — pipelines that require the model to interact with environments, inspect its own renders, and iteratively refine them, plus RL with environment feedback and agent-based verification grounded in real user flows for frontend work. The line they use is direct: code lets the model build and change the world, vision lets it enter the world people see and use.

Benchmarks reported by Z.ai, all to be read as directional until independently reproduced:

• Coding / agentic vs GLM-5.2: Terminal Bench 2.1 81.0 → 84.3, DeepSWE v1.1 46.2 → 63.4, NL2Repo-Bench 48.9 → 56.3, Toolathlon Verified 59.9 → 78.4, AutomationBench v1.0.6 26.2 → 48.8, Agents’ Last Exam 20.4 → 26.3, HLE w/ Tools 54.7 → 55.3, GDPval-AA v2 1504 → 1773. Z.ai shows Flash approaching Opus 4.8 overall on these suites (e.g. Terminal 85.0, DeepSWE 58.0) and ahead on several agentic cuts.
• Intelligence per cost: Artificial Analysis Intelligence Index v4.1.1 — 57 at $0.045 per task (discounted), a level Z.ai frames as previously requiring roughly 10× the cost.
• In-house code bench: Z.ai Code Bench v1.0 on Claude Code 2.1.207 — Flash outperforms 5.2 at every effort level, and at max effort nearly matches Opus 4.8 (29.0 vs 29.5).
• Base model: GLM-5.3-Flash-Base (18B active / 320B total) versus GLM-4.5-Base (32B/355B) and GLM-5-Base (40B/744B) and DeepSeek-V4-Flash-Base (13B/284B) — MMLU 88.1, BBH 86.6, HellaSwag 87.1, LiveCodeBench-Base 37.6 (vs 28.1 / 34.4 / 29.9), SimpleQA 33.5 — competitive with the larger GLM-5-Base despite far fewer active parameters.
• Vision: OfficeQA Pro 62.4, CharXiv Reasoning w/ Tools 89.4, Chartography w/ Tools 78.0, with hour-scale video and document understanding claimed as native capability.

Serving: Z.ai reports a week of production serving of ox-alpha on a large-scale cluster of Chinese AI chips, on a serving stack built atop SGLang with a dedicated engine for this architecture. The stack uses intra-node tensor parallelism for linear attention and LM head, ReplaySSM, W8A8 quantization, hybrid INT8/FP8/BF16 cache quantization, Layer Split, and an Encode–Prefill–Decode disaggregated architecture across tens of thousands of accelerators. They report a 3× end-to-end improvement over their initial baseline on the same hardware, reaching per-token cost comparable to mainstream NVIDIA GPUs, and note the infrastructure agent powered by GLM-5.3 itself assisted in kernel development and bottleneck diagnosis.

Local path: weights are MIT, with BF16 and FP8 variants on Hugging Face, plus community GGUF via Unsloth and AtomicChat. Z.ai lists SGLang, vLLM, and TokenSpeed as currently supported for local deployment.

Meaning

The useful signal is that the same hybrid-attention + indexer pattern appearing in Qwen3.8-Flash appears here with a different tradeoff, and GLM pushes it toward cost rather than capacity.

Both families combine linear attention for local state with sparse attention for global retrieval through a lightweight indexer. GLM adds IndexPool — compressing four indexer keys into one by weighted pooling — specifically to keep the indexer viable at 1M context. Without that, the indexer’s own KV and selection cost reintroduces the long-context expense the hybrid was meant to remove. With it, per-head attention compute falls to the lowest among the models Z.ai compares, and KV per layer drops by more than 4× versus the larger GLM-5.3.

The second change is depth. Cutting from 92 layers to 45 while keeping total parameters near 320B is a direct statement about where scaling efficiency comes from. Fewer layers means fewer sequential steps per token, less KV to carry, and less coordination across the MoE. The residual fix — Manifold-Constrained Hyper-Connections with hc_mult: 4 and Sinkhorn normalization over 20 iterations — is the enabler. When depth is reduced but expert count stays large (288 routed), the risk is that information mixing across experts and layers becomes brittle. A constrained hyper-connection that explicitly controls routing of residuals across the manifold is consistent with training stability under sparse activation and short depth.

The practical consequence is local feasibility despite a large total count. 320B total is not 18B in storage — BF16 weights are on the order of 600+ GB across shards — but per-token work tracks active parameters and attention cost. At 18B active and 45 layers, with FP8 or GGUF quantization (Unsloth/AtomicChat already publish Flash GGUF), the model becomes runnable with tiered offload on high-memory workstations, and natively on 8× H200-class nodes without the 92-layer coordination overhead. That is why the Strix Halo question is interesting here: a 128 GB unified-memory system is not a home for a 45-layer 18B-active MoE at full 1M context, but with aggressive quantization, layer-split, and reduced context it is at the edge of what becomes testable — the same edge Qwen3.8-Flash and Qwen3.8-27B now sit on.

The visual-coding claim also matters for deployment. This is the first natively multimodal GLM-5, and Z.ai frames vision not as an extra input type but as part of the coding loop — render, observe, verify, refine. That requires the model to judge its own visual output and to accept screenshots and interaction traces as feedback. If that training holds, the benchmark gains on frontend, GUI judgment, and document work (OfficeQA, Chartography) are not separate from the coding gains — they are the same capability extended to where code meets a rendered surface.

Connection

GLM-5.3-Flash and Qwen3.8-Flash are two answers to the same bottleneck.

A year ago the constraint was per-token compute. MoE broke that by scaling total capacity while holding active steady. Then it was per-token attention on long context. Hybrid linear-sparse attention broke that by carrying local state in a linear recurrence and fetching only indexed global context. The next constraint became indexer cost at very long context and stable scaling of sparse, shallow models — which is where IndexPool and manifold-constrained hyper-connections sit.

The broader pattern is the same as with Qwen’s n-gram host-memory embedding and GLM’s disaggregated EPD serving: when arithmetic gets cheaper, the system moves to memory, bandwidth, and coordination. GLM’s reported 3× serving gain on domestic accelerators from compute-for-bandwidth and communication-for-bandwidth techniques, plus hybrid cache quantization, is that move in infrastructure form. The model helped optimize the system that serves it.

If the reported deltas over the already well-liked GLM-5.2 reproduce outside Z.ai’s harness — particularly DeepSWE and AutomationBench — the practical effect is not that a larger total wins, but that a shallower, sparser model with cheaper attention produces more intelligence per unit of work. Primitive implementation in the sense that 45 layers and IndexPool are still early, correct vector in that efficiency without capability loss is the direction both labs are converging on.