Qwen3.8-27B: The Dense 27B That Carries the Max Lineage

Qwen released Qwen3.8-27B on Hugging Face, the dense counterpart to the Max-class Qwen3.8-2.4T-A95B. The repository contains 18 safetensors shards under Apache 2.0, with qwen3_5 architecture support in Transformers, vLLM, SGLang, and TokenSpeed. Qwen Cloud will serve the same weights with 1M context, vision, and built-in tools by default.

Following Qwen3.5 and Qwen3.6, Qwen3.8 is presented as the most capable generation in the Qwen open-model family to date. The 27B brings those gains to a single-GPU-class dense model rather than a data center MoE.

Finding

Qwen3.8-27B is a 27B dense causal language model with a vision encoder, built on the Qwen3.5 foundation.

The specification from config.json:

• 64 layers, hidden size 5120, vocab 248,320 — layout is 16 repeats of 3x (Gated DeltaNet → FFN) → 1x (Gated Attention → FFN), for 48 linear layers and 16 full attention layers (full_attention_interval: 4).
• Gated DeltaNet (linear attention): 48 value heads and 16 QK heads, head dimension 128, convolution kernel 4. Linear recurrence that carries state at constant cost per step.
• Gated Attention: 24 Q heads, 4 KV heads, head dimension 256, RoPE dimension 64, rope_theta 10M, partial_rotary_factor 0.25.
• FFN: intermediate size 17,408, SwiGLU, bfloat16.
• Context: 262,144 natively, extensible to 1,000,000 tokens. Trained with multi-token prediction (MTP).
• Thinking is flexible: thinking mode on by default with <think>\n...\n</think>\n\n delimiters. enable_thinking can be toggled per request, reasoning_effort: xhigh (default) / medium / low controls depth, and preserve_thinking: true by default retains reasoning context across turns for agent continuity and KV cache reuse. Instruct (non-thinking) mode uses temperature 0.7, top_p 0.80, presence_penalty 1.5.

Qwen’s reported benchmarks show substantial gains over Qwen3.6-27B across the same harness where the Max model was evaluated. Against Qwen3.6-27B on coding and agentic tasks:

• Terminal Bench 2.1 (Terminus) 63.4 → 73.0, SWE-bench Pro 53.5 → 61.7, NL2Repo-Bench 36.2 → 42.3, DeepSWE 1.1 13.3 → 42.2, QwenSWEBench 49.3 → 79.0
• CoWorkBench 61.0 → 70.7, JobBench 21.8 → 33.4, Agents’ Last Exam Pass@1 10.6 → 20.4 (Score 27.3 → 42.9)
• IFBench 69.1 → 79.5

Multimodal and vision intelligence also move: Vision2Web 45.0 → 62.9, SWE-MM 25.7 → 38.6, OmniDocBench 1.5 89.4 → 91.1, with hour-scale video understanding and STEM diagram comprehension claimed as native capability.

Results are from Qwen’s reported table versus Qwen3.6-27B, Qwen3.7-Plus, Muse Glimmer-30B, and Opus 4.6 Max on the same prompts, averaged over harnesses. They should be read as directional until independently reproduced, but the DeepSWE and QwenSWEBench jumps are the largest in the Qwen3 lineage so far for a dense model at this scale.

Recommended sampling is temperature 1.0, top_p 0.95, top_k 20 in thinking mode, with framework-specific tuning for vLLM, SGLang, and TokenSpeed.

Meaning

The signal in this release is that the Max architecture compresses.

Qwen3.8-2.4T-A95B demonstrated the hybrid pattern at data center scale: 512 experts, 10 routed + 1 shared, 3:1 linear-to-full attention, and mandatory thinking for long-horizon agents. Qwen3.8-27B keeps the same 3:1 hybrid — 48 Gated DeltaNet layers and 16 full attention layers — in a dense model that fits on a single H100 or a high-memory workstation with quantization. The 2.4T model is not runnable on a workstation at full context; the 27B is the intended local complement.

That matters for two reasons.

First, the linear attention ratio reduces the quadratic cost of long context without requiring expert parallelism or tiered KV storage. At 262K to 1M context, 48 linear layers carry state at constant per-token cost while 16 full layers anchor retrieval. Agent tasks that grow a shared prefix each turn — tool calls, environment feedback, multi-step coding — benefit directly from this without the MoE coordination overhead.

Second, the thinking control becomes practical. The Max model requires thinking on every turn; the 27B lets it be toggled and tuned per request with reasoning_effort and preserve_thinking. That allows fast per-turn responses at low or direct answers at enable_thinking: false when latency matters, and thorough analysis at xhigh when it does not. Qwen notes explicitly that lower reasoning effort does not always reduce total completion time for agents — insufficient analysis leads to more retries.

About deployment: 27B in bfloat16 is approximately 54 GB before KV cache. FP8 is approximately 27 GB. It fits on a single 80GB H100 at full context with cache headroom, or on a 48GB workstation GPU with quantization at reduced context. The SGLang cookbook at docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B and the vLLM and TokenSpeed recipes are the reference implementation; older framework versions lack the required Gated DeltaNet kernels. Native image and video inputs use the same chat templates as text, with frame sampling configurable in vLLM via fps.

Unlike the Max variant, this is Apache 2.0, not qwen3.8-max, with no additional use restriction.

Connection

Qwen3.8 makes the lineage plain.

Early scaling kept all parameters active on every token. MoE broke the link between total capacity and per-token compute. Linear attention broke the link between context length and quadratic attention cost. Qwen3.8 applies both at the top end and then distills the pattern into a dense model that keeps the second break while dropping the first.

The operational consequence is the same as with DiffusionGemma and WeatherNext seen this week: the bottleneck moves from arithmetic to memory and coordination. When per-token compute and per-token attention cost fall, the remaining constraint becomes what state can be retained across a long session. The 27B answers that by being small enough to keep the full session resident, with thinking that can be budgeted per step rather than fixed for the whole run.

If the reported agentic gains transfer beyond Qwen’s harness, the practical effect is not that a 27B replaces a 2.4T Max. It is that the Max pattern becomes usable without a data center — a single open model that carries long context, vision, and controllable reasoning to where workstation and single-GPU inference already exist.