Topic
Linear Attention
4 episodes
-
“Take work away from frontier LLMs”: Subquadratic’s decision model claims 12x Jev’s context window
Subquadratic launches SubQ Snap, a decision model with 384K-token context window — 12x Jev's 32K limit — claiming it can process 100K tokens and return a decision in ~1 second. Built on Subquadratic Sparse Attention (SSA) for linear attention scaling, Snap targets enterprise agent workflows: file selection, tool routing, test verification. Benchmarks show Snap beating Jev on 13 of 16 enterprise tests (Decision LCR 74 vs 48, Nimble 82.9% vs 76%), though Jev wins on When2Call. All results are self-reported; independent verification pending. Pricing at $0.05/M input tokens, free outputs. The real question: does a 384K window actually eliminate retrieval/chunking fragility for coding agents, or just push the bottleneck elsewhere?
-
Making LLMs Say What They Think: Measuring and Improving CoT Interpretability Alignment
Asteria and Draco discuss a September 2026 paper introducing CoT-Interpretability Alignment, a metric and training signal for checking whether chain-of-thought traces match internal strategies detected by interpretability tools.
-
Kimi K3 Kimi API Platform
Two friends unpack the Kimi K3 API docs, debating its 1M‑token claim, hybrid attention, and tool dynamics, and weigh who should pay the price for the hype.
-
Morphing into Hybrid Attention Models
Talon and Wildflower discuss FlashMorph, a new method for choosing which Transformer layers should keep full attention when converting pretrained LLMs into hybrid attention models.