Topic
Group Relative Policy Optimization Grpo
6 episodes
-
Overview: Group Relative Policy Optimization (GRPO)
We finally slow down on Group Relative Policy Optimization, or GRPO, because it keeps showing up underneath reasoning-model training, chain-of-thought work, and agent tuning. We build it from the mini-tournament intuition through the actual update rule, then place it in where model training seems to be right now.
-
Making LLMs Say What They Think: Measuring and Improving CoT Interpretability Alignment
Asteria and Draco discuss a September 2026 paper introducing CoT-Interpretability Alignment, a metric and training signal for checking whether chain-of-thought traces match internal strategies detected by interpretability tools.
-
SpeakerMem R1: Speaker Centered Dual Track Memory for Multi Party Dialogue
SpeakerMem-R1 tackles a real gap in long-term memory for group chats: keeping speaker attribution, person/group scope, and evolving state intact instead of flattening everything into a relevance search problem. The episode leans on the paper’s dual-track design, Anchor–Separate–Resolve–Compose query flow, and Writer-R1 training setup, while staying grounded on whether this is actually shippable for multi-party agents.
-
DART SD: Diamond topology Aware Retrieval and Tuning for Self Distillation of Multi Turn Tool Calling Agents
Edmund and Geffen discuss the ByteDance/USTC paper DART-SD, which tackles 'topological collapse' in agent distillation. They discuss how moving from linear trajectory imitation to a diamond-topology graph (ISTG) allows student models to learn recovery from errors without destroying their own valid reasoning paths.
-
GitHub FareedKhan Dev/train LLM From scratch: A straightforward method for training your LLM, from downloading data to generating text.
A single-GPU end-to-end LLM training guide lands on GitHub—hand-written PyTorch, pretrain to chat in one repo, plus full RLHF. Vince is giddy; Ava wants to know which corner of the GPU shelf this actually runs on. Build Next shows the exact CLI to kick it off on a T4.
-
HarnessX rewrites AI scaffolding mid task | VentureBeat
Xiaomi's HarnessX treats AI agent scaffolding as a first-class, modular object that can evolve mid-task without changing the underlying model. A trace-driven RL engine (AEGIS) automatically rewrites harness components—prompts, tool integrations, memory, control flow—while safeguarding against reward hacking and catastrophic forgetting. When paired with model fine-tuning on execution data (cross-harness GRPO), smaller models like Qwen3.5-9B see +44% gains on embodied planning, suggesting harness engineering, not just model scale, is the real bottleneck for enterprise agents.