Exploring Next
Exploring Next — Ep 460 w/ Justy & Cody — MemTrain: Self-Supervised Context Memory Training
Self-supervised framework MemTrain improves LLM context memory by training on unlabeled Wikipedia with coupled proxy tasks—masked reconstruction and memory recall—using GRPO. Achieves up to 17.67-point gains on long-horizon reasoning without task-specific labels.