Exploring Next
Exploring Next — Ep 438 w/ Justy & Cody — The Infrastructure Behind Making Local LLM Agents Actually Useful | Towards Data Science
A conversation about making local LLM agents actually usable, focusing on the infrastructure challenges of running scientific agents with open-weight models. The hosts discuss the author's experience building a single-cell RNA-seq analysis agent, the problem of fixed prefix costs in long tool-use loops, vLLM optimizations for inference speed, and context management for long-running sessions.