Topic
Structured Output
34 episodes
-
Build more responsive apps with the Decisions API
OpenAI’s generally available Decisions API turns bounded model judgments into a dedicated, low-latency interface. The interesting claim is less that classification is new and more that routing, risk checks, and next-action choices can move into an application’s hot path when the output space is deliberately constrained and evaluated.
-
CTGT releases open source tool Concordance to trace why AI agents make inconsistent decisions
Jessica and Cathy unpack CTGT's Concordance, an Apache two-point-zero open source tool for tracing where AI agent workflows start making inconsistent decisions.
-
Decision models like Jev handle what SQL rules cannot. Here's how data teams use them
Jessica and Cathy dig into decision models — specifically TypeSafe AI's Jev — and the argument that structured, typed ML judgments are the missing middle layer between SQL rules and full LLM calls in data pipelines.
-
NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning | NVIDIA Technical Blog
NVIDIA drops Nemotron 3.5 Super VL — a 120B-parameter open-weight multimodal MoE model built for specialized enterprise agents. Jessica and Cathy dig into the real argument: post-training on your own data beats the frontier on your specific task, and the efficiency story (40x inference cost reduction, 10x latency) is where the product actually lives.
-
The Roadmap to Mastering Agentic AI Systems in 2026
A practical read of a four-stage agentic AI roadmap: the valuable progression is from typed tool calls and durable state to observable, recoverable workflows, not from increasingly ornate agent personas to multi-agent theater.
-
How to Build a Model Router in the Harness
LangChain's Open SWE team built a model router inside their agent harness that cut median cost per coding task by 64% with no measurable quality drop. The key insight: routing belongs in the harness, not a generic gateway, because the harness already has the task context (prompt, tools, domain knowledge) needed to pick the right model. They classified tasks via LangSmith traces, picked three models on the Pareto frontier (GLM-5.3-Flash, GPT-5.6 Sol, GPT-6 Astra), and validated with A/B tests on 973 threads — merged PR rate held at ~29% while median cost dropped from $2.61 to $0.94. 56% of routed threads went to the balanced tier, 34% to fast, only 10% to performance. The classifier now runs on Jev, a new decision model that's 50x faster. Mid-flight routing and router evals are the stated next steps.
-
Automating Knowledge Graph Population: Extracting Entities and Triples from Unstructured Text with an LLM MachineLearningMastery
Pippa and Tyler dig into a practical recipe for turning unstructured text into knowledge-graph quads with a local LLM, but keep the real question in view: whether the extraction pipeline is useful enough to trust, not just easy to demo. They focus on the article’s SPOC framing, JSON-mode prompting, and QuadStore loading, while poking at where this kind of setup gets brittle in the real world.
-
Jev offers a cheaper way to make AI decisions | VentureBeat
Jev’s real innovation is treating many enterprise AI calls as typed control decisions instead of text generation. Onyx and Echo examine its speed, price, calibration claims, open reproductions, and the practical limits of replacing LLM calls with classifiers.
-
Jev in the Agent Loop: A Complete Guide to Decision Layer Automation
Justy and Cody dig into "Jev in the Agent Loop," arguing the real move isn’t smarter models, but cheaper, structured decision-making—replacing frontier-priced LLMs on every internal fork with Jev. They debate where Jev actually fits, what it can’t do, and why this reframes agent infrastructure as cost optimization at the decision layer, not generation. Cody pushes back on the limits: Jev can’t generate, so it only works in pipelines where you can cleanly separate choosing from doing. Justy sees immediate product wins in agent tooling, and they swap examples and a little banter on what gets hyped as "agentic" when it’s really just moving decisions around.
-
Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
Jessica and Cathy dig into Adversarial Review, a minimal reviewer–critic protocol for agentic code review that beats bigger multi-agent setups by forcing disagreement to stay structured and evidence-grounded.
-
Can Jev Be a Better Agent Evaluator?
LangChain's evaluation of TypeSafe AI's Jev as an agent evaluator reveals a fundamentally different architecture — not an LLM that generates text, but a 'System One' model that returns typed decisions with calibrated probabilities. In a narrow test against GPT-5.6 Luna, Terra, and Claude Sonnet 4.6, Jev matched human oracle accuracy on binary pass/fail decisions (100% vs. 80–99.8%), achieved 92–913x lower variance on continuous scoring, and cost $0.00035 per call versus $28.17 for Claude. The implication: agent evals may have a third viable path beyond code-based (narrow, deterministic) and LLM-as-judge (slow, expensive, non-deterministic). The catch: this is one narrow test on five weather requests, and low cost can amplify mistakes at scale.
-
Introducing System One Models & Jev TypeSafe AI Blog
TypeSafe AI launches Jev, the first 'System One' model: no text generation, but blazingly fast, type-safe, parallel, probabilistic decisions for automation at the code layer. Jev claims 40–200x speedups versus LLMs like GPT-5.6 Terra, with zero type errors and per-decision calibration, aiming to finally unlock real-world automation where string output can't go. This episode dives into how Jev works, what it means to give up freeform text, and whether TypeSafe’s new architecture marks the start of a real post-LLM automation wave.
-
What Is Jev? A Guide to TypeSafe AI’s System One Model
Masonry and Eyre dig into Jev as a different kind of model bet: not a chatty LLM replacement, but a fast structured-decision layer for agent loops. They focus on the real product angle, the mechanics behind parallel typed questions, and where the claims are strong versus still a little hand-wavy.
-
Enabling Creative Explorationfor Vibe Design Agents
Vince and Ava dig into a paper about separating design exploration from code generation in vibe design agents. They focus on why temperature is too blunt, how structured design specs plus external selection give you controlled variety, and what the offline and online results actually say about usefulness versus uncertainty.
-
Your LLM Can Return Perfect JSON and Still Be Wrong
A real-world trap in Structured Outputs: enforcing schema validity does not guarantee data truthfulness. When a required field is missing from source text, the model invents a plausible value instead of returning null, producing type-correct but false data. The fix requires three layers: nullable fields to allow absence, evidence fields to show provenance, and post-parse validators to catch nonsense values. The essay walks through a payment-reconciliation pipeline where 2–3% of transactions had fabricated dates, caught only downstream.
-
Overview: Dynamic Code Execution
We finally slow down and explain dynamic code execution from the ground up — what it actually is, how the loop works, why it makes models meaningfully more capable, and where the real costs and failure modes live.
-
Recursive Experiential–Working Memory Evolution for Long Horizon Agent Harnesses
Talon and Wildflower unpack Recuris, a paper proposing a verified working-memory layer that selects skills from experiential memory during long-running agent tasks, then uses structured traces and held-out validation to patch the harness across tasks. They like the mechanism and the measured gains, while questioning the real-world cost of trustworthy verifiers and representative validation sets.
-
Introducing the Admin Plugin for ChatGPT Work and Codex
Justy and Cody debate OpenAI's new Admin plugin for ChatGPT Work and Codex, weighing its practical value for workspace admins against the risk of smoothing over messy permission and governance edge cases.
-
Codex as a platform: build on the open agent harness | OpenAI Developers
OpenAI open-sources the Codex harness—the execution layer behind Codex app, CLI, and IDE extensions—so developers can embed agent loops into existing products instead of forcing users into a generic chat box. The harness handles context, tool use, approval flows, and sandboxing; developers control the interface, business logic, and integration. Real implementations already shipping: GitHub, JetBrains, Cisco, tax-prep workflows. Justy sees a product win (agent-native integration beats chat-box retrofit); Cody questions whether the harness abstraction actually reduces integration complexity or just moves friction around.
-
1Aggregate results comparing DFM Mimir 1B against the HRM Text 1B, Qwen 3.5 2B and Gemma 4 E2B, displaying highly competitive performance across 20 benchmarks.
Pippa and Tyler examine DFM Mimir v1, a one-billion-parameter HRM trained from scratch with permissible post-training data, and ask whether its strong benchmark results make it genuinely useful for low-resource language work.
-
Why managed agents are the next big thing in agent building
Jessica and Cathy dig into Harrison Chase’s argument that “managed agents” are the next stage of agent building, where harness plus infra are bundled and driven by emerging standards like AGENTS dot md, MCP, and skills. They unpack what problems this actually solves in production, how LangChain’s Managed Deep Agents fits into their running harness-as-product thesis, where the piece is strongest, and what’s still underspecified.
-
Claude, Explained: Agents, Loops, and Graphs
Jessica and Cathy dig into a post that tries to boil Claude work down to agents, loops, and graphs. Cathy likes the structure but pushes back on the article’s tendency to make every layer sound universally useful, while Jessica argues the real win is that it gives people a practical ladder instead of vague agent hype.
-
Agent frameworks vs the AI Backend — AgentField Docs
Justy and Cody unpack AgentField's claim that agent frameworks are for authoring behavior, while an AI Backend is for running agents as production services.
-
Graph Based Agentic AI with LangGraph: Workflow Pathways for Long Running Stateful Business Processes
Cooper and Miles dig into a practitioner paper on LangGraph as a control-plane for long-running business workflows, not a benchmark toy. They focus on the three recipes in the paper—SQL repair loops, evidence-gated RAG, and human-in-the-loop policy review—and on when a graph is actually worth the extra structure.
-
eve – The Agent Framework Vercel
Jessica and Cathy dig into Vercel's eve, a filesystem-first framework for durable AI agents, and why its boring production defaults may matter more than the agent hype around it.
-
Overview: Structured Output
We slow down and explain structured output from the ground up: why free-form model text is awkward for software, how schemas and constrained decoding make it usable, and where the format guarantee stops.
-
Building Knowledge Graphs with Claude: A Prompt Based Engineering Playbook
Anthropic's knowledge-graph engineering playbook replaces classical NLP pipelines (trained NER, relation classifiers, entity-resolution heuristics) with a sequence of Claude API structured-output calls. The entire extraction-resolution-assembly-querying loop becomes prompt-based, scaling from Haiku (high-volume extraction) to Sonnet (reasoning). The graph serves multi-agent systems as shared memory, grounding layer for evaluator-optimizer loops, and persistent world model across sessions. The paper maps this onto Anthropic's five canonical agent patterns and reports precision/recall against a gold set.
-
Think through hard problems in voice mode | Claude by Anthropic
Asteria and Draco dig into Anthropic’s update to Claude voice mode, where Opus and Sonnet now power spoken sessions, connected tools are usable from voice, and multilingual support expands. They focus on the real argument: voice mode becomes useful when it’s no longer just fast chatter, but a place to work through half-formed thinking and then hand off to action. They also question where the feature stops being a convenience and starts being a real workflow, especially given model switching, permission prompts, and the different value between free and paid tiers.
-
12 Ways to Reduce LLM Latency and Inference Costs in Production KDnuggets
A practical KDnuggets piece argues that most LLM production latency/cost gains come from cutting unnecessary work instead of bigger models or more GPUs. They list 12 levers: measure the right metrics, cut output tokens, route to smaller models, collapse LLM calls, prefix caching, add multiple cache layers, control RAG context, batch offline work, tune batching for user latency, and manage KV cache. Tyler pushes back on the article’s overgeneralization of cache reuse across all tasks, the thin technical depth behind some tips, and the implication that routing to small models never backfires. Pippa highlights the piece’s strongest point—measuring TTFT, P95/P99, and queue time—because that’s where teams most often mis-diagnose bottlenecks. They land on: the article’s monitoring advice and batch-tuning guidance are solid; several recommendations work only for read-heavy workloads; and routing to tiny models is risky until you have cheap, high-confidence evaluators. They wrap with a Build Next command to try vLLM continuous batching and two open-source RAG-caching projects (Harmonia and From Prefix Cache to Fusion RAG Cache).
-
Overview: Constraint Verification
We keep running into constraint verification in different forms, so we finally sat down and made the idea click from the ground up. We talk through how checking rules, schemas, and hard boundaries works in AI systems, and why that gatekeeper layer matters so much.
-
CrewAI Review 2026: Features, Pricing, Pros & Cons
A casual chat about CrewAI, a multi‑agent platform, weighing its promise against real‑world practicality, pricing, and use cases.
-
ChatGPT Work: Turning Chat Into an Execution Layer for Business Tasks
Asteria and Draco dig into OpenAI's ChatGPT Work page and land on the real argument underneath the product gloss: this is OpenAI trying to turn ChatGPT from a chat surface into a work execution layer that can pull context from business tools, choose an output format, and keep multi-step projects moving under human approval. They like the product direction more than the evidence on the page, with Draco noting the article mostly shows polished scenarios rather than hard proof, and Asteria arguing the practical audience is obvious anyway: teams drowning in scattered context and repetitive document assembly.
-
Overview: Tool use and function calling
We finally sit down and make tool use and function calling click from the ground up. We keep coming back to the same idea: a model can draft the request, but something outside it has to actually do the thing.
-
The No BS Guide to Build a Context Graph
Najmuzzaman's 'No BS Guide to Build a Context Graph' lays out how a production context infrastructure moves beyond CRM data to capture decision traces, organizational patterns, and lived heuristics. The real breakthrough is surfacing context and patterns that aren't captured in playbooks, making organizational intelligence accessible and explainable. The hard technical work: entity resolution, permissioning, and evolving patterns transparently.