MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM Powered Assistants

Izzo Your AI assistant is about to stop giving you walls of text and start building you actual apps.

Izzo You're listening to Exploring Next, episode two-twenty-three. I'm Izzo, and Boone's here to walk us through some genuinely exciting research that could flip how we think about AI interfaces.

Boone Yeah, this MiniAppBench paper hit different. Instead of asking GPT for a paragraph about mortgage calculations, you get a working calculator with sliders and real-time updates.

Izzo Right, and that shift from text to interactive HTML — that's not just a nice-to-have. That's unlocking entirely new product categories.

Boone The timing makes sense too. We've got LLMs that can generate decent code, but we've been stuck evaluating them on algorithmic correctness or static layouts.

Izzo Exactly. And this team pulled from 10 million real generations to build their benchmark. That's not academic toy problems — that's what people actually want.

Boone So here's what they built: 500 tasks across six domains. Games, science tools, utilities, data viz, educational apps, productivity helpers.

Izzo Boone, walk me through how this actually works. How do you evaluate an interactive app when there's no single right answer?

Boone That's the clever part. They built MiniAppEval — an automated framework that uses browser automation to test apps like a human would. It's not just checking if the code compiles.

Boone It evaluates three dimensions. Intention — does the app do what the user asked for? Static — is the UI properly structured? Dynamic — do the interactions actually work?

Izzo Hold on, browser automation for testing AI-generated apps? That's... actually brilliant.

Boone Right? It opens the app in a real browser, clicks buttons, enters data, checks if state updates correctly. Way more sophisticated than string matching against expected output.

Izzo From a product perspective, this is huge. We're talking about AI assistants that don't just give you information — they build you tools on the fly.

Boone And the evaluation framework scales. You can't have humans manually test every generated calculator or game, but you can teach an agent to explore systematically.

Izzo What did they find when they ran current models through this benchmark?

Boone Models are still struggling. Even the best ones hit significant challenges generating high-quality MiniApps. The gap between code generation and interactive experience design is real.

Izzo That makes sense though. Writing a function is different from designing user interactions that feel natural and handle edge cases gracefully.

Boone Exactly. And their evaluation framework shows high alignment with human judgment, so we've got a reliable way to measure progress as models improve.

Izzo I'm thinking about the product implications here. Customer support that builds you a personalized dashboard instead of linking to docs. Educational tools that generate custom practice problems.

Boone Or debugging tools that create visual interfaces for your specific codebase. The shift from 'here's an answer' to 'here's a custom tool' changes everything.

Izzo The evaluation approach is what gets me excited though. If we can automatically test interactive experiences, we can iterate way faster on AI-generated UIs.

Boone And it's not just pass-fail testing. The framework gives you detailed feedback on what's broken — interaction logic, visual layout, intention alignment.

Izzo Okay, I'm giving this research a solid A-minus. It identifies a real shift happening in AI interfaces and builds infrastructure to measure it properly.

Boone Agreed. Plus they open-sourced everything, so we can actually build on this instead of just reading about it.

Izzo Speaking of building — what should listeners go experiment with right now?

Boone First, clone the MiniAppBench repo on GitHub. It's got the full dataset of 500 tasks, so you can see exactly what interactive applications look like.

Boone Second, try their evaluation framework on your own generated apps. Even if you're just having ChatGPT build simple tools, you can use their methodology to test them properly.

Izzo And third — start thinking about your own use cases. What repetitive tasks could become interactive mini-applications instead of static responses? I'm definitely adding a weekend project to build my own MiniApp generator. The evaluation framework makes it actually feasible to iterate on this stuff. The shift from text-first to interaction-first AI is happening whether we're ready or not. This research just gave us the tools to do it right.