Generative UI in Production: Lessons from a Pydantic-to-DSL Migration
On this page
Our agent dashboards used to take 30-40 seconds and cost roughly $0.50 per report. Now they stream in under 10 seconds for around $0.08, on the same design system as the rest of the product. It took two architectures to get there.
A year ago, asking a CloudThinker agent for a cost dashboard meant watching a spinner. The agent called a dashboard() tool, the model filled in a strict Pydantic schema, and about 30 seconds later, once the whole JSON tree validated, the dashboard appeared at once. It used the design system, but it was slow enough that users tabbed away.
Now the dashboard streams in: KPI cards first, then a line chart, then a service breakdown. It is rendered by the product’s own components, so it is interactive: click the expensive region and you drill in.
| Metric | Was | Now | Change |
|---|---|---|---|
| Wall-clock latency | 30–40s | ~10s | ~75% faster end-to-end |
| Cost per report | ~$0.50 | ~$0.08 | ~84% cheaper per artifact |
| Render mode | Atomic | Streaming | Progressive, line by line |
| Design consistency | Drifted | Locked | Enforced by construction |
The two options we ruled out
Matplotlib PNGs. Let the LLM write Python, run it in a sandbox, capture the PNG. The MatPlotAgent paper notes it “struggles to handle complex charts that require fine-grained numerical accuracy”, and Are LLMs Ready for Visualization? found repeated attempts to change default chart settings failed. The same prompt gives different charts on each run. import matplotlib adds a 4-8 second cold start. PNGs are not interactive, hard to theme, and invisible to screen readers (the ACM DIS 2025 GenUI study lists accessibility as a frequent complaint). And every chart means running model-written code in a sandbox.
Raw HTML or JSX. Frontier models write good React, so why not let the agent write the UI? Hardik Pandya’s Expose your design system to LLMs explains the problem: LLMs “fabricate token names, drift on values within a session, lose all context between sessions”. A dashboard costs 2,000-4,000 tokens of markup before any data. Half-streamed HTML is broken HTML, so it cannot render progressively. And it is a security hole.
OWASP LLM05: Improper Output Handling. Cloud resource names are user-controlled. Someone can tag an EC2 instance with an
onerrorpayload, and the next dashboard that mentions that resource passes the tag through the model’s output intodangerouslySetInnerHTML. The agent is not attacked; it is a pass-through. The fix is a format that cannot represent a<script>tag at all.
See OWASP LLM05 and the PortSwigger LLM lab for the attack.
Version 1: typed JSON with Pydantic
We shipped a dashboard() tool. The agent calls it with typed Pydantic widgets (KPI cards, charts, tables) placed on a 12-column grid, using structured-output mode so decoding is constrained by the schema. The frontend renders each widget with the product’s own React components. Trimmed:
class ChartWidget(ChartStructuredOutput):
layout: LayoutConfig # x, y, w, h on a 12-col grid
type: Literal["chart"]
DashboardWidget = Annotated[
KpiCardWidget | TableWidget | ChartWidget,
Discriminator("type"),
]
class Dashboard(BaseModel):
title: str
widgets: list[DashboardWidget]
Validators enforce layout rules (KPI cards must be in row 0, chart width at least 3 columns) and return structured errors the model can fix. This got a lot right:
- Design consistency. The agent selects components; it does not style them.
- No XSS surface. The schema has no field for
<script>. - No code execution on the hot path.
- Auditable. Dashboards are stored as typed JSON in PostgreSQL.
It ran for about ten months. By month six the problems were clear:
- Tokens. A four-widget dashboard was 1,500-3,000 tokens of JSON before any data point. Every widget repeats
type,layout: {x, y, w, h},section_idand its full chart config. - No streaming. Constrained decoding means nothing is usable until the whole JSON tree is valid. Users watched a spinner for 30+ seconds.
- Rigid schema. A new chart type meant a new union variant, a validator change, regenerated frontend types, and a backend and a frontend release. About a sprint per component.
- One artifact per conversation. The tool was built around one dashboard’s lifecycle; a second report overwrote the first.
- Grid arithmetic. The agent spent tokens computing absolute coordinates instead of describing what to show.
Result: 30-40 seconds and about $0.50 per report.
What the rest of the industry did
In early 2026 Generative UI became its own category:
| Project | What it is |
|---|---|
| OpenUI Lang (our choice) | Compact, line-oriented DSL for LLMs to stream UI. |
| Vercel AI SDK | streamUI with React Server Components. RSC development is now paused; Vercel recommends AI SDK UI. |
| LangChain / LangGraph | Tool calls mapped to pre-registered React components, streamed with useStream(). |
| Thesys C1 | OpenAI-compatible endpoint that returns UI components instead of text. |
| CopilotKit AG-UI | Event protocol between agent and UI; 120K+ weekly installs, adopted by Google, AWS, LangChain, Microsoft, Mastra and PydanticAI. |
| Google A2UI | Declarative spec for agent-generated UIs. |
CopilotKit describes three patterns: Controlled, Declarative and Open-ended. Our Pydantic version was Controlled; only the payload format was wrong. Every production-leaning project shares four properties:
- A closed component vocabulary. The agent can only emit what the renderer already draws.
- A typed payload that a parser validates before it reaches the DOM.
- A streaming-friendly grammar. Partial output must still render something.
- Renderer-owned styling. The agent says
status: "success"; the renderer picks the green.
OpenUI Lang’s published benchmark: 67.1% fewer tokens than Vercel JSON-Render, 65.4% fewer than Thesys C1 JSON and 61.4% fewer than YAML across seven UI scenarios, and up to 3.0× faster render on deeply nested dashboards.
We did not write our own. Designing a DSL is easy; the parser, validator, streaming renderer, error messages and examples take months. We needed three things: fewer tokens than our JSON, a grammar where a partial parse renders, and a mapping onto the components we already had. OpenUI Lang met all three. Each kpi = KPICard(...) line parses on its own, so half a dashboard renders as half a dashboard.
Version 2: the Artifact system
The agent calls an artifact() tool with a type, a title and a body written in a line-oriented DSL based on OpenUI Lang and adapted to our FinOps components. The body is checked against a component whitelist, stored as plain text in PostgreSQL, and streamed to the frontend.
The body for a Q4 cost dashboard. Each line is a complete component and renders when it arrives:
kpi_spend = KPICard("Total Spend", "$12,450", "-8%", "success")
kpi_save = KPICard("Savings", "$2,400", "+12%", "success")
trend = LineChart(["Jan", "Feb", "Mar"], [Series("Cost", [13500, 12450, 11800])])
breakdown = BarChart(["EC2", "RDS", "S3"], [Series("Cost", [5200, 3100, 1800])])
root = Stack([
Stack([kpi_spend, kpi_save], "row"),
Card([CardHeader("Cost Trend", "Monthly spend trajectory"), trend]),
Card([CardHeader("By Service", "Current month breakdown"), breakdown])
])
The same dashboard in the old JSON carried discriminators, grid coordinates and full axis configs for every widget: several times the tokens.
The vocabulary is small on purpose:
- Layout:
Stack,Card,CardHeader,Divider - Data:
KPICard,Table,TextContent,Callout,Tag - Charts:
BarChart,LineChart,AreaChart,PieChart,RadarChart,ScatterChart,SingleStackedBarChart - Advanced:
GaugeChart,Treemap,Heatmap,FeatureMatrix,ProsCons,ScoreTable
The agent cannot emit a <script> tag, a made-up color or an unvetted chart library. The web renders the DSL with Highcharts and our own components; the same body feeds a WeasyPrint renderer for PDFs.
| Matplotlib | Raw HTML | Pydantic JSON | Artifact DSL | |
|---|---|---|---|---|
| Streaming | No, atomic | Half-HTML is broken | Waits for valid JSON | Line by line |
| Design consistency | Matplotlib defaults | Drifts | Locked | Locked |
| XSS | Sandbox risk | OWASP LLM05 | Unrepresentable | Unrepresentable |
| Output tokens | Python code | 2k–4k / report | 1.5k–3k / report | 60–70% less than JSON |
| New component | Code only | Anything goes | A sprint | Add to the vocabulary |
| PDF export | Already an image | Headless browser | Custom renderer | Same DSL |
| Debugging | Opaque binary | Opaque DOM | Verbose JSON diff | Plain-text diff |
Lessons
1. The constraint is the product. Both versions work for the same reason: the agent picks from a vocabulary we control and the renderer owns the pixels. The ACM DIS 2025 study found 37 UX professionals judged GenUI output “not production-ready”, mostly over accessibility and domain fit. With a whitelist, accessibility is a property of each component (we audit LineChart once), and the vocabulary is the domain. A new component is added once, not per prompt.
2. Streaming changes the experience more than the latency. You cannot stream a PNG, half an HTML page is broken, and constrained JSON is not valid until its closing brace. A line-oriented grammar streams because every line parses. A dashboard that takes 14.2 seconds as JSON streams in 4.9 seconds in OpenUI Lang. Users see KPIs within a couple of seconds, so perceived latency drops even more than measured latency.
3. Token savings compound at agent volume. One investigation can produce many artifacts, so the per-artifact difference becomes a per-user cost difference. The bigger saving is in instructions: teaching a small vocabulary takes a much shorter system prompt and fewer examples than teaching a large JSON schema. Smaller output, a shorter prompt and no waiting on a JSON tree together give the 30-40 s to ~10 s and $0.50 to $0.08 change.
4. One body, many surfaces. The same body renders to the web (React, Highcharts in Cards) and to PDF (WeasyPrint), and could map to Slack Block Kit or email templates. The Pydantic JSON was renderer-neutral in principle, but the React and PDF renderers drifted because the schema was too rich to implement twice. Edits work the same way: “make the chart bigger” becomes a diff against the stored DSL, not a new generation.
5. Plain text is easy to debug. Because artifacts are stored as DSL text: a change in the agent’s choices is a SELECT and a text diff (JSON diffs were mostly reordered positions); a past artifact re-renders with no model call or sandbox, which makes bug reports and regression tests cheap; and every report is a readable document for audits.
What we would do differently
- Expect to rewrite the payload format once. The Pydantic version worked, but it was a step, not the destination. Plan for a second format from the start.
- Write the whole vocabulary down first. The second time we built components as we needed them. We should have locked the full vocabulary, tested it against a handful of target reports, and then built the components, as we had done with the Pydantic schemas.
- Most of the work is prompting. The DSL is a few dozen components and a few hundred lines of parser. Teaching the agent when to use
KPICardorCallout, a gauge or a single number, a row or a column took longest. Thesys’s guide helped more than we expected. - Make validation errors readable by the model. Our first parser returned Python tracebacks, and the agent made new mistakes trying to fix them. Now it returns hints: “You called
LineChartwithout a labels array. Expected:LineChart(labels, series).” Error messages are prompts. - Build the PDF renderer early. A second renderer showed at once which components secretly needed client-side JavaScript.
Next: from snapshots to live workspaces
Today artifacts are read-only. Every follow-up question creates a new artifact next to the old one. We are building three changes:
- Two-way artifacts. Clicking a region, dragging a date range or hovering a row becomes input the agent acts on in the next turn.
- Edits as DSL diffs. “Make it 90 days” produces a small diff, the renderer repaints in place, and the log records the change, so artifacts can be versioned and replayed.
- Controls in the DSL.
Filter(),DateRange(),Toggle()andSlider(), wired to a re-run by the renderer. In a PDF aDateRange()becomes a static caption.
The contract stays the same: the agent describes intent, the renderer owns the pixels and the events.
Further reading: OpenUI Lang benchmarks · CopilotKit: The Developer’s Guide to Generative UI in 2026 · OWASP LLM05 · ACM DIS 2025: The GenUI Study · Generative UI and the Ephemeral Interface
First published on cloudthinker.io on April 6, 2026.