Next.js Architecture for AI Products
Next.js App Router is a great fit for AI products, but only if you respect what serverless and the runtime split actually do. Here are the patterns we standardized across our products in India and the World, and the sharp edges we hit so you do not have to.
Route handlers: the synchronous part stays small
Route handlers (`app/api/.../route.js`) are our API surface. The discipline is that a handler the user waits on does validation, authorization, and a small write, then returns. Anything expensive either streams or moves to background work. A handler that awaits a full LLM completion before responding is a handler that eats cold-start latency and risks timing out.
Streaming: optimize time-to-first-token
For chat and any user-facing generation, stream. Users judge speed by when text starts appearing, not when it finishes. App Router makes this clean — return a streaming `Response` and pipe model tokens through:
```js export async function POST(req) { const { question, context } = await req.json(); const stream = model.stream({ context, question }); return new Response(stream, { headers: { 'Content-Type': 'text/event-stream' }, }); } ```
A six-second answer that starts rendering at 600ms feels fast; the same answer delivered as one blob at second six feels broken.
after(): the only safe way to do background work
This is the pattern that, more than any other, made our AI products reliable. On serverless, work started but not awaited is dropped the instant you respond. `after()` keeps the function alive to finish tail work after the response is sent:
```js import { after } from 'next/server';
export async function POST(req) { const job = await createPendingJob(req); after(async () => { await runGeneration(job); // safe: function stays alive await persist(job); }); return Response.json({ id: job.id, status: 'pending' }); } ```
We treat a bare un-awaited promise in a route handler as a bug in code review. If it is not awaited, streamed, or wrapped in `after()`, it does not merge.
Server actions for mutations from the UI
For form-driven mutations we use server actions rather than hand-rolling fetches to route handlers. They keep the mutation co-located with the component, run on the server with secrets safely out of the bundle, and integrate with revalidation. We still push long AI work off the action's synchronous path with the same `after()`/pending-record discipline — a server action is still a serverless invocation with a clock.
Edge vs node: the tradeoffs that actually bite
This is where teams lose days. Edge runtime is fast to start and globally distributed, but it is a constrained environment. Node runtime is heavier but full-featured. Our rules, learned from real breakage:
- Anything that needs custom fonts for rendering (OG images with brand typefaces) runs on node. Bundled fonts on edge tend to fail to load; we shipped an OG card that silently fell back to system fonts on edge until we moved it to node with bundled font files.
- Anything that needs Chromium — server-side PDF generation for reports — runs on node. Edge cannot run a headless browser.
- Lightweight, latency-sensitive, dependency-free handlers (a geo lookup, a simple redirect decision) are fine on edge.
The heuristic: if it needs the filesystem, a binary, or fonts, it is node. Default to node when unsure; reach for edge deliberately.
File-convention OG images and metadata for SEO and AEO
AI products still live or die by discovery, including by answer engines. App Router's file conventions make this maintainable. We generate Open Graph images with a shared render function on the node runtime, keep OG copy to a line or two so it renders cleanly, and use the metadata API for titles, descriptions, and structured data per route. For programmatic pages we generate metadata dynamically from the same data that drives the page.
One dedup gotcha worth flagging: if you inject structured data both from a default layout and from a page, you can emit duplicate schema blocks. We added a skip-default flag so a page that supplies its own structured data suppresses the layout's. Validate your structured data; answer engines are unforgiving of duplicates.
Build hygiene
A small but real footgun: running `next build` while the dev server is running can corrupt `.next` and produce a false 'Cannot find module for page' failure. Our fix is mechanical — stop the dev server and `rm -rf .next` before a production build. It is the kind of thing that wastes an afternoon if you do not know it.
The summary
Keep the handler thin, stream what the user watches, push everything else into `after()` or a job, pick node when you touch fonts or Chromium, and let file conventions carry your SEO. Next.js gives you the primitives; the discipline is in respecting the serverless clock.
If you are architecting an AI product on Next.js and want to skip the expensive lessons, GrahAI Systems can help — reach us at support@grahai.com.