Shipping LLM Features in 2026: A Checklist I Use Before Every Deploy
Le Do NghiemAI Engineer
2026-06-29 12 min read
Getting a chatbot to work on your laptop feels great. Shipping it to real users is a different story.
In 2026, adding an LLM to a product is easier than ever. But "it runs in the demo" is not the same as "people can trust it in production." I learned that the hard way-and now I use a simple checklist before every deploy.
This post is that checklist. It works whether you are building RAG over your docs, a chat UI in Next.js, or a small agent with LangGraph. No ML lab required. Just clear thinking and a few habits that save you from painful surprises later.
Before you install LangChain, ask three honest questions:
If the answer to all three is "maybe not," pause. LLMs are powerful, but they are also slow, costly, and sometimes wrong. Use them when the job needs language, summarization, or flexible reasoning-not because "everyone has AI now."
Write one sentence: who uses this, when, and what success looks like.
Example: "Visitors can ask questions about my blog posts and get answers based only on what I wrote-not the whole internet."
Also write what failure looks like. Wrong answers? Slow replies? High cost? Pick the top two risks and design around them.
For v1, you do not need perfect accuracy. You need clear limits, good fallbacks, and a way to improve after launch.
Run through these before writing much code.
What: One clear job, not "add AI to the app."
Why: Vague goals lead to vague prompts and unhappy users.
How I verify: I can explain the feature in one sentence to a non-technical friend.
Red flag: The only goal is a marketing badge that says "Powered by AI."
What: Know what goes in (text, files, context) and what comes out (plain text, JSON, stream, citations).
Why: The API and UI depend on this contract.
How I verify: I sketched the happy path and the error path on paper.
Red flag: "The model will figure it out."
What: List trusted sources-your MDX files, database tables, internal APIs.
Why: Without boundaries, the model will guess-and sound confident doing it.
How I verify: I wrote down what is in scope and out of scope.
Red flag: The system can answer questions about anything.
What: A path when AI is unsure: show sources, suggest contact, or escalate.
Why: Trust beats cleverness in production.
How I verify: I tested questions the system should refuse-and the UI handles it calmly.
Red flag: Every wrong answer looks like a bug with no way out.
What: What v1 will not do (multi-agent, fine-tuning, voice, etc.).
Why: Scope creep kills LLM projects fast.
How I verify: Non-goals are in the README or ticket, not just in my head.
Most production LLM features need good context. Bad data beats a bad model almost every time.
What: Decide: RAG, long context, or both.
Why: Dumping entire docs into the prompt does not scale. Tiny chunks with no metadata retrieve the wrong stuff.
How I verify: I drew a simple flow: question → retrieve → augment → answer.
Red flag: "We will just paste everything into the prompt."
What: Try chunk size, overlap, and fields like title, slug, and tags.
Why: Retrieval quality lives or dies here.
How I verify: I ran 10 real questions and checked if the right chunks came back.
Red flag: Raw markdown with no structure.
What: Pick Qdrant, pgvector, Milvus, or similar-and know how you back up and filter.
Why: Dev on laptop and prod on cloud should not be a surprise mismatch.
How I verify: I can re-index all content with one command or script.
Red flag: Embeddings only exist on one developer's machine.
What: Tell the model: answer only from context, cite sources, say "I don't know" when needed.
Why: This cuts hallucinations more than swapping models.
How I verify: Off-topic questions get a polite refusal, not a made-up essay.
What: When you publish new content, how do embeddings update?
Why: Stale RAG is worse than no RAG-confident wrong answers.
How I verify: I added a step to the publish flow (even if manual at first).
Agents are trendy. They are also easy to overbuild.
What: Start with one chain. Add agents only when you need tools and branching.
Why: Simple flows are faster, cheaper, and easier to debug.
How I verify: I listed each tool and explained why a single LLM call cannot do it.
Red flag: Twelve tools for a FAQ lookup.
What: Define inputs, outputs, timeouts, and errors for each tool (Zod or similar helps).
Why: Loose tools become security and reliability holes.
How I verify: Bad tool input returns a clear error, not a stack trace to the user.
What: Cap loops in LangGraph or your own orchestrator.
Why: Runaway agents burn money and time.
How I verify: I forced a path that would loop and confirmed it stops.
What: Retry a little, then tell the user what happened.
Why: Production is messy; APIs fail.
How I verify: I unplugged a tool on purpose and the UI still made sense.
This is where my software engineering background pays off. LLM features are still web features.
What: API keys for Azure OpenAI, OpenAI, etc. only in server env vars.
Why: Keys in the browser will leak.
How I verify: Nothing sensitive in client bundles or public repos.
What: Stream tokens to the UI; show loading and cancellation.
Why: Waiting 30 seconds for a blank screen feels broken.
How I verify: I tested slow responses and mid-stream navigation away.
What: Max message length, max tokens, request timeout.
Why: One huge paste should not take down your API bill.
How I verify: I sent an oversized input and got a clean error.
What: Per IP or per user limits on public endpoints.
Why: Bots love free LLM APIs.
How I verify: I hit the endpoint repeatedly and got throttled.
What: Dev might use Ollama locally; prod uses Azure OpenAI-with separate config.
Why: Accidentally shipping dev config to prod is a classic mistake.
How I verify: Environment names are obvious in logs (without printing secrets).
// app/api/chat/route.ts - minimal pattern
import { NextResponse } from "next/server";
export async function POST(request: Request) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 30_000);
try {
const { message } = await request.json();
if (!message?.trim() || message.length > 4_000) {
return NextResponse.json({ error: "Invalid message" }, { status: 400 });
}
// retrieve context, call model, stream response...
return NextResponse.json({ reply: "..." });
} catch {
return NextResponse.json({ error: "Something went wrong" }, { status: 500 });
} finally {
clearTimeout(timeout);
}
}
You do not need a legal team to get the basics right.
What: Decide what you log-and what you never log.
Why: Prompts often contain emails, names, and private details.
How I verify: Logs redact or skip user content where possible.
What: Untrusted user text should not override system rules or trigger dangerous tools.
Why: "Ignore previous instructions" is not a joke in production.
How I verify: I tried a few injection-style prompts and the system held its ground.
What: Know your provider's region, retention, and training opt-out (Azure OpenAI, etc.).
Why: Your users' data passes through their stack.
How I verify: Links to policy docs are in the project notes.
What: Short note: AI can be wrong; check important facts.
Why: Sets expectations and builds trust.
You do not need a benchmark leaderboard. You need repeatable checks.
What: Real questions users might ask, with notes on what a good answer looks like.
Why: "Feels fine" is not a test plan.
How I verify: Spreadsheet or markdown file in the repo.
| Question | Good answer should… | Pass? |
|---|---|---|
| What is this blog about? | Mention AI / software topics from my posts | ☐ |
| Who wrote post X? | Correct author or say not found | ☐ |
| Random off-topic trivia | Refuse or say not in knowledge base | ☐ |
What: Re-run golden questions when you change prompts, chunks, or models.
Why: Small prompt edits can break unrelated answers.
How I verify: Last deploy note says "golden set: 28/30 pass."
What: Deliberately ask things not in your data.
Why: Confident wrong answers destroy trust faster than "I don't know."
What: Empty input, huge paste, other languages, nonsense strings.
Why: Users will try all of these on day one.
LLM features have a meter running. Treat cost like any other cloud bill.
What: Rough math: tokens in + tokens out × price.
Why: Surprise invoices hurt.
How I verify: I tested a typical session and wrote down the number.
What: Time to first token and total time-including retrieval.
Why: Slow AI feels broken even when answers are good.
How I verify: p95 noted somewhere (even a sticky note counts at first).
What: request id, model name, token counts, retrieval hit count-not raw secrets.
Why: You cannot fix what you cannot see.
What: Feature flag or env var to turn AI off without redeploying everything.
Why: Bad deploys happen; midnight pages should be rare.
How I verify: I turned the feature off once in staging and the app still worked.
What: You first, then friends, then everyone.
Why: Real traffic finds bugs demos never will.
What: Thumbs up/down or a simple "was this helpful?"
Why: Free signal for what to fix next.
What: If the model is down, rate-limited, or answers are wrong-what do you do?
Why: Panic is optional; a short doc helps.
What: Date and note when you change system prompt or swap models.
Why: "What changed last Tuesday?" should be answerable.
## Before build
- [ ] User job is one clear sentence
- [ ] Input/output contract defined
- [ ] Data boundaries written
- [ ] Human fallback exists
- [ ] Non-goals listed
## Before deploy
- [ ] Retrieval/chunking tested on real questions
- [ ] Secrets only on server; rate limits on
- [ ] Streaming + timeouts + error states work
- [ ] Golden set run (note pass count)
- [ ] Off-topic questions refused correctly
- [ ] Cost and p95 latency noted
- [ ] Kill switch tested
- [ ] Disclaimer shown to users
## After launch
- [ ] Feedback collected
- [ ] Logs reviewed weekly
- [ ] Prompt/model changes documented
Here is how I would map this checklist to a small feature-chat that answers from my own blog posts:
That is a real v1. Not flashy-but shippable, measurable, and improvable.
My older post on building AI apps with Next.js and LangChain was a good starting point for a simple chat endpoint. This checklist is what I wish I had before calling that endpoint "production."
A few honest mistakes I keep learning from:
The checklist does not make you perfect. It makes surprises smaller and fixes faster.
Shipping LLM features in 2026 is not about chasing the newest model. It is about treating AI like any other product feature: clear problem, tight scope, real tests, and respect for your users' time and trust.
Keep this checklist somewhere handy. Run through it before deploy-even if you skip a few items at first, write down which ones and why. That honesty is part of shipping, too.
Next up in my queue: a full walkthrough of production RAG on this very blog-ingest, Qdrant, Azure OpenAI, and the golden set in code. If that sounds useful, follow along here or on GitHub.
Happy shipping-and may your refusals be polite and your citations be real.