Building AI-Powered Apps with Next.js and LangChain

Why I wrote this

The first time I added AI to a Next.js app, I wanted a chatbot on the screen by end of day. Not a research project. Not RAG. Just: user types, model replies, page updates.

That is still a valid starting point. A raw LLM call through LangChain is enough for internal tools, prototypes, and "let's see if this idea feels useful" demos.

This post is that first step — API route, minimal client, env secrets. If you are past the demo stage, read how the job changed around AI and my pre-deploy checklist next.

---

When a plain LLM call is enough

What: Send user text to a model, return text. No retrieval, no agent graph.

Why: Fast to build. Good for brainstorming UIs, admin helpers, or features with a narrow, forgiving scope.

When I stop here:

• Wrong answers are annoying but not dangerous
• No private documents in scope
• I can add guardrails later

Red flag: Shipping this to paying users with no evals, no refusal path, and no cost cap. The code works. The product does not.

---

The API route

I keep the model call on the server. Never expose API keys in the client.

// app/api/chat/route.ts
import { NextResponse } from "next/server";
import { ChatOpenAI } from "langchain/chat_models/openai";
import { HumanMessage } from "langchain/schema";

export async function POST(request: Request) {
  const { message } = await request.json();
  const model = new ChatOpenAI({ openAIApiKey: process.env.OPENAI_API_KEY });
  const response = await model.call([new HumanMessage(message)]);
  return NextResponse.json({ reply: response.content });
}

What I do before this runs in any real environment:

• OPENAI_API_KEY lives in .env.local, not git
• Basic validation on message (non-empty string, max length)
• A try/catch that returns a generic error — not the raw stack trace

Why: This is the smallest secure loop. Everything else — streaming, RAG, tracing — builds on top.

---

A minimal client

The UI does not need to be clever. It needs to show loading state and handle empty input.

"use client";

import { useState } from "react";

export default function Chat() {
  const [message, setMessage] = useState("");
  const [reply, setReply] = useState("");
  const [loading, setLoading] = useState(false);

  const handleSubmit = async (e) => {
    e.preventDefault();
    if (!message.trim()) return;
    setLoading(true);
    try {
      const res = await fetch("/api/chat", {
        method: "POST",
        body: JSON.stringify({ message }),
        headers: { "Content-Type": "application/json" },
      });
      const data = await res.json();
      setReply(data.reply);
    } finally {
      setLoading(false);
    }
  };

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <input
          type="text"
          value={message}
          onChange={(e) => setMessage(e.target.value)}
          disabled={loading}
        />
        <button type="submit" disabled={loading}>
          {loading ? "..." : "Send"}
        </button>
      </form>
      {reply && <p>{reply}</p>}
    </div>
  );
}

Why client component: Input state and fetch on submit need the browser. The page shell around this can stay a Server Component.

---

What I skipped on purpose

I did not add these in v1 — and that was intentional:

Skipped	Why
RAG / vector DB	No doc Q&A requirement yet
Streaming	Nice UX, but more moving parts
Conversation memory	Needed product decision on history + privacy
Tool calling / agents	One LLM call solved the job
Eval suite	Prototype only — but I wrote that down

Skipping is fine. Pretending you shipped production when you shipped a demo is not.

---

What I got wrong early

• No rate limiting. One curious user — or bot — can burn through tokens fast.
• Trusting the first reply. Models sound confident when they are wrong. I added a one-line disclaimer in the UI before any external launch.
• Huge system prompts in code. Moved prompts to a separate file once I started iterating. Diff noise matters.

---

Before you call it production

You have a working loop: Next.js API route, LangChain model call, simple client. That is a real milestone.

Before you call it production, walk through the checklist I use before every deploy. Start with "do we need an LLM?" and "what does failure look like?" — those two questions save more time than any prompt tweak.

Next experiments I'd try on this same stack: streaming responses, a system prompt with clear scope, and ten golden questions to see where the model lies. Small steps. Measurable steps.