Building an AI MVP in 2026: scope, stack, what changes the timeline and what drives cost

An AI MVP fails differently from a normal MVP. The screens, login and payments can all work while the product is quietly wrong on a share of requests nobody measured. This guide covers how to scope an AI MVP so it can be tested, a reference stack in JavaScript/TypeScript, the build phases and what makes them longer or shorter, what drives cost, and a launch checklist. It includes a small Node.js eval harness you can copy.

What makes an AI MVP different

A normal MVP tests whether people want the product. An AI MVP has to answer a second question first: can the model do the core job well enough, often enough, at a cost that leaves you a margin? Four things change.

• Behavior isn't fixed. The same input can produce different outputs, and a new model version can change behavior you relied on. OpenAI's evals guide calls writing evals "an essential component to building reliable applications", especially "when upgrading or trying new models".
• Evals are your test suite. Unit tests check your code. Evals check the model's answers against cases you wrote down in advance. Without them, "it seems better" is the only signal you have.
• Data is part of the product. You need real example inputs to test with, content for retrieval, and a clear answer to what you're allowed to send to a third-party model.
• Unit cost varies. Every request costs money in proportion to the tokens it uses, so a power user can cost far more than an average one.

Scope it: one AI job, one metric, one eval set

Most AI MVPs that drag on had no written definition of "good". Before anyone writes product code:

1. Name the one AI job. "Draft a first reply to a support ticket from our help center articles" is a job. "An AI assistant for support" is not.
2. Write the success metric. Anthropic's guidance says good criteria are specific, measurable, achievable and relevant, and suggests "accurate sentiment classification" over "good performance". For example: the correct ticket category in 9 of 10 cases, and an escalation flag on every refund request.
3. Collect the eval set. Gather real inputs with the expected outcome for each, including the awkward ones: empty messages, other languages, angry customers, prompt injection attempts. Anthropic's advice is to mirror "your real-world task distribution" and not to forget edge cases.
4. Decide what the human does. Does the AI act on its own, or does a person approve its output? A human-approval step makes a lower accuracy bar acceptable for launch.
5. Write down what's out of scope. A second AI feature or a fine-tuned model usually belongs after launch.

A reference stack in JavaScript/TypeScript

This is the stack we start from for a web-based AI MVP. A boring default lets you spend the budget on the AI job.

Two details worth knowing about pgvector: the vector type can be indexed up to 2,000 dimensions (halfvec up to 4,000), so check your embedding model's output size before you pick it. The pgvector-node package covers common clients, including node-postgres, Prisma and Drizzle.

If the MVP is the first version of a subscription product, the non-AI parts (tenancy, roles, billing, admin) are normal SaaS development work, so plan them that way. If you already have a product and want to add AI to it, our guide to integrating AI into an existing SaaS product is the better starting point.

Build phases and what changes the timeline

We don't publish week counts, because the same phase varies widely between projects. Here is what each phase must produce before the next starts.

What makes these phases longer or shorter:

• Eval data. If labeled examples exist (past tickets, reviewed documents), discovery is short. If someone has to create and label them, that's usually the critical path.
• Integrations. Each system the AI reads from or writes to (CRM, helpdesk, Shopify, an ERP) adds auth, mapping and failure handling.
• Retrieval quality. Clean, current documents are quick to index. Scanned PDFs, duplicates and conflicting versions are not.
• Compliance review. A data protection impact assessment, customer security questionnaires or a regulated sector add review cycles.
• Decision speed. One founder with a metric decides "good enough" faster than a committee.

A minimal eval harness in Node.js

This script runs every case in evals/cases.jsonl through the model, checks the structured answer against the expected one, and exits with an error if the pass rate drops below your threshold, so you can run it in CI. It uses the AI SDK's generateText with Output.object and a zod schema. A plain creator/model-name string is routed through Vercel's AI Gateway (key in AI_GATEWAY_API_KEY); you can pass a provider instance such as openai('model-id') instead.

// evals/run.mjs  (npm i ai zod) run: EVAL_MODEL="creator/model-name" node evals/run.mjs
import { generateText, Output } from 'ai';
import { z } from 'zod';
import { readFile, writeFile } from 'node:fs/promises';
const MODEL = process.env.EVAL_MODEL;
const THRESHOLD = Number(process.env.EVAL_THRESHOLD ?? 0.9);
const REPEATS = Number(process.env.EVAL_REPEATS ?? 3); // outputs vary, so run each case more than once
const system = await readFile('prompts/triage.txt', 'utf8');
const schema = z.object({
  category: z.enum(['billing', 'bug', 'how_to', 'refund', 'other']),
  escalate: z.boolean(),
});
// each line: {"id":"t-001","input":"...","expected":{"category":"refund","escalate":true}}
const cases = (await readFile('evals/cases.jsonl', 'utf8')).trim().split('\n').map((line) => JSON.parse(line));
const results = [];
for (const c of cases) {
  for (let run = 1; run <= REPEATS; run++) {
    const started = Date.now();
    try {
      const { output, usage } = await generateText({ model: MODEL, system, prompt: c.input, output: Output.object({ schema }), maxOutputTokens: 300 });
      const pass = output.category === c.expected.category && output.escalate === c.expected.escalate;
      results.push({ id: c.id, run, pass, output, ms: Date.now() - started, inTok: usage.inputTokens, outTok: usage.outputTokens });
    } catch (err) {
      results.push({ id: c.id, run, pass: false, error: err.name, ms: Date.now() - started });
    }
  }
}
const passRate = results.filter((r) => r.pass).length / results.length;
const flaky = cases.filter((c) => new Set(results.filter((r) => r.id === c.id).map((r) => r.pass)).size > 1).map((c) => c.id);
const ms = results.map((r) => r.ms).sort((a, b) => a - b);
const p95 = ms[Math.min(ms.length - 1, Math.floor(ms.length * 0.95))];
const tokens = results.reduce((t, r) => ({ in: t.in + (r.inTok ?? 0), out: t.out + (r.outTok ?? 0) }), { in: 0, out: 0 });
const summary = { model: MODEL, at: new Date().toISOString(), passRate, p95Ms: p95, tokens, flaky, failures: results.filter((r) => !r.pass) };
await writeFile(`evals/result-${Date.now()}.json`, JSON.stringify(summary, null, 2));
console.log(`pass ${(passRate * 100).toFixed(1)}% | p95 ${p95} ms | tokens in ${tokens.in} out ${tokens.out} | flaky: ${flaky.join(', ') || 'none'}`);
process.exit(passRate >= THRESHOLD ? 0 : 1);

How to use it:

• Keep prompts in files under version control, so every result is tied to a prompt version and a model.
• Start with code-graded checks like the one above (exact category, valid JSON, a citation present). Anthropic recommends automating grading where possible: more cases with slightly noisier automated grading beat fewer hand-graded ones. Add LLM or human grading only for qualities code can't measure, such as tone.
• Re-run the harness before every prompt change and every model switch, and compare token totals too: a cheaper model that needs a longer prompt may not be cheaper.

What drives cost

We don't publish prices, and any fixed number for "an AI MVP" is a guess. These drivers move the number, so you can compare quotes on the same basis.

On model usage, the major providers publish per-token prices on their own pages: OpenAI, Anthropic and Google Gemini. As of September 25, 2026, all three price input and output tokens separately, offer discounted cached input, and list a Batch API at half the standard price for work that doesn't need an instant answer. A rough model for one task is: (input tokens × input price + output tokens × output price) × model calls per task. Two things catch founders out:

• Tokens aren't words. Anthropic's rough guide is about 4 characters or 0.75 English words per token, and it notes its newer tokenizer produces about 30% more tokens for the same text. Measure with your own prompts instead of estimating.
• Residency and privacy options can cost more. For example, Anthropic applies a 1.1x multiplier for US-only inference on recent models. Check the equivalent on your provider's pricing page if you need data to stay in a region.

Buy vs build for each layer

Hosted models have one real drawback: the vendor controls the model's lifecycle (Anthropic's pricing page, for example, lists several older models as retired). Keep the model ID in config and the eval harness green, so a forced upgrade means a round of testing, not a rewrite. For the model layer, our comparison of RAG vs fine-tuning vs prompt engineering explains which fix to try first, and our guide to AI agents for business covers when an agent beats a plain workflow.

Common failure modes

• No eval set until after launch. Every prompt change becomes a guess, and regressions reach customers first.
• Demo-driven scope. Five inputs that looked great in a pitch aren't your real distribution.
• Free-tier data terms in production. Google's Gemini pricing page states that free-tier content is "used to improve our products" and paid-tier content is not. Read the data terms of the exact tier you use.
• No cost ceiling per user. One user or one script can run up a bill in a day.
• Trusting model output as code input. Validate structured output against a schema and treat retrieved documents and user text as untrusted, because prompt injection is real.
• No fallback. When the provider is slow or down, the product should degrade gracefully, not hang.

When you shouldn't build yet

Hold off on a custom AI MVP if an existing product already does the job well enough for your first customers, if you can't collect even 30 real examples of the task, or if nobody can say what "good" means in a sentence. In those cases, spend two weeks on a manual or no-code version and collect the examples. That is cheaper than any build, including one from us.

Launch checklist

1. The eval harness runs in CI, and a failing pass rate blocks the deploy.
2. Every model call logs model, prompt version, tokens, latency and outcome, without storing more personal data than you need.
3. Per-user rate limits, a maximum output length and a monthly usage cap are in place. OpenAI's safety guidance also recommends authenticated users, limits on input and output length, adversarial testing and a moderation check.
4. Users can report a bad answer in one click, and those reports become new eval cases.
5. A provider outage shows a clear message or falls back to a second model.
6. The privacy policy names the AI providers you send data to, what you send and why. Check each provider's data controls; OpenAI, for example, says API data isn't used for training unless you opt in and describes its retention options.
7. For UK users, work through the ICO's guidance on AI and data protection, including whether you need a DPIA. The ICO notes the guidance is under review after the Data (Use and Access) Act.
8. For EU users, Article 50 of the EU AI Act has applied since August 2, 2026. If people interact with your AI system directly, they must be told they are interacting with AI from the first interaction, unless it's obvious. Providers of systems that generate synthetic content must mark it in a machine-readable way; systems already on the market before August 2, 2026 have until December 2, 2026 for that marking duty. Check with a lawyer whether your use case falls into a high-risk area such as hiring or credit scoring.
9. If US buyers ask about AI governance, the voluntary NIST AI Risk Management Framework gives you a shared vocabulary (Govern, Map, Measure, Manage).
10. Cost alerts are set on every provider account and on your hosting.

Want help building your AI MVP?

We scope and build AI MVPs in TypeScript: the one-page scope and eval set first, then the product around it. Tell us the AI job and the data you have on our MVP development page, and we'll tell you honestly whether it needs a custom build yet.

Sources

• OpenAI: Working with evals
• Anthropic: Define success criteria and build evaluations
• Vercel: AI SDK introduction
• pgvector: Open-source vector similarity search for Postgres
• Stripe: Basic usage-based billing
• OpenAI: API pricing
• Anthropic: Pricing
• Google AI for Developers: Gemini Developer API pricing
• OpenAI: Safety best practices
• European Commission: Transparency obligations under Article 50 of the AI Act
• EUR-Lex: Regulation (EU) 2024/1689 (AI Act)
• ICO: Guidance on AI and data protection