Add AI Features to Your App: Code-First Guide

Last updated: June 6, 2026.

Most "add AI to your app" guides are written by agencies trying to sell you a six-figure consulting engagement. This one gives you the code-first approach: working OpenAI and Anthropic API calls, streaming UI with the Vercel AI SDK, cost formulas you can plug into a spreadsheet, and production patterns that keep your app reliable when the LLM decides to be weird. You'll integrate AI into your existing app without rewriting anything.

Quick Summary, What You Can Build (and What It Costs)

Before you pick which AI feature to ship first, here's a realistic breakdown. These estimates assume 100 active users and gpt-4o-mini as the default model unless the feature requires something heavier.

Pick the feature that's easiest and highest-value for your product. For most SaaS apps, that's either an in-app chat assistant or content summarization. Start there, prove it works, then expand.

The rest of this guide walks through every step, from your first API call to production-hardened deployment.

Before You Write a Line of Code, When NOT to Add AI

Here's something no one else will tell you: don't use an LLM if a regex, a SQL query, or a simple if statement solves the problem. Every AI API call costs money, adds latency, and introduces non-determinism. Before you integrate AI into your existing app, run the "regex test."

The Regex Test

When AI Actually Adds Value

Do use an LLM when the input is messy, unstructured, or varies wildly, and when the output needs to be natural, context-aware, or creative. If your data is clean and your rules are clear, skip the AI and save your budget.

A quick cost reality check: even gpt-4o-mini at $0.15 per million input tokens adds up. One thousand users making 10 requests per day at 500 tokens each = 5 million tokens/month = about $0.75/month in input costs. Cheap, but not free. And if you accidentally route those requests to gpt-4o ($2.50/1M tokens), that's $12.50/month, still manageable, but 16x more expensive for tasks that don't need the extra intelligence.

Choosing Your Model and Provider

You've got three major providers worth considering for most SaaS LLM integration work. Here's where they stand in early 2026.

Pricing sourced from OpenAI, Anthropic, and Google AI as of June 2026.

Start Cheap, Upgrade When Needed

Here's the approach that saves you money: start with gpt-4o-mini or claude-haiku-4.5 for everything. Run it for a week, measure quality with real user feedback, and only upgrade to a bigger model for the specific tasks where the cheap model falls short. Most summarization, classification, and autocomplete features work perfectly fine on mini-tier models.

For a deeper look at building your entire AI stack, check out our AI SaaS Stack Guide.

Your First AI Feature, API Integration

Time to write code. Here's the exact same operation, a chat completion call, in both Python and TypeScript. Pick the one your backend uses.

Python (OpenAI SDK)

# pip install openai
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def ask_ai(user_message: str) -> str:
    """Call the LLM and return the response text."""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a helpful assistant for our SaaS product."},
            {"role": "user", "content": user_message},
        ],
        temperature=0.7,
        max_tokens=1024,
    )
    return response.choices[0].message.content

TypeScript (OpenAI SDK)

// npm install openai
import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function askAI(userMessage: string): Promise<string> {
  const response = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [
      { role: "system", content: "You are a helpful assistant for our SaaS product." },
      { role: "user", content: userMessage },
    ],
    temperature: 0.7,
    max_tokens: 1024,
  });
  return response.choices[0].message.content ?? "";
}

Where This Code Lives in Your App

Don't call OpenAI from your frontend. Ever. This code belongs in:

• Next.js: an API route (app/api/chat/route.ts)
• FastAPI: an endpoint (@app.post("/api/chat"))
• Express: a handler (router.post("/api/chat", ...))

Your frontend sends a request to your backend, your backend calls OpenAI, and returns the result. This keeps your OPENAI_API_KEY on the server where it belongs.

That's it. You've got a working AI feature. But it feels sluggish, the user clicks "send" and stares at a blank screen for 2-3 seconds. Streaming fixes that.

Making It Feel Real, Streaming AI Responses

A 2-3 second wait with no feedback feels broken. Streaming makes the same response feel instant by showing tokens as they arrive, the typewriter effect you've seen in ChatGPT. Every production AI app uses it, and it's surprisingly easy to implement.

Server-Side Streaming (Python + TypeScript)

Here's the Python approach using FastAPI and Server-Sent Events:

# pip install fastapi openai sse-starlette
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from openai import OpenAI
import os

app = FastAPI()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

@app.post("/api/chat")
async def chat(user_message: str):
    """Stream the LLM response token by token."""
    def generate():
        stream = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": "You are a helpful SaaS assistant."},
                {"role": "user", "content": user_message},
            ],
            stream=True,
        )
        for chunk in stream:
            if chunk.choices[0].delta.content:
                yield chunk.choices[0].delta.content

    return StreamingResponse(generate(), media_type="text/event-stream")

And the TypeScript equivalent using Next.js with the Vercel AI SDK, which handles the streaming plumbing for you:

// npm install ai openai
// app/api/chat/route.ts (Next.js App Router)
import { openai } from "@ai-sdk/openai";
import { streamText } from "ai";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai("gpt-4o-mini"),
    system: "You are a helpful SaaS assistant.",
    messages,
  });

  return result.toDataStreamResponse();
}

Client-Side: The Vercel AI SDK Way

On the React side, the useChat hook handles everything, message state, streaming, error handling:

// components/Chat.tsx
"use client";
import { useChat } from "@ai-sdk/react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: "/api/chat",
  });

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id} className={m.role === "user" ? "user-msg" : "ai-msg"}>
          {m.content}
        </div>
      ))}
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Ask something..." />
        <button type="submit" disabled={isLoading}>Send</button>
      </form>
    </div>
  );
}

That's a fully functional streaming AI chat in about 40 lines of code across the server and client. The useChat hook manages the message array, appends streamed tokens in real-time, and handles loading states automatically. You don't touch EventSource or ReadableStream directly. For more on how streaming works under the hood, the Vercel AI SDK documentation is the definitive reference.

Making Outputs Reliable, Structured Outputs and Function Calling

Raw LLM text is great for chat. It's terrible for anything your code needs to parse. If you're extracting data, triggering actions, or building structured UI, you need structured outputs.

Structured Outputs (JSON Mode)

OpenAI's response_format parameter forces the model to return valid JSON matching your schema. No more hoping the model outputs parseable text:

from pydantic import BaseModel
from openai import OpenAI

client = OpenAI()

class ProductReview(BaseModel):
    sentiment: str  # "positive", "negative", "neutral"
    key_points: list[str]
    rating: int  # 1-5
    recommended: bool

response = client.beta.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Extract a structured review from user text."},
        {"role": "user", "content": "Amazing product! Fast shipping, great quality. Only downside is the price."},
    ],
    response_format=ProductReview,
)

review = response.choices[0].message.parsed
print(review.sentiment)    # "positive"
print(review.rating)       # 4
print(review.key_points)   # ["Fast shipping", "Great quality", "High price"]

The model is constrained to return exactly the fields you define. No parsing errors, no regex extraction, no "sometimes it returns markdown and sometimes it doesn't." For the complete reference on schemas, modes, and edge cases, see our LLM Structured Outputs Guide.

Function Calling for App Actions

Function calling lets the LLM trigger actions in your application, updating a database record, sending an email, or calling an external API. You define the available tools, and the model decides when to use them:

import OpenAI from "openai";

const client = new OpenAI();

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Update my email to [email protected]" }],
  tools: [
    {
      type: "function",
      function: {
        name: "update_user_profile",
        description: "Updates a field on the user's profile",
        parameters: {
          type: "object",
          properties: {
            field: { type: "string", enum: ["email", "name", "avatar_url"] },
            value: { type: "string" },
          },
          required: ["field", "value"],
        },
      },
    },
  ],
});

// The model returns a tool_call -- you execute it in your backend
const toolCall = response.choices[0].message.tool_calls?.[0];
if (toolCall?.function.name === "update_user_profile") {
  const args = JSON.parse(toolCall.function.arguments);
  await db.users.update({ [args.field]: args.value }); // Your DB call
}

The model doesn't execute anything directly. It tells you what to call and with what arguments, and you run the actual function in your secure backend. This is how you build AI features that go beyond chat and actually do things. For advanced patterns like multi-step tool chains and parallel calls, see our LLM Function Calling Guide. Anthropic has a similar tool use API if you're using Claude.

Adding Knowledge, RAG in 50 Lines

Your LLM doesn't know about your product, your docs, or your users. RAG (Retrieval-Augmented Generation) fixes that: search your data first, then feed the relevant chunks to the model as context. It's the most common pattern for making AI features company-specific.

The Pattern: Search, Then Ask

1. Embed your documents into vectors (once, at ingestion time)
2. Store the vectors in a database (pgvector, Pinecone, Qdrant, Weaviate)
3. Retrieve the most relevant chunks when a user asks a question
4. Inject those chunks into the LLM prompt as context

Minimal RAG Implementation

# pip install openai numpy psycopg2-binary pgvector
from openai import OpenAI
import numpy as np

client = OpenAI()

# Step 1: Embed a document chunk
def embed(text: str) -> list[float]:
    response = client.embeddings.create(model="text-embedding-3-small", input=text)
    return response.data[0].embedding

# Step 2: Store in pgvector (assumes table with vector column exists)
def store_chunk(cursor, text: str, embedding: list[float]):
    cursor.execute(
        "INSERT INTO documents (content, embedding) VALUES (%s, %s)",
        (text, np.array(embedding).tolist()),
    )

# Step 3: Retrieve relevant chunks
def search(cursor, query: str, top_k: int = 3) -> list[str]:
    query_embedding = embed(query)
    cursor.execute(
        """SELECT content FROM documents
           ORDER BY embedding <=> %s::vector LIMIT %s""",
        (np.array(query_embedding).tolist(), top_k),
    )
    return [row[0] for row in cursor.fetchall()]

# Step 4: Ask the LLM with context
def ask_with_context(question: str, cursor) -> str:
    chunks = search(cursor, question)
    context = "\n\n".join(chunks)
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"Answer using this context:\n\n{context}"},
            {"role": "user", "content": question},
        ],
    )
    return response.choices[0].message.content

That's the entire RAG pipeline in about 40 lines. For a production-ready setup with chunking strategies, hybrid search, and evaluation, see our full guide to building a RAG application. If you're evaluating frameworks, LangChain and LlamaIndex both provide higher-level abstractions.

Production Patterns, Cost, Security, and Error Handling

Everything above works great in development. Production is where things get interesting. This section covers the problems you'll hit two weeks after deploying your AI feature, and how to solve them before they cost you sleep (or money).

Token Budget Math (What Your AI Feature Actually Costs)

Stop guessing. Here's the formula: users x requests/day x avgtokens x costper_token = monthly cost.

The Growth tier is where people get surprised. At 5,000 users, you'll want model routing: send easy requests (summarization, classification) to gpt-4o-mini and only route complex requests (multi-step reasoning, code generation) to gpt-4o. This can cut costs by 60-70%.

Other cost optimization tactics:

• Prompt caching: OpenAI and Anthropic both offer up to 50-90% savings on repeated prompt prefixes
• max_tokens** limits**: cap output length so the model doesn't ramble
• Semantic caching: if a user asks the same question twice, return the cached response

For advanced prompt optimization and caching strategies, see our guide on context engineering techniques.

API Key Security (The Backend Proxy Pattern)

This should be obvious, but it keeps showing up in production apps: never expose your API keys in frontend code. Not in environment variables that get bundled into the client. Not in a "hidden" JavaScript variable. The OWASP Top 10 for LLM Applications lists sensitive information disclosure (LLM02:2025) as a top risk.

The fix is simple: your frontend calls your backend API. Your backend calls OpenAI. The API key lives exclusively on the server, loaded from an environment variable or a secrets manager.

Also implement per-user rate limiting to prevent a single user from draining your API budget. Which brings us to:

Rate Limiting Per User

Track usage at the user level, not just the global level. A free-tier user who discovers your AI endpoint and fires 10,000 requests will make your CFO very unhappy.

Error Handling and Fallback Chains

LLM APIs go down. They return garbage. They hit rate limits. Your app needs to handle all of it gracefully. Here's a retry-with-fallback pattern:

import OpenAI from "openai";
import Anthropic from "@anthropic-ai/sdk";

const openai = new OpenAI();
const anthropic = new Anthropic();

async function aiWithFallback(prompt: string): Promise<string> {
  const models = [
    () => callOpenAI("gpt-4o-mini", prompt),
    () => callOpenAI("gpt-4o", prompt),
    () => callAnthropic("claude-3-5-haiku-latest", prompt),
  ];

  for (const callModel of models) {
    try {
      return await withRetry(callModel, { maxRetries: 2, baseDelay: 1000 });
    } catch (err) {
      console.warn(`Model failed, trying next fallback...`, err);
    }
  }
  // All models failed -- return cached or static response
  return "I'm temporarily unable to process your request. Please try again shortly.";
}

async function withRetry<T>(fn: () => Promise<T>, opts: { maxRetries: number; baseDelay: number }): Promise<T> {
  for (let i = 0; i <= opts.maxRetries; i++) {
    try {
      return await fn();
    } catch (err: any) {
      if (i === opts.maxRetries) throw err;
      if (err?.status === 429 || err?.status >= 500) {
        await new Promise((r) => setTimeout(r, opts.baseDelay * 2 ** i)); // Exponential backoff
      } else {
        throw err; // Don't retry client errors (400, 401, etc.)
      }
    }
  }
  throw new Error("Unreachable");
}

The key principles: retry 429 and 5xx errors with exponential backoff, fall through to the next model provider when retries are exhausted, and always have a final fallback (cached response, static content, or a clear error message). Never let an AI failure crash your app.

How Techsy Approaches AI Feature Development

We start every AI project with the same question from Section 3: "Does this actually need an LLM, or is there a simpler solution?" You'd be surprised how often the answer is "a well-designed SQL query handles 80% of this."

When AI is the right call, here's our process:

1. Prototype fast, working proof-of-concept in 1-2 weeks using gpt-4o-mini and the simplest possible architecture
2. Measure with real users, not synthetic benchmarks, actual user satisfaction (thumbs up/down, task completion rate)
3. Iterate with evals, automated LLM evaluations that catch quality regressions before users do
4. Harden for production, rate limits, fallback chains, cost monitoring, and the security patterns from this guide
5. Optimize costs, model routing, prompt caching, and right-sizing models per feature

What to Expect on Real Builds: Public-Benchmark Guidance

We don't publish client metrics without permission, but the following figures are grounded in published model benchmarks and public API pricing data, useful as engineering targets before you have your own numbers:

• Streaming first-token latency on gpt-4o-mini typically lands between 200ms and 600ms under normal load. gpt-4o is similar or slightly higher. Expect p95 to be 1.5-2x the median.
• Cost per conversation for a typical 600-token support reply (400 input + 200 output) on gpt-4o-mini: (400/1,000,000 × $0.15) + (200/1,000,000 × $0.60) = $0.000060 + $0.000120 = $0.00018 per reply, under a cent even at 5,000 replies/day.
• RAG overhead: embedding each query via text-embedding-3-small ($0.02/1M tokens) adds roughly $0.000010 per lookup, negligible compared to the completion call.

These are representative starting points. If you ship and instrument your own calls, actual numbers will vary by prompt length, system-message size, and traffic spikes. If you've run Techsy-built integrations and want to share benchmark data for this guide, contact us.

For strategies to reduce what you spend at scale, see our LLM API cost reduction guide.

We've built streaming AI chat, RAG-powered knowledge bases, and AI-driven classification systems for SaaS products. The patterns in this guide are the same ones we use in client projects, nothing is withheld. If your "AI feature" is drifting toward a full autonomous agent, our guide to when to hire an AI agent development agency covers the cost ranges, stack, and disqualifiers so you can tell whether DIY is still the right call.

Building AI features and need a second pair of eyes? Get a free architecture review.

Frequently Asked Questions

How do I add AI features to my SaaS without rebuilding from scratch?

You don't rebuild. You add a backend API route that calls OpenAI or Anthropic, wire it to your existing UI, and deploy. The code examples in this guide show exactly that, a new endpoint, not a new architecture. Start with one feature like chat or summarization and expand from there.

What is the fastest way to integrate OpenAI into an existing app?

Install the SDK (pip install openai or npm install openai), create a backend API route, call chat.completions.create(), and return the result. With the Vercel AI SDK's useChat hook, you can have streaming AI chat running in under 30 minutes.

How much does it cost to add AI features to a SaaS app?

API costs for a 1,000-user app range from $15-150/month depending on the model and usage patterns. gpt-4o-mini at $0.15/1M input tokens keeps costs very low. Development time is typically 1-4 weeks for your first feature. See the token budget math section for detailed scenarios.

Should I use RAG or fine-tuning to add AI to my product?

RAG for 90% of use cases. Fine-tuning only when you need the model to learn a specific style or domain knowledge that can't be provided through context. RAG is cheaper, faster to implement, and much easier to update, you just add new documents to your vector store instead of retraining a model.

How do I avoid exposing my OpenAI API key in a web app?

Never call the OpenAI API from the frontend. Create a backend proxy, your frontend calls your API, your backend calls OpenAI. Store the key in environment variables on the server side. Add per-user rate limiting so no one can abuse your endpoint.

How long does it take to add AI features to an existing app?

A basic chat feature takes 1-2 days. Streaming UI adds 2-3 days. RAG with your company data takes 1-2 weeks. Full production hardening with rate limits, error handling, and cost controls takes 2-4 weeks. You can ship the basic version in days and iterate from there.

When should I use GPT-4o vs Claude vs Gemini?

GPT-4o for general tasks with the largest ecosystem and best tool support. Claude Sonnet 4.6 for long documents, careful instruction following, and coding tasks. Gemini 2.5 Pro for multimodal work (images + text) and Google Cloud integration. Start with GPT-4o-mini for cost savings, upgrade only when you can measure a quality difference.

How do I handle AI errors in production?

Implement retry logic with exponential backoff for 429 (rate limit) and 5xx errors. Build a fallback model chain, try your primary model, fall back to an alternative provider, then fall back to a cached or static response. Never let an AI failure crash your app or show a blank screen.

What AI features should a SaaS app have in 2026?

Start with the highest-value, lowest-complexity feature for your specific product. For most SaaS apps: AI-powered search, content summarization, or an in-app assistant. Check the Quick Summary table at the top of this guide for cost and difficulty estimates per feature type.

How do I know if my AI feature is actually working?

Set up LLM evals, automated tests that measure response quality, relevance, and safety on a set of representative inputs. Track user satisfaction metrics like thumbs up/down ratings and follow-up question frequency. Compare AI-assisted task completion against the non-AI flow. If users aren't completing tasks faster or more successfully, the feature needs work.

Your AI Feature Shipping Checklist

You've got the full picture now. Here's your step-by-step path to shipping:

1. Pick your feature, use the Quick Summary table to choose the highest-value, lowest-difficulty option for your product
2. Run the regex test, confirm AI is genuinely the right tool for this task
3. Start with a cheap model, gpt-4o-mini or claude-haiku-4.5, measure quality before upgrading
4. Build the basic API call, Python or TypeScript, behind a backend proxy
5. Add streaming, the Vercel AI SDK makes this trivial for React apps
6. Add structured outputs, if your feature needs parseable data, not free-form text
7. Calculate your token budget, users x requests x tokens x cost = monthly bill
8. Implement rate limits and error handling, per-user limits, retry logic, fallback chain
9. Deploy and measure, track user satisfaction, not just whether the API returns 200

Your first AI feature is closer than you think. The hardest part isn't the code, it's deciding which feature to build first. Pick one, ship it this week, and iterate based on what real users tell you.

Sources

• OpenAI API Pricing
• Anthropic Claude API Pricing
• Vercel AI SDK, Streaming
• OpenAI Structured Outputs Guide
• OpenAI Production Best Practices
• OWASP Top 10 for LLM Applications 2025
• Anthropic Tool Use Documentation
• LangChain RAG Tutorial
• LlamaIndex Introduction to RAG