Adding an LLM feature to a demo costs an afternoon. Adding one that stays accurate, fast, and affordable in production is real engineering. The cost has two parts: building it, and running it.
Figures are planning ranges based on typical project shapes. They are not quotes.
Build cost by feature type
| Feature | Example | Timeline | Planning range (USD) |
|---|---|---|---|
| Single-purpose assistant | Summarise or draft inside one workflow | 4–8 weeks | $20k–$50k |
| RAG over your data | Answers grounded in documents or records | 2–4 months | $50k–$150k |
| Agentic workflow | Tool-calling across several systems | 3–6 months | $100k–$250k+ |
Run cost: the part that surprises teams
Model spend scales with tokens in and tokens out, times requests. A long prompt sent on every request is the most common budget leak. The levers:
- Caching repeated context and answers.
- Model routing: small models for easy tasks, larger ones only when needed.
- Prompt discipline: trim context, retrieve only what is relevant.
- Budgets and alerts per feature, so a bug cannot become a bill.
What else you are paying for
- Retrieval: ingestion, chunking, a vector store, and keeping it fresh.
- Evaluation: a test set and automated evals so quality is measured, not eyeballed.
- Guardrails: output validation, PII handling, and refusal behaviour.
- Observability: log prompts, outputs, latency, and cost per request.
- Fallbacks: provider outages and rate limits need a plan.
Regulated industries
In fintech and healthcare, add data-residency review, audit trails on every model call, and human-in-the-loop steps. These are design decisions, not afterthoughts.
Checklist before you start
- One narrow use case with a success metric
- A test set of real examples
- A cost-per-request target
- A fallback when the model is wrong or down
- An owner for ongoing quality
FAQ
Which provider should we use? OpenAI, Anthropic, and Gemini are all viable. Design the integration so you can switch.
Can we start small? Yes, and you should. One feature, measured, beats a broad rollout.
See how we approach AI integration and LLM apps, or tell us your use case.
