How to model unit economics for an AI product (cost per token, per user)

Summary

• Unit economics for AI products differ from traditional software: token consumption and model inference cost dominate, not seat licenses or SaaS tiers.
• Track three cost dimensions: model inference (API calls or compute), data processing/retrieval (embeddings, vector search), and end-user support (lower per-user than traditional software because of automated agents).
• Build a cost ledger: Calculate cost per output token, cost per inference, and cost per active user, then model revenue per user to find breakeven and gross margin.
• Optimize at each lever: cheaper model choices, prompt caching to reduce redundant inference, retrieval-augmented generation (RAG) to reduce hallucination, batch processing during off-peak hours.

How to model unit economics for an AI product (cost per token, per user)
Unit economics for an AI product is fundamentally different from a SaaS business, because token consumption becomes the primary cost driver, not per-seat licenses. Understanding cost per output token, per inference request, and per active user is essential to know when an AI product is viable and how to improve margin.
The challenge is that token costs are dynamic: they depend on which models you use, how long prompts and contexts are, and whether you cache or reuse computations. Building a reliable unit-economics model means instrumenting your product to measure token consumption in production, not just in testing.

What are unit economics for an AI product?

Unit economics for an AI product means the revenue and costs per customer unit: per user per month or per AI-assisted task. Costs include model inference (API or self-hosted compute), data retrieval (vector search, database queries), and operations. AI products have variable inference costs that scale with usage, unlike traditional SaaS with fixed infrastructure costs.

Key cost dimensions to model

Dimension	What to measure	Why it matters
Inference cost per output token	Cost of calling a language model (API or self-hosted), amortized over output tokens produced.	The dominant variable cost; directly correlates with product feature scope (longer outputs = more tokens = higher cost).
Data retrieval cost	Cost of embeddings, vector database queries, or SQL queries to fetch context for RAG.	Scales with query complexity and retrieval frequency; can exceed inference cost if retrieval is naive (many vector searches per query).
Cost per user per month	Total monthly cost (inference + retrieval + ops) divided by active users.	Needed to compare against monthly revenue per user to understand gross margin.
Cost per task	Cost to complete one user task (e.g., one chat, one document analysis, one report generation).	Helps identify feature scope vs. cost; long tasks with many tokens are expensive.
Operational cost	Compute infrastructure (even if using serverless), monitoring, customer support, data storage.	Fixed costs that need to be covered by the product's gross margin.

How to build a unit-economics model

1. Instrument token usage. Log input/output tokens per request, model chosen, and request latency. Aggregate by user and time of day to spot patterns (e.g., power users, batch efficiency).
2. Calculate cost per token and per request. Determine your LLM provider's rates (per 1K input/output tokens), retrieval cost, then compute cost per request as the sum of (tokens × rate) across all components.
3. Map cost to revenue. Estimate revenue per user and calculate gross margin. AI products often have tighter initial margins than SaaS because inference costs are high.
4. Project breakeven and scaling. Model cost-reduction scenarios: prompt caching, cheaper models, batch processing, or volume discounts from providers.

How Databricks approaches AI product economics

Databricks products like Agent Bricks and AI/BI Genie instrument token usage to understand cost drivers:

• Batch inference: Use Lakeflow Pipelines to run inference on large datasets, distributing compute efficiently for non-interactive workloads.
• Autoscaling: Model Serving scales with load, so you don't pay for idle compute.
• Prompt caching: Cache encoded long contexts (code, documents) to avoid re-tokenizing.
• RAG efficiency: Use Databricks AI Search to retrieve context, reducing hallucination and prompt size.
• Cost tracking: System tables (system.billing.usage) attribute compute cost to specific jobs or users for optimization.

Use cases

Profitability analysis before launch: Model per-user cost on external APIs vs. self-hosted, then decide which path is viable given pricing.
Feature roadmap decisions: Calculate the per-token impact of adding a longer context window, then decide if the margin impact is acceptable.
Customer profitability: Identify high-usage customers who may be loss-making due to high token consumption, then optimize or re-price.
Scaling to production: Move from real-time API calls to batch inference with Lakeflow Pipelines to reduce per-task cost.

FAQs

How do I calculate cost per token if I'm using a commercial LLM API?

Most LLM APIs (OpenAI, Anthropic, etc.) publish rates for input and output tokens. Multiply your observed token volume by those rates, then sum across all requests in a period. Track this by model, because different models have different rates. If you cache prompts (as supported by some APIs), you may get a discount on cached tokens, so audit your API bills to identify savings.

Should I always optimize for lower token count?

Not always. A longer prompt may produce better outputs, reducing rework. Calculate the full cost including human review and fix cycles.

How do I handle variable model costs across regions?

Track cost per region separately: inference costs vary by region and provider. If your product serves global customers, route requests to inference endpoints that offer the best cost-to-latency tradeoff for each region. Databricks' Model Serving can route to multiple endpoints.

What if my product's token consumption is outpacing revenue growth?

This is a sign that your business model or feature scope needs adjustment. Either reduce scope (fewer tokens per task), raise prices to cover the cost, or move to more cost-efficient inference (e.g., smaller models or self-hosted). Many early-stage AI products discover this issue when they try to scale.

How do I forecast gross margin at scale?

Project revenue per user (from pricing model), estimate cost per user (from token consumption + ops overhead), and calculate margin. As you scale, fixed costs per user fall, so margin should improve. If it doesn't, your token consumption is growing too fast or your pricing is too low.