The provider invoice tells you what you spent. It rarely tells you which feature, tenant, model or release caused the change, or whether the spend produced a successful outcome. That is the gap a cost monitoring purchase is meant to close. If the tool cannot connect a line on the invoice to a decision someone can make, it is a dashboard rather than a control.
Start with the decisions, not the features
Different people need different numbers, and a tool that serves one group badly will be abandoned by the others.
- Engineering needs to know which request types, prompts and releases are expensive, and whether retries or context growth caused a spike.
- Product needs cost per successful task and cost per tenant, not cost per API call.
- Finance needs a daily or monthly figure that reconciles to the provider invoice, with currency and tax treatment stated.
- Leadership needs budget against actual, a forecast and the unit economics of the service.
- Customers or internal business units may need a chargeback or showback report if AI is resold or shared.
Write down the decisions each group will make with the data. Then ask whether the tool produces that number at the required frequency and granularity.
Requirements that separate products
- Request-level records. Tenant, feature, environment, model, prompt version, input tokens, output tokens, cached tokens, latency, retry count and outcome.
- Provider coverage. Direct model APIs, cloud-hosted models, self-hosted models and embedding or reranking services. Ask for the connector list, not a logo wall.
- Invoice reconciliation. The ability to compare internal usage with the provider invoice, explain the variance and handle multiple currencies.
- Budget enforcement. Alerts are easy; hard limits at project, feature and tenant level are harder. Know which one the product actually provides.
- Attribution. A way to split spend by product, customer, cost centre and environment without manual spreadsheet work.
- Anomaly detection with routing. An alert must reach an owner who can explain and act on it.
- Retention and export. Raw records in a usable format, with a retention period you choose.
- Privacy. Prompt and response logging should be optional, redactable and consistent with your data protection obligations.
- Deployment. SaaS, private cloud or self-hosted, with a clear answer on where usage data and prompts reside.
- Outcome measurement. The ability to join cost to a successful task, resolved ticket or completed order.
The five provider categories
| Category | Strength | Blind spot | Fits when |
|---|---|---|---|
| Provider-native dashboards | Fast, included, accurate for one provider | Weak cross-provider attribution and internal cost centres | One provider, one feature, engineering-only reporting |
| LLM gateway or proxy | Central keys, routing, fallback and request metering | Sits in the request path, so reliability becomes critical | Multiple models and a need for central control |
| APM and observability platform | Traces, latency, errors and cost in one investigation | Billing reconciliation and finance allocation may be thin | Quality and latency problems matter as much as cost |
| Cloud FinOps tool | Strong allocation and forecasting across cloud spend | May not understand tokens, prompts or model routing | AI spend is a line inside a larger cloud budget |
| Custom metering plus warehouse | Exact fit, full ownership, custom chargeback | You fund the build, monitoring and maintenance | Attribution is a product requirement or provider data is insufficient |
Most businesses end up with a combination. A gateway provides the request record, an observability platform provides the trace, and a warehouse or finance tool holds the allocation. The buying decision is which layer you need to own.
Scorecard
| Criterion | Weight | Evidence to request |
|---|---|---|
| Attribution depth | 20 | A live report split by tenant, feature and environment, using your data shape |
| Invoice reconciliation | 15 | A worked reconciliation for a past month, including currency and tax |
| Provider coverage | 15 | Connector list and a test with your second provider |
| Budget enforcement | 10 | A demonstrated hard limit, not a screenshot of an alert |
| Privacy and residency | 10 | Data flow, retention options, redaction and subprocessor list |
| Integration effort | 10 | SDK, API and identity requirements, with an estimate in days |
| Reporting and export | 10 | Raw export and a scheduled report delivered to finance |
| Support and reliability | 5 | Status page, incident history and support terms |
| Exit | 5 | Export format, deletion confirmation and transition assistance |
Score each criterion from zero to three and multiply by the weight. Keep the evidence attached to the score. A high total with no tested export or reconciliation is not a shortlist candidate.
Pricing questions
- What is the billing unit: token, request, host, seat, gigabyte, retention or a platform fee?
- Is there a minimum commit, and what happens to unused commitment?
- What is the overage rate, and is it capped?
- Does the price change at higher volume, and is the change written into the contract?
- Are implementation, connectors, custom reports and support charged separately?
- Which currency is used, and who bears exchange-rate movement?
- Is there a charge for API access, raw export or long-term retention?
- What is the renewal increase, and how much notice is required?
A per-seat price can look small until finance, product and three engineering teams need access. A per-token price can look predictable until retries and agent loops multiply volume. Model the unit at current volume, at double, and at a failed launch where retries spike.
A pilot that proves the tool
Run a 30 to 45 day pilot on one production feature. Keep the provider invoice and the tool side by side. Test the following before you sign:
- Reconcile the daily total to the invoice within an agreed tolerance, and explain the difference.
- Attribute spend to a feature, a tenant and an environment without manual matching.
- Switch one request to a different model and show the cost and quality change in the same view.
- Trigger a budget breach and confirm the alert reaches the named owner. If a hard stop is claimed, confirm it stops.
- Export the raw records and open them in your own spreadsheet or warehouse.
- Delete or redact a test record and confirm the retention behaviour.
- Ask for a forecast based on the pilot data, then compare it with the next invoice.
Contract and exit terms
The contract should state that customer data is not used to train models, list subprocessors, define retention and deletion, and give you a usable export. If the product sits in the request path, add availability, latency and support commitments. Ask for price protection and a termination assistance clause. The exit test is simple: can you reconstruct last month's cost by feature and tenant after the contract ends?
Red flags
- Only totals are available, with no request-level record or export.
- Cross-provider coverage is claimed but no connector list or test is offered.
- The tool logs prompts and responses by default with no redaction.
- Anomaly detection cannot explain which dimension changed.
- A long minimum commit is required before a pilot.
- Finance reports are a custom project with no fixed acceptance criteria.
- Currency, tax and provider credits are ignored in the internal figure.
If you need the metering, routing and reporting built around your own products, start with Business Automation. The pricing page explains how implementation and support are scoped. For the failures this purchase is meant to prevent, read the AI model cost monitoring troubleshooting guide.
Sources
- FinOps Foundation, FinOps Framework, accessed 17 November 2026.
- OpenTelemetry, Generative AI semantic conventions, accessed 17 November 2026.
- Amazon Web Services, Using cost allocation tags, accessed 17 November 2026.
- Google Cloud, Export Cloud Billing data to BigQuery, accessed 17 November 2026.
Responses (0)
No comments yet. Add the first one below.
Leave a response
Your email address will not be published. Required fields are marked *.