LLM Cost Analysis
Home > Select Project > Analysis > Cost Analysis
Cost Analysis analyzes LLM API usage costs from multiple angles. Use it to understand cost status, compare costs by model, analyze cost efficiency relative to performance, and identify optimization targets.
The screen is organized top to bottom in the order Summary > Trend/Detail > Cache/Performance Efficiency > Table, allowing you to naturally progress from understanding the current status to selecting optimization targets.
Set the time range, filters, and search conditions in the top option bar, then query widget data. In the summary cards, you can select a comparison period (Previous Same Period or Same Period Last Week), and drag on charts to narrow the time range.
Summary Cards
Cards that show key metrics for the selected time range at a glance.
Total Cost
Displays the total cost ($) incurred within the selected time range. This is the sum of Input cost and Output cost.
- As the starting point of cost analysis, you can grasp the overall cost scale at a glance.
- Viewing alongside cost change shows how costs have shifted compared to the previous period.
Cost Change
Displays the cost change rate (%) and change amount ($) compared to the previous period.
- Positive (▲ red): Costs increased. Check for increased request volume, higher usage of expensive models, or prompt size changes.
- Negative (▼ blue): Costs decreased. This may indicate optimization effects or traffic reduction.
Total Tokens
Displays the total number of tokens used (Input + Output) within the selected time range.
- Viewing alongside total cost helps determine whether cost increases are due to token usage or expensive model usage.
- If tokens increased but costs remain similar, it means the transition to cheaper models is working well.
- Click to navigate to the Token Trend page.
Token Change
Displays the token usage change rate (%) and change count compared to the previous period.
- Comparing with cost change helps identify the cause of cost fluctuations.
- Token change rate > Cost change rate: The proportion of cheaper models has increased.
- Token change rate < Cost change rate: The proportion of expensive models has increased.
- Click to navigate to the Token Trend page with the comparison period range.
Cost Trend
| Property | Value |
|---|---|
| Chart Type | Bar chart (requests) + Line chart (cost, cost per request) combined |
| Filter | Tag filter |
| Action | View individually, View merged, View details |
Displays the trend of requests, cost, and cost per request by time. Bar chart represents request count; lines represent cost ($) and cost per request ($).
- If requests and cost move together, it is a normal cost increase proportional to usage.
- If requests remain unchanged but cost rises, cost per request has increased -- check for higher proportion of expensive models or prompt size changes.
- A sudden spike in cost per request line may indicate model switching or provider pricing changes.
Token Detail
| Property | Value |
|---|---|
| Chart Type | Card list (card per model) |
| Filter | Model / Provider / Operation / Agent tab switch |
| Action | View details |
Organizes cost, share (%), average Latency, and error rate by tag (Model, Provider, Operation, Agent) in card format.
Each card displays the following information:
| Property | Description |
|---|---|
| Model/Agent Name | Tag value |
| Cost | Total cost ($) for the group |
| Share | Group cost / Total cost x 100 (%) |
| Avg Latency | Average Latency for the group |
| Error Rate | Group error count / Total request count x 100 (%) |
| Cache Hit Rate | Group cached_tokens / input_tokens x 100 (%) |
- High-cost items are sorted at the top, allowing you to immediately identify models/agents with the highest cost contribution.
- Items with high error rates incur costs even on failed requests, so reducing errors has a direct cost-saving effect.
Cache Savings Trend
| Property | Value |
|---|---|
| Chart Type | Line chart ($) |
| Filter | Tag filter |
| Action | View individually, View merged, View details |
Displays the time-based trend of savings ($) achieved through prompt caching. Cached tokens are charged at a lower rate than regular input tokens, and the difference is calculated as the savings.
- Consistent savings indicate that the caching strategy is working effectively.
- If savings decrease, check whether cache hit rate has dropped.
- The ratio of savings to total cost shows the overall cost reduction contribution of caching.
Cost vs. Performance
| Property | Value |
|---|---|
| Chart Type | Bubble chart (x-axis: performance metric, y-axis: cost per request, bubble size: request count) |
| Filter | Model / Provider / Operation / Agent tab switch |
| Action | View details, Select x-axis metric |
Displays the relationship between performance and cost (cost per request) by model/agent as a bubble chart. Bubble size represents request count, and dotted lines indicate the overall average baseline.
Selectable x-axis metrics
| Metric | Description |
|---|---|
| Avg Latency (ms) | Average response completion time |
| Avg TTFT (ms) | Average time to first token |
| Avg TPOT (ms) | Average time per output token |
| p95 Latency (ms) | 95th percentile response time |
| p95 TTFT (ms) | 95th percentile time to first token |
| p95 TPOT (ms) | 95th percentile time per output token |
Quadrant interpretation
| Position | Meaning | Recommended Action |
|---|---|---|
| Bottom-left (fast and cheap) | Most cost-efficient area | Maintain or increase usage of this model |
| Top-right (slow and expensive) | Area where cost optimization is most urgent | Consider alternative models or prompt optimization |
| Top-left (fast but expensive) | Good performance but high cost | Review whether the workload truly requires this performance level |
| Bottom-right (slow but cheap) | Low cost but insufficient performance | Suitable for workloads with low performance requirements |
- Larger bubbles represent a greater share of total cost, so optimizing them yields the greatest impact.
Cost Details
| Property | Value |
|---|---|
| Chart Type | Table |
| Filter | Model / Provider / Operation / Agent tab switch |
| Action | View details, CSV download, Column settings |
Organizes key cost-related metrics by tag (Model, Provider, Operation, Agent) in a table.
| Column | Description |
|---|---|
Name | Tag value (model name, agent name, etc.) |
Requests | Request count |
Input Tokens | Input token count |
Output Tokens | Output token count |
Total Tokens | Total token count |
Input Cost ($) | Input cost |
Output Cost ($) | Output cost |
Total Cost ($) | Total cost |
Share (%) | Cost share (group cost / total cost) |
Cost/Req ($) | Cost per request |
Avg Latency | Average response time |
Cache Hit (%) | Cache hit rate |
Cached Cost ($) | Cached token cost |
Error Rate (%) | Error rate |
Error Cost ($) | Cost incurred from errors |
- Focus on items with high
Total Cost ($)to identify cost optimization targets. - Items with high
Cost/Req ($)indicate expensive models per request -- consider alternative models or prompt optimization. - Items with high
Error Rate (%)incur costs even on failed requests -- resolving errors has a direct cost-saving effect. Error Cost ($)shows the exact amount wasted on errors.- Use the download button in the upper right to export current data as CSV.
- Use column settings to select which columns to display.
Reference
Comparison Period Calculation
| Comparison Mode | Calculation |
|---|---|
| Previous Same Period | Compares with data from a point equally far back as the current query period. Example: querying today 09:00-12:00 compares with 06:00-09:00 |
| Same Period Last Week | Compares with data from the same time range exactly 7 days ago. Example: querying 4/6 09:00-12:00 compares with 3/30 09:00-12:00 |
Custom Token Pricing per Model
You can specify token rates per model directly. Values you specify take precedence over the agent's automatic calculation (the genai-prices package), so you can calculate costs for models that are not in the public rate table — such as self-hosted, fine-tuned, or proxied models — or correct the rate to the actual one when your contract price differs from the list price.
llm_model_pricing=model|input_per_1m|output_per_1m|cached_per_1m
# llm_model_pricing=gpt-4o|2.50|10.00|1.25,gpt-4o-mini|0.15|0.60|0.075,my-custom-model|1.00|5.00
-
Separate models with a comma (
,) and fields with a pipe (|). -
The fourth field can be either omitted (
model|1.00|5.00) or left blank (model|1.00|5.00|). -
Configuration changes take effect immediately and do not require a restart.