Overview Dashboard Widget Reference

This document is a reference that organizes the widgets provided by the Overview dashboard by dashboard area. Check the chart type, filters, and usage for each widget. For the full screen layout and default preset, see Overview dashboard.

Available Widgets

LLM Performance Metrics

Widget	Description
LLM API Request Count	Request count + error rate by time period
Request Count by Status Code	HTTP status code (2xx/4xx/5xx) distribution
Latency Average	Average response time trend by time period
Latency Percentile	Latency p50/p75/p95/p99 trend
Latency Percentile Individual Widgets	Individual widget per percentile
Latency Distribution Comparison	Latency box plot by model
TTFT Average	TTFT average trend by time period
TTFT Percentile	TTFT p50/p75/p95/p99 trend
TTFT Percentile Individual Widgets	Individual widget per percentile
TTFT Distribution Comparison	TTFT box plot by model
TPOT Average	TPOT average trend by time period
TPOT Percentile	TPOT p50/p75/p95/p99 trend
TPOT Percentile Individual Widgets	Individual widget per percentile
TPOT Distribution Comparison	TPOT box plot by model
Overall Performance Comparison	TTFT/TPOT/Output TPS comparison by model
Output Token Throughput	Output tokens per second (tok/s)

Tokens & Cost

Widget	Description
Token Usage	Input/output token trend by time period
Average Tokens per Request	Average input/output tokens per request
Token Usage Distribution	Input/output token total comparison by model
Today's Token Usage	Today's cumulative tokens + comparison with previous day
Cache Hit Rate	Cache token ratio (%) trend
Cache Hit Rate Distribution	Cache hit rate comparison by model
Cache Cost Savings	Cost saved through caching ($) trend
Cache Cost Savings Distribution	Cache cost savings comparison by model
Cost Usage	Input/output cost ($) trend by time period
Average Cost per Request	Average cost per request
Cost Usage Distribution	Cost total comparison by model
Today's Cost	Today's cumulative cost + comparison with previous day

Model Comparison & Analysis

Widget	Description
Overall Performance Comparison	TTFT/TPOT/Output TPS comparison by model
Token Usage Distribution	Input/output token total comparison by model
Error Rate Distribution	Error rate (%) comparison by model
Error Count	API error and program error count by time period
Request Count by Status Code	HTTP status code (4xx/5xx) distribution

Summary counts

Widget	Description
Trajectory count	Total trajectories newly started in the query range
LLM API call count	Total LLM API calls in the query range
Tool call count	Total tool calls (tool use) in the query range

Live Observation

Widget	Description
Active LLM API	Running LLM API calls and the long-running list
Active Trajectory	Number of trajectories in progress and the distribution by level
Real-time prompts	The 50 most recently finished LLM transactions and their prompts

---

Widget Details

Summary counts

A widget group that shows the total counts for the query range as one line of large numbers. All three widgets use a default query range of 10 minutes, and in real-time mode the total of the most recent 10-minute moving range refreshes every 5 seconds. Clicking the number or the detail icon in the header opens a modal with a trend chart that splits the same range into 5-second intervals.

The three widgets are placed by default in the first row of the AI agent main dashboard. On editable dashboards you can add them from the Summary group in the widget menu.

Note

The counts in these three widgets aggregate the values sent by the SDK. On SDK versions that do not support count collection, all three widgets show 0.

Trajectory count

Property	Value
Chart type	Large number + trend chart (modal)
Filter	Dashboard application filter
Action	View details

Shows the total number of trajectories newly started during the query range. The counting unit is not a transaction but a trajectory, that is, one flow grouped by the same mtid. Even if several transactions are grouped into one trajectory, it is counted as one.

• Below the number, the number of trajectories currently running is also shown. The large number is the count started within the range, while the running count is a snapshot of the current moment. The reference points differ, so do not interpret them by adding or subtracting.
• The SDK sends the number of trajectories newly started in each 5-second interval, and the widget adds up the values that fall within the query range. Trajectories that start and end within 5 seconds are not missed.
• A flow that inherited its mtid from a parent is not counted as newly started. Even if one trajectory passes through several processes or hosts, the start count increases only once.
• Clicking the number opens a 5-second interval trend graph, so you can see when trajectories were concentrated within the range.

LLM API call count

Property	Value
Chart type	Large number + trend chart (modal)
Filter	Dashboard application filter
Action	View details

Shows the total number of LLM API calls during the query range.

• Below the number, the average number of LLM API calls per trajectory is also shown. It is the call count of the same range divided by the trajectory start count.
• To view request volume and error rate together, use the LLM API Request Count widget.

Tool call count

Property	Value
Chart type	Large number + trend chart (modal)
Filter	Dashboard application filter
Action	View details

Shows the total number of tool calls (tool use) that occurred during the query range. It is the sum of only the tool_use items in the feature usage statistics.

• Below the number, the ratio (%) of tool calls against all LLM API calls in the same range is also shown.
• If the tool call ratio rises suddenly, check whether an agent is calling the same tool repeatedly, together with the level distribution in the Active Trajectory widget.

LLM API Requests

A widget group for monitoring LLM API request volume and status codes.

LLM API Request Count

Property	Value
Chart Type	Bar chart (request count) + Line chart (error rate) combo
Filter	Tag filter
Action	View individually, View merged, View details

Displays request count and error rate (%) by time period for LLM API calls. The bar chart (left Y-axis) shows request volume, and the line (right Y-axis) shows error ratio against total requests.

• If request volume spikes with a simultaneous error rate increase, Rate Limit exceeded or provider outage is likely.
• If request volume is unchanged but error rate rises, an issue may be occurring in a specific model or agent. Use View individually to narrow down the cause.

Request Count by Status Code

Property	Value
Chart Type	Stacked bar chart
Filter	None
Action	View details

Displays HTTP status code distribution by time period, divided into 2xx (success), 4xx (client error), and 5xx (server error).

• 4xx increase — Check for invalid request parameters, authentication failures, or token limit exceeded.
• 5xx increase — Indicates LLM provider server failure or temporary overload.
• Cross-reference with the LLM API Request Count widget's error rate for quick error classification.

LLM API Response Performance

A widget group for analyzing LLM API overall response time (Latency) from various perspectives. Latency is the total elapsed time from when a request is sent to when the response is fully completed.

Latency Average

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View merged, View details

Displays the Latency average trend over time.

• If Latency average rises overall, check for model response speed degradation or network delays.
• Use View individually to compare Latency by model and identify which model is causing delays.

Latency Percentile

Property	Value
Chart Type	Line chart (p50, p75, p95, p99 — 4 series)
Filter	None
Action	View details

Displays Latency percentile trends over time in four series: p50, p75, p95, and p99. For example, a p95 value of 3 seconds means 95% of all requests completed within 3 seconds.

• A large gap between p50 and p99 indicates most requests are fast but some experience severe delays.
• If p50 and p99 rise together, it indicates overall performance degradation.

Latency Percentile Individual Widgets

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View merged, View details

Provides each percentile as an individual widget. The meaning of each percentile is as follows:

Percentile	Description
p50	Half of all requests completed within this time. Represents average response speed perceived by typical users.
p75	75% of all requests completed within this time. Gap from p50 reveals additional delay in top 25% of requests.
p95	95% of all requests completed within this time. Frequently used as SLA (Service Level Agreement) baseline metric.
p99	99% of all requests completed within this time. Represents the slowest 1% of requests for system stability assessment.

Latency Distribution Comparison

Property	Value
Chart Type	Box plot (x-axis: model)
Filter	Tag filter (always visible)
Action	View details

Compares Latency distribution by tag (model, etc.) using a box plot. The box area represents p25–p75 (middle 50% range), the center line is the median, and whiskers show min–max values.

• Models with wide boxes have high response time variance and may need stability improvements.
• Models with long whiskers experience occasional extreme delays.
• Compare distributions side by side to identify the most stable model.

TTFT (Time To First Token)

TTFT is the time from when a request is sent until the first response token arrives. As TTFT increases, users perceive that "the response hasn't started yet."

TTFT Average

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View merged, View details

Displays the TTFT average trend over time.

• If TTFT spikes at a specific time, check for increased model queue wait or provider-side delay.
• Use View individually to compare which model has slower initial responses.

TTFT Percentile

Property	Value
Chart Type	Line chart (p50, p75, p95, p99 — 4 series)
Filter	None
Action	View details

Displays TTFT percentile trends over time in four series: p50, p75, p95, and p99. For example, a p95 value of 2 seconds means 95% of all requests received the first token within 2 seconds.

• If p50 is stable but p99 spikes, it signals intermittent queue wait occurrences.
• If all percentiles rise together, overall initial response delay is occurring.

TTFT Percentile Individual Widgets

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View merged, View details

Provides each percentile as an individual widget.

Percentile	Description
p50	Initial response wait time perceived by most users.
p75	Gap from p50 reveals additional initial delay in top 25% of requests.
p95	SLA baseline metric: "This initial response wait is guaranteed for most users."
p99	Initial response time of the slowest 1%. Check for intermittent timeouts or delays under specific conditions.

TTFT Distribution Comparison

Property	Value
Chart Type	Box plot (x-axis: model)
Filter	Tag filter (always visible)
Action	View details

Compares TTFT distribution by tag (model, etc.) using a box plot. The box area represents p25–p75, the center line is the median, and whiskers show min–max values.

• Models with low median and narrow box provide the most stable and fast initial responses.
• Models with long whiskers experience occasional extreme initial delays.

TPOT (Time Per Output Token)

TPOT is the average time to generate one output token. Lower TPOT results in smoother streaming responses; higher TPOT may cause a "stuttering response" perception.

TPOT Average

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View merged, View details

Displays the TPOT average trend over time.

• If TPOT spikes at a specific time, check for increased model load or Rate Limit impact.
• Use View individually to compare token generation speed by model.

TPOT Percentile

Property	Value
Chart Type	Line chart (p50, p75, p95, p99 — 4 series)
Filter	None
Action	View details

Displays TPOT percentile trends over time in four series: p50, p75, p95, and p99. For example, a p95 value of 40ms means 95% of all requests generated each token within 40ms.

• If p50 is stable but p99 rises, it signals intermittent token generation stalls.
• If all percentiles rise together, it indicates overall streaming speed degradation.

TPOT Percentile Individual Widgets

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View merged, View details

Provides each percentile as an individual widget.

Percentile	Description
p50	Typical streaming speed. Identify models providing the smoothest streaming.
p75	Gap from p50 reveals streaming speed degradation in top 25% of requests.
p95	SLA baseline streaming speed metric.
p99	Token generation speed of the slowest 1%. Check for intermittent stall issues.

TPOT Distribution Comparison

Property	Value
Chart Type	Box plot (x-axis: model)
Filter	Tag filter (always visible)
Action	View details

Compares TPOT distribution by tag (model, etc.) using a box plot. The box area represents p25–p75, the center line is the median, and whiskers show min–max values.

• Models with low median and narrow box provide the most stable streaming performance.
• Models with long whiskers experience occasional significant token generation slowdowns.

Overall Performance Comparison

Property	Value
Chart Type	Horizontal bar chart (per model, 3 metrics)
Filter	Tag filter (always visible)
Action	View details, p95/p99 tab toggle

Compares three key performance metrics per model in a horizontal bar chart. Toggle between p95/p99 tabs to check typical performance and worst-case performance respectively.

Metric	Description	Direction
TTFT	Time users wait for the first response	Lower is better
TPOT	Time to generate one token (streaming speed)	Lower is better
Output TPS	Output tokens generated per second (overall throughput)	Higher is better

By combining all three metrics, you can identify at a glance models that "start quickly, stream smoothly, and deliver high throughput."

Tokens

A widget group for monitoring token usage patterns and cache efficiency.

Token Usage

Property	Value
Chart Type	Line chart (input, output — 2 series)
Filter	Tag filter
Action	View individually, View merged, View details

Displays input token count and output token count trends over time. Input tokens are tokens included in the prompt; output tokens are tokens generated by the model as responses.

• If token usage spikes at a specific time, check for traffic increase, prompt changes, or abnormal request influx.
• Use View individually to identify which model or agent is consuming the most tokens.

Average Tokens per Request

Property	Value
Chart Type	Line chart (input, output — 2 series)
Filter	Tag filter
Action	View individually, View merged, View details

Displays the trend of average input/output tokens used per request over time.

• If average input tokens per request spike, prompts may have gotten longer or long contexts may have been included.
• If average output tokens per request spike, response lengths have increased, directly impacting cost.

Output Token Throughput

Property	Value
Chart Type	Line chart
Filter	Tag filter
Action	View individually, View details

A throughput metric showing output tokens generated per second (Tokens Per Second).

• Periods with decreased throughput should be checked for model response bottlenecks or Rate Limit impact.
• Cross-reference with token usage to identify periods where "tokens are consumed heavily but processing speed is slow."

Token Usage Distribution

Property	Value
Chart Type	Histogram (grouped by tag)
Filter	Tag filter (always visible)
Action	View details

A distribution chart comparing input/output token totals by tag (model, etc.).

• Quickly identify which model consumes the most tokens.
• Models with high input ratio have room for cost reduction through prompt optimization.
• Models with high output ratio may benefit from adjusting response length limits (max_tokens).

Today's Token Usage

Property	Value
Chart Type	Daily comparison chart (today/previous day)
Filter	None
Action	Date comparison

Displays today's cumulative token usage (input + output) and change compared to the same time on the previous day.

• If token consumption rate is faster than the previous day, check for traffic increase or prompt size changes.
• Useful for proactively detecting Rate Limit thresholds or daily budget overruns.

Cache Hit Rate

Property	Value
Chart Type	Line chart (%)
Filter	Tag filter
Action	View individually, View details

Displays the ratio (%) of tokens retrieved from cache out of total input tokens over time.

• Higher hit rate means prompt caching is working effectively with significant cost savings.
• If hit rate drops suddenly, check for prompt content changes, cache TTL expiration, or new request type influx.
• Use View individually to compare cache efficiency by model.

Formula

Cache Hit Rate = (sum of cached_tokens in period / sum of input_tokens in period) × 100

Cache Hit Rate Distribution

Property	Value
Chart Type	Histogram (by tag)
Filter	Tag filter (always visible)
Action	View details

A distribution chart comparing cache hit rates (%) by tag (model, etc.).

• Models with high hit rate have well-utilized caching; models with low hit rate need prompt structure review for caching targets.
• If hit rate gaps between models are large, differentiate caching strategies per model for additional cost savings.

Cache Cost Savings

Property	Value
Chart Type	Line chart ($)
Filter	Tag filter
Action	View individually, View details

Displays the actual cost savings ($) from prompt caching over time. Cached tokens are charged at a lower rate than regular input tokens, and this difference is calculated as cost savings.

• Consistent cost savings indicate the caching strategy is working effectively.
• If cost savings decrease, cross-reference with cache hit rate to check for increased cache misses.
• Use View individually to identify which model yields the highest savings.

Cache Cost Savings Distribution

Property	Value
Chart Type	Histogram (by tag)
Filter	Tag filter (always visible)
Action	View details

A distribution chart comparing cache cost savings ($) by tag (model, etc.).

• Models with high savings benefit from caching, so maintain or strengthen their caching configuration.
• Models with low savings should be reviewed alongside cache hit rate to improve prompt structure for caching or adjust cache TTL.

Cost

A widget group for monitoring LLM API usage cost.

Cost Usage

Property	Value
Chart Type	Line chart (input, output — 2 series)
Filter	Tag filter
Action	View individually, View merged, View details

Displays input/output cost ($) trends over time. Input cost is prompt processing cost; output cost is response generation cost.

• If cost spikes at certain times, check request volume and tokens per request for that period.
• Compare the ratio of input cost to output cost to decide whether to focus on prompt optimization or response length limits.
• Use View individually to compare cost contribution by model.

Average Cost per Request

Property	Value
Chart Type	Line chart (input, output — 2 series)
Filter	Tag filter
Action	View individually, View merged, View details

Displays the trend of average input/output cost ($) per request over time.

• If cost per request changes suddenly, check for model switching, provider pricing changes, or prompt size changes.
• If total cost increases but cost per request is unchanged, request volume has simply increased — focus on request volume management.

Cost Usage Distribution

Property	Value
Chart Type	Histogram (grouped by tag)
Filter	Tag filter (always visible)
Action	View details

A distribution chart comparing input/output cost totals by tag (model, etc.).

• Quickly identify which model has the highest cost.
• Check usage frequency and tokens per request for high-cost models to select cost optimization targets.

Today's Cost

Property	Value
Chart Type	Daily comparison chart (today/previous day)
Filter	None
Action	Date comparison

Displays today's cumulative cost ($) and change compared to the same time on the previous day.

• If cost consumption rate is faster than the previous day, check for request volume increase, model switching, or prompt size changes.
• Useful for proactively detecting daily budget overruns.

Errors

A widget group for monitoring LLM API error status.

Error Rate Distribution

Property	Value
Chart Type	Horizontal bar chart
Filter	Tag filter
Action	View individually, View merged, View details

Compares error rates (%) by tag (model, etc.) in a horizontal bar chart. Error rate is the ratio of errors (status_code 400 or above) to total requests in the group.

• Quickly identify whether errors are concentrated in a specific model or agent.
• Models with high error rates should be checked for provider outage, invalid parameters, or Rate Limit exceeded.
• Use View individually to narrow down error causes by Agent, Provider, and other criteria.

Error Count

Property	Value
Chart Type	Stacked bar chart
Filter	None
Action	View details

Displays error counts by time period, divided into API errors and program errors.

• API Error — Errors returned by the LLM provider, including Rate Limit exceeded, server errors, and authentication failures.
• Program Error — Client-side errors, including response parsing failures, timeouts, and connection errors.
• Error type trends help quickly distinguish whether the issue is on the provider side or client side.

Live Observation

This widget group lets you check the LLM calls and trajectories that are running right now. A trajectory is the unit that groups the transactions participating in a single LLM workflow. The list screens of both widgets use running data queried directly from the SDK, so there is no past range. If the dashboard is paused or you are viewing a past time, no path to the list is provided.

Active LLM API

Property	Value
Chart Type	Swarm chart (track per model)
Filter	Dashboard application filter
Action	View details, Open in new window

Displays the LLM API calls that are currently running as dots on a track for each model. One dot is one call, and the horizontal position indicates the Normal, Slow, or Very Slow range. Data for the three ranges is aggregated according to the values set in the SDK options.

• If dots pile up in the Very Slow range, check whether calls running longer than the criteria set in the SDK are concentrated in a specific model or provider.
• Clicking the widget body opens the long-running list in a modal. Expanding the prompt lets you check the system prompt and input prompt that are running, and selecting a call in the list moves to the trajectory view that the call belongs to, where you can check the entire execution flow.
• Clicking the View details button in the widget header opens the same list on a wide screen. Clicking the Open in new window button in the modal header opens it in a separate window.

Active Trajectory

Property	Value
Chart Type	Orbit chart + count list by level
Filter	Dashboard application filter
Action	View details, Open in new window

Displays the number of currently active trajectories and the number of steps of each trajectory, divided into the Light, Middle, and Heavy levels. A level indicates how many times and how repeatedly steps were called; it does not mean speed or execution time.

• If even one of the multi-transactions it belongs to is running, that trajectory is counted as live.
• The level boundaries follow the criteria set in the SDK options.
• Clicking a level opens the trajectory list filtered by that level. Selecting a row in the list leads to the trajectory view.
• Clicking the orbit chart opens the entire list without the level filter.

Trajectory List

The list consists of a group header for each trajectory and the transaction rows below it. The group header shows mtid, the number of grouped transactions, the user, and the session. A standalone flow with no mtid is displayed as a standalone run. You can export the list as CSV.

Column	Description
Transaction	Transaction name
Status	Transaction progress status (Running, Completed, Failed)
mtid	Key that groups a trajectory
Steps	Number of steps the transaction performed
Agent	Number of AI subagents the transaction ran
LLM Calls	Number of LLM calls that occurred within the transaction
Elapsed	Time elapsed since the transaction started
User	user_id attached to the log
Session	session_id attached to the log

Real-time prompts

Property	Value
Chart Type	Table
Filter	Dashboard application filter
Action	Refresh, Open in new window

Shows the 50 LLM transactions with the most recent end times in a table, together with their prompts. The query range follows the dashboard time setting, and the table refreshes every 5 seconds.

• To the right of the title, the actual query range and the number of rows placed in the table (for example, Latest 50) are shown.
• The table is not cleared while it refreshes. The loading indicator appears only when you click the refresh button yourself.
• The Prompt column shows the input of the first step and the output of the last step of that transaction in one cell, along with the evaluation result if there is one. Clicking a prompt opens the LLM trace details of that transaction in a new window.
• The only sort criterion is the end time. Transactions that ended with an error also appear, and in that case the response, tokens, and cost may be empty.
• Because log ingestion lags by a few seconds, a transaction that has just finished moves to the top at the next refresh.

This widget is placed by default across a whole row at the bottom of the AI agent main dashboard. On editable dashboards you can add it from the LLM API Requests group in the widget menu.

Reference

Collected Fields

Field	Description
request_count	LLM API call count
status_code	HTTP response status code (2xx, 4xx, 5xx)
latency	Request start to response completion total time (ms)
ttft	Time To First Token. Request start to first token received (ms)
tpot	Time Per Output Token. Average interval between output tokens (ms)
input_tokens	Total input token count (including cached)
output_tokens	Output token count
cached_tokens	Input tokens retrieved from cache (subset of input_tokens)
input_tokens_cost	Total input token cost ($). Includes cached_tokens_cost
output_tokens_cost	Output token cost ($)
cached_tokens_cost	Cached token cost ($)
error_type	Error type (api_error, program_error)

Widget Group Summary

Group	Widgets	Description
Summary counts	3	Total trajectory, LLM API call, and tool call counts for the query range
LLM API Requests	2	Request volume and status code monitoring
LLM API Response Performance	8	Latency trend and distribution analysis
TTFT	7	Time to first token analysis
TPOT	7	Token generation speed analysis
Tokens	9	Token usage patterns and cache efficiency analysis
Cost	4	LLM API cost monitoring
Errors	2	Error status analysis by type
Live Observation	3	Status of running LLM calls and trajectories, and recently finished transactions
Total	45

Terminology

Term	Description
Latency	Total elapsed time from when an LLM API request is sent to when the response is fully completed (ms)
TTFT	Time To First Token. Time from when a request is sent until the first response token arrives (ms)
TPOT	Time Per Output Token. Average time to generate one output token (ms)
Output TPS	Tokens Per Second. Output tokens generated per second (tokens/s)
Percentile	The value below which a given percentage of data falls when sorted in order. Example: p95 = 95% of all data is at or below this value
Cache Hit Rate	Ratio of tokens retrieved from cache out of total input tokens (%). Higher values mean greater cost savings
Tag Filter	Feature to filter data by Agent, Model, Provider, Operation criteria
Preset	Saved widget configuration (type, position, size). Custom presets can be created
Trajectory	Unit that groups the transactions taking part in one LLM workflow by mtid
Tool call	A call in which the model invoked an external tool (tool use). It is aggregated from the tool_use item in the feature usage statistics.