LLM Metrics Glossary

This document is a glossary of the key metrics and concepts used in AI Agent Observability. It contains the definitions and units of indicators commonly used across multiple menus such as the dashboard, token trend, and cost analysis.

LLM Performance Metrics

Metric	Description
Latency	Total elapsed time from LLM API request to full response completion (ms). Overall user-perceived wait time
TTFT	Time To First Token. Time until the first response token arrives after the request (ms). Perceived response start point in streaming environments
TPOT	Time Per Output Token. Average time to generate a single output token (ms). Higher values create a choppy response feeling
Output TPS	Tokens Per Second. Output tokens generated per second. Overall model throughput indicator

Tokens and Cost

Metric	Description
Input Tokens	Total input token count in the prompt. Includes Cached Tokens
Output Tokens	Response token count generated by the model. Directly affects cost
Cached Tokens	Input token count retrieved from cache. Subset of Input Tokens. Lower rate than regular input
Cache Hit Rate	Ratio of cached tokens to total input tokens (%). Higher values mean greater cost savings
I/O Ratio	Ratio of output tokens to total tokens (%). Used to identify workload characteristics (input-heavy vs. generation-heavy)

Percentiles

Percentile	Description
p50	50% of all data at or below this value. Typical user-perceived performance
p75	75% of all data at or below this value. Gap from p50 shows additional latency for upper-tier requests
p95	95% of all data at or below this value. Commonly used as SLA threshold
p99	99% of all data at or below this value. Slowest 1% performance, used to assess system stability

Quality Evaluation Metrics

Metric	Description
Hallucination	Hallucination score. Degree of false information or contradiction with context. Lower is better (0.0 ~ 1.0)
Answer Relevance	Answer relevance score. Whether the question was answered appropriately. Higher is better (0.0 ~ 1.0)
Toxicity	Toxicity score. Level of hate speech, profanity, violence, or sexual content. Lower is better (0.0 ~ 1.0)
Combined Judge	Overall risk score. max(hallucination, toxicity, 1 − answer_relevance). Maximum risk value among the three metrics. Lower is better (0.0 ~ 1.0)
Prompt Injection	Degree of detected prompt injection attempts. Lower is better (0.0 ~ 1.0)
Factuality	Factual accuracy score of the response. Higher is better (0.0 ~ 1.0)
PII Leak	Degree of personally identifiable information (PII) exposure
URL Scan	Risk level of URLs included in the response
Eval Success	Success of the evaluation itself (1 = normal, 0 = judge call/parsing failure)

Tag-Based Filters

All data in AI Agent Observability can be filtered or grouped by the following tags.

Tag	Description	Example
Agent	LLM agent (application instance)	llm-app-01, llm-app-02
Model	LLM model used	gpt-4o, claude-sonnet-4-20250514
Provider	LLM API provider	api.openai.com, api.anthropic.com
Operation	User-defined call unit label (chain/prompt/agent name, etc.). Defaults to default if not set. Configured via prompt_meta	checkout_chain, rag_query, default
Prompt Version	Version of the prompt/chain	v1, v2, v3