LLM API Analysis (Transaction Linkage)

Home > Select Project > Analysis > LLM API Analysis

The LLM trace details in AI Agent Observability is a window that shows the steps, prompts, tokens, cost, and execution flow of an LLM transaction in one view. Clicking an LLM API step in the Table View opens the LLM API Call panel, where you can view prompt input/output, token usage, cost, error information, and GPU infrastructure correlation.

Note

Viewing prompt input/output requires log view permission.

How to Access

1. Open the LLM trace details by dragging an area in the Hitmap on a dashboard or in Analysis > Hitmap, or by clicking a prompt in Prompt and response search, Real-time prompts, or User and Session Search.
2. In the Table View of the LLM trace details, click an [LLM API] step.
3. The LLM API Call panel opens on the right.

LLM trace details

The LLM trace details open in a new window. When opened by dragging the Hitmap, the top of the window shows the Trace analysis title and the number of items found, with the transaction list for the range on the left and the details of the selected transaction on the right. When opened by clicking a prompt, the top of the window shows the Transaction info title and only the details of that transaction.

Above the list, you can filter by All / Error and use Column settings, TXT and CSV download, search, and AI analysis. AI analysis appears only in environments where WhaTap AI is enabled. Clicking Copy URL at the right of the title line copies the address of the current view. The list columns are No, Application name (oname), Transaction, Elapsed time, Start time, End time, LLM API call count, LLM API call time (ms), Error message, and Multi-transaction ID.

Tab	Description
Record Summary	Elapsed, LLM wait, tokens, cache hit rate, cost, TTFT distribution, transaction info, endpoint and finish reason distribution, and aggregation by model
Table View	Step table. Each LLM API step shows an Input/Output summary, evaluation scores, and a prompt delta summary line.
Trajectory View	Execution flow of the trajectory this transaction belongs to. If the transaction is not a standalone run and running transactions remain, the Stop button is also shown.

If you open this window without conditions, "No query conditions. Drag the Hitmap or reopen it from the prompt log." is displayed.

Summary and Prompt

This area shows key information about the LLM call at a glance and allows you to view the original prompt input/output.

Identity Tags

Displays the target model and provider information as tags.

Tag	Description	Example
Model	LLM model used	gpt-4o, claude-sonnet-4-20250514
Provider	LLM API provider	api.openai.com, api.anthropic.com
Operation	Operation type of the call	chat, completion

Key Metrics

Displayed inline next to the Identity tags.

Metric	Description
Tokens	Total token count (Input + Output)
Cost	Cost of the call. Amounts under $1 are shown in ¢ (cents).
Latency	Time from request start to response completion. Shown in ms under 1 second and in seconds for 1 second or more.

HTTP Information

Property	Description
URL	Call endpoint (host:port + path)
Elapsed time	Time taken by the HTTP call. Shown in ms under 1 second and in seconds for 1 second or more.
Step ID	Unique identifier for this LLM call. The key used by the Log Search button to query the logs of that step ID.

Click the Log Search button next to Step ID to open the log explorer filtered by that step ID and transaction in a new tab. This button appears only if you have log view permission and logs for this call were found.

Error Information

Displayed only when an error occurred in the call.

Property	Description
Error Class	Error class name (e.g., RateLimitError, TimeoutError)
Error Message	Detailed error message

Input Messages

A collapsible section that displays the input messages sent to the LLM.

Header Badges

Badge	Description
Input Tokens	Input token count
Cache Tokens	Token count retrieved from cache (displayed only when applicable)
Input Cost ($)	Input cost

Message Types

Label	Description
SYSTEM	System message. Instructions defining the model's role and behavior.
USER	User input message. The actual prompt content.

When messages are collected in chunks, they are automatically assembled in the correct order to display the complete text.

Output Messages

A collapsible section that displays the response generated by the model.

Header Badges

Badge	Description
Output Tokens	Output token count
Reasoning Tokens	Reasoning token count (displayed only when applicable)
Output Cost ($)	Output cost

Message Types

Label	Description
ASSISTANT	Text response from the model.
TOOL CALL	Tool call (Function Calling) requested by the model.
TOOL RESULT	Execution result of the tool call.

Prompt Delta

In calls that build up conversation context and invoke the LLM several times, such as agents or workflows, the beginning of the prompt is almost identical at every step. The truncated beginning shown in the list alone does not tell you which step asked something new.

To reduce this problem, Prompt Delta folds away the part that overlaps with the previous LLM call and leaves only the part newly added in this call in the body. It is applied automatically to consecutive LLM calls in the Table View of the LLM trace details and in the step details of the Trajectory View.

Delta summary line

A one-line delta summary is attached above the prompt body. From the left, the order is comparison type tag → number of characters removed → number of characters added → secondary values.

Tag	Meaning
No comparison target	This is the first LLM call in the transaction, so there is no previous call to compare with.
Same as previous · Same as #{stepNumber}	The prompt is exactly the same as that of the previous call.
Compared with previous · Compared with #{stepNumber}	The previous prompt was carried over as is and content was appended after it. This is the most common form.
Context reduced	Part of the beginning is gone. It appears when processing that trims accumulated context was applied.
Branched	The common beginning is shared, but the part after it diverged.
New context	There is no beginning that overlaps with the previous prompt. You can regard it as a newly started conversation.

The secondary values are shown as follows.

Display	Meaning
+{n} chars	Number of characters newly added in this call
−{n} chars	Number of characters that existed only in the previous call
±0	Exactly the same as the previous call, with no characters added or removed
Overlap {n}%	Ratio of characters in this prompt that overlap with the previous one. It is rounded down.
{n} leading blocks shared	Number of blocks shared from the beginning when the prompt is viewed in message block units
{a} chars → {b} chars	Total length before and after the reduction

Note

The delta is a value estimated by comparing the beginning with the previous call. If the prompt body is not parsed as an array of message blocks, the block-level decision is skipped and only a character-level comparison is made. In that case, a notice saying that whether the beginning was carried over as is was not determined is shown as well.

Block summary display

The newly added part is not cut from the original text as is; it is summarized by block type. When the beginning follows a fixed schema, cutting the original text as is makes every row look the same.

Block type	Displayed content
reasoning	Shows only the reasoning summary body. The encrypted body (encrypted_content) is not shown.
function_call	Shown in the form name(arguments).
function_call_output	Shown in the form call ID → result.
{role, content}	Shows the content body.

The number of resent characters is counted based on the original length, not the summarized length. Counting by the summarized length would leave large fields such as encrypted bodies out of the statistics.

Comparing in the details

Clicking the delta summary line or the prompt opens the LLM API Call panel. In the Input area of the panel, you can continue to check the result compared with the previous call.

• Leading {n} chars omitted — The beginning that overlapped with the previous call and was folded away. Expand it to see the original text.
• {n} new chars — The part newly added in this call.
• {n} chars only in the previous call — The part that was dropped in this call.
• Clicking the Go to new part button at the top of the Input Messages area moves straight to the part newly added in this call.
• The Previous LLM call and Next LLM call buttons let you move between the neighboring calls of the same transaction. The header shows the position as LLM call {current} / {total}.

GPU Correlation

Displayed on the right side of the LLM API Call panel when a multi-transaction (mtid exists) is present. Shows real-time charts of GPU infrastructure status for the Pod that processed the LLM call, helping determine whether model response delays are caused by GPU resource shortages.

GPU Information Header

Property	Description
Pod	Kubernetes Pod name that performed the LLM inference
Pod List button	Click to open the Pod list in Kubernetes monitoring, filtered by this Pod name, in a new tab.

GPU Charts

Displays GPU metrics as line charts for a 5-minute window around the LLM call time.

Chart	Metric	Unit	Description
GPU Utilization	DCGM_FI_DEV_GPU_UTIL (DCGM_FI_PROF_GR_ENGINE_ACTIVE × 100 for MIG instances)	%	GPU compute resource utilization. Near 100% indicates GPU saturation, which may cause LLM response delays.
VRAM Usage	DCGM_FI_DEV_FB_USED	MiB	GPU memory (Video RAM) usage. Used for model loading and inference; insufficient memory causes OOM or swapping.
GPU Temperature	DCGM_FI_DEV_GPU_TEMP	°C	GPU temperature. Overheating triggers automatic throttling, degrading performance.
Power Usage	DCGM_FI_DEV_POWER_USAGE	W	GPU power consumption. Viewed alongside Utilization to gauge actual compute load.

Note

GPU correlation is displayed when all of the following conditions are met:

• The transaction is a multi-transaction (mtid exists)
• The Pod name that performed LLM inference is identified
• DCGM GPU metrics are being collected in the linked Kubernetes project

Analysis Scenarios

Identifying LLM Response Delay Causes

1. Drag an area with long response times in the Hitmap of the Overview dashboard. Select a transaction from the list in the LLM trace details that opens.
2. In the Table View, check the Elapsed time of the LLM API step.
3. Click the LLM API step to open the LLM API Call panel, and compare the Latency value with the GPU Utilization chart.
4. If GPU Utilization is high when Latency spikes, GPU resource shortage is the cause.
5. If GPU Utilization is low but Latency is high, suspect provider-side delays (queue wait, Rate Limit, etc.).

Reproducing Failed LLM Calls

1. Select the Error filter in the LLM trace details list, and click an LLM API step marked with an error in the Table View.
2. Check the error type from the error class and error message in the LLM API Call panel.
3. Review the SYSTEM and USER message originals in the Input Messages section.
4. Use the collected prompt originals and model/parameter information to reproduce the same call and analyze the issue.

Analyzing High-Cost LLM Calls

1. Identify high-cost time periods on the Cost Analysis page.
2. Click that point in the Cost Trend chart to open Prompt and response search, click a prompt to open the LLM trace details, and then click an LLM API step.
3. Check the Tokens count and Cost in the LLM API Call panel.
4. Check the Input Tokens badge in the Input Messages section and review whether prompts are unnecessarily long.
5. If there is no Cache Tokens badge, caching was not applied to this call; review whether the prompt is a caching candidate.

Checking stopped or blocked calls

1. Drag the relevant point in the Hitmap on a dashboard or in Analysis > Hitmap to open the LLM trace details, and select the transaction from the list.
2. Check the message steps in the Table View. If you stopped the trajectory, WhaTap: LLM trajectory stopped by control is shown; if the call is from a blocked user, WhaTap: LLM call blocked for user … is shown.
3. If the WhaTap: LLM trajectory stopped by control message is present, the transaction ended because the trajectory was stopped, not because of an application error. If the WhaTap: LLM call blocked for user … message is present, the call was blocked by the user blocking setting and is returned to the application as a connection error (for example, APIConnectionError).

Reference

Data Collection Architecture

The LLM API Call panel combines two data sources.

Data Source	Description	Usage
APM Trace	HTTP call information (URL, host, port, elapsed, error class/message), Step ID	LLM API step identification in the Table View and HTTP-level information display
#LlmCallLog Log	Model, provider, tokens, cost, message content	Prompt original restoration and LLM-level metadata display

The two data sources are linked by Step ID.

Message Chunk Assembly

LLM prompts and responses can be long, so the server collects them split into multiple chunks. The LLM API Call panel assembles chunks in the correct order based on the chunk_index field to restore the complete message. Up to 100 chunks per type are supported.