LLM API Analysis (Transaction Linkage)

Home > Select Project > Analysis > LLM API Analysis

AI Agent Observability integrates with WhaTap APM to trace LLM API calls as part of application transactions. Clicking an LLM HTTPC step in a transaction profile opens the LLM API Detail Drawer, where you can view prompt input/output, token usage, cost, error information, and GPU infrastructure correlation all in one screen.

Note

Viewing prompt input/output requires log read permission.

How to Access

1. Select a transaction from the hitmap or Transaction Search on the LLM Dashboard.
2. In the transaction profile, click the LLM HTTPC step (HTTP external call made to an LLM provider URL).
3. The LLM API Detail Drawer opens on the right.

Summary and Prompt

This area shows key information about the LLM call at a glance and allows you to view the original prompt input/output.

Identity Tags

Displays the target model and provider information as tags.

Tag	Description	Example
Model	LLM model used	gpt-4o, claude-sonnet-4-20250514
Provider	LLM API provider	api.openai.com, api.anthropic.com
Operation	Operation type of the call	chat, completion

If the call failed (success=false), a red Error tag is additionally displayed.

Key Metrics

Displayed inline next to the Identity tags.

Metric	Description
Token	Total token count (Input + Output)
Cost	Cost of the call ($)
Latency	Time from request start to response completion (ms)

HTTP Information

Property	Description
URL	Call endpoint (host:port + path)
Elapsed	HTTP call elapsed time (ms)
Step ID	Unique identifier for this LLM call. Used as the key for navigating to log search.

Click the Log Search button next to Step ID to open the log explorer filtered by that Step ID in a new window.

Error Information

Displayed only when an error occurred in the call.

Property	Description
Error Class	Error class name (e.g., RateLimitError, TimeoutError)
Error Message	Detailed error message

Prompt Input (Input)

A collapsible section that displays the input messages sent to the LLM.

Header Badges

Badge	Description
Input Tokens	Input token count
Cached Tokens	Token count retrieved from cache (displayed only when applicable)
Input Cost ($)	Input cost

Message Types

Label	Description
SYSTEM	System message. Instructions defining the model's role and behavior.
USER	User input message. The actual prompt content.

When messages are collected in chunks, they are automatically assembled in the correct order to display the complete text.

Model Response (Output)

A collapsible section that displays the response generated by the model.

Header Badges

Badge	Description
Output Tokens	Output token count
Reasoning Tokens	Reasoning token count (displayed only when applicable)
Output Cost ($)	Output cost

Message Types

Label	Description
ASSISTANT	Text response from the model.
TOOL CALL	Tool call (Function Calling) requested by the model.
TOOL RESULT	Execution result of the tool call.

Prompt Delta

In calls that build up conversation context and invoke the LLM several times, such as agents or workflows, the beginning of the prompt is almost identical at every step. The truncated beginning shown in the list alone does not tell you which step asked something new.

To reduce this problem, Prompt Delta folds away the part that overlaps with the previous LLM call and leaves only the part newly added in this call in the body. It is applied automatically to consecutive LLM calls in the step table of the transaction profile.

Delta summary line

A one-line delta summary is attached above the prompt body. From the left, the order is comparison type tag → number of characters removed → number of characters added → secondary values.

Tag	Meaning
No comparison target	This is the first LLM call in the transaction, so there is no previous call to compare with.
Same as previous · Same as #{stepNumber}	The prompt is exactly the same as that of the previous call.
Compared with previous · Compared with #{stepNumber}	The previous prompt was carried over as is and content was appended after it. This is the most common form.
Context reduced	Part of the beginning is gone. It appears when processing that trims accumulated context was applied.
Branched	The common beginning is shared, but the part after it diverged.
New context	There is no beginning that overlaps with the previous prompt. You can regard it as a newly started conversation.

The secondary values are shown as follows.

Display	Meaning
+{n} chars	Number of characters newly added in this call
−{n} chars	Number of characters that existed only in the previous call
±0	Exactly the same as the previous call, with no characters added or removed
Overlap {n}%	Ratio of characters in this prompt that overlap with the previous one. It is rounded down.
{n} leading blocks shared	Number of blocks shared from the beginning when the prompt is viewed in message block units
{a} chars → {b} chars	Total length before and after the reduction

Note

The delta is a value estimated by comparing the beginning with the previous call. If the prompt body is not parsed as an array of message blocks, the block-level decision is skipped and only a character-level comparison is made. In that case, a notice saying that whether the beginning was carried over as is was not determined is shown as well.

Block summary display

The newly added part is not cut from the original text as is; it is summarized by block type. When the beginning follows a fixed schema, cutting the original text as is makes every row look the same.

Block type	Displayed content
reasoning	Shows only the reasoning summary body. The encrypted body (encrypted_content) is not shown.
function_call	Shown in the form name(arguments).
function_call_output	Shown in the form call ID → result.
{role, content}	Shows the content body.

The number of resent characters is counted based on the original length, not the summarized length. Counting by the summarized length would leave large fields such as encrypted bodies out of the statistics.

Comparing in the details

Clicking the delta summary line or the prompt opens the LLM call details. In the Input area of the details, you can continue to check the result compared with the previous call.

• Leading {n} chars omitted — The beginning that overlapped with the previous call and was folded away. Expand it to see the original text.
• {n} new chars — The part newly added in this call.
• {n} chars only in the previous call — The part that was dropped in this call.
• Clicking the Go to new part button scrolls straight to the newly added part.
• The Previous LLM call and Next LLM call buttons let you move between the neighboring calls of the same transaction. The header shows the position as LLM call {current} / {total}.

GPU Correlation

Displayed on the right side of the Drawer when a multi-transaction (mtid exists) is present. Shows real-time charts of GPU infrastructure status for the Pod that processed the LLM call, helping determine whether model response delays are caused by GPU resource shortages.

GPU Information Header

Property	Description
Pod	Kubernetes Pod name that performed the LLM inference
Pod Detail button	Click to open the Pod detail page in Kubernetes monitoring in a new window.

GPU Charts

Displays GPU metrics as line charts for a 5-minute window around the LLM call time.

Chart	Metric	Unit	Description
GPU Utilization	DCGM_FI_DEV_WEIGHTED_GPU_UTIL	%	GPU compute resource utilization. Near 100% indicates GPU saturation, which may cause LLM response delays.
VRAM Usage	DCGM_FI_DEV_FB_USED	MiB	GPU memory (Video RAM) usage. Used for model loading and inference; insufficient memory causes OOM or swapping.
GPU Temperature	DCGM_FI_DEV_GPU_TEMP	°C	GPU temperature. Overheating triggers automatic throttling, degrading performance.
Power Usage	DCGM_FI_DEV_POWER_USAGE	W	GPU power consumption. Viewed alongside Utilization to gauge actual compute load.

Note

GPU correlation is displayed when all of the following conditions are met:

• The transaction is a multi-transaction (mtid exists)
• The Pod name that performed LLM inference is identified
• DCGM GPU metrics are being collected in the linked Kubernetes project

Analysis Scenarios

Identifying LLM Response Delay Causes

1. Click a transaction with long response time from the hitmap on the LLM Dashboard.
2. Check the Elapsed time of the LLM HTTPC step in the transaction profile.
3. Open the LLM API Detail Drawer and compare the Latency value with the GPU Utilization chart.
4. If GPU Utilization is high when Latency spikes, GPU resource shortage is the cause.
5. If GPU Utilization is low but Latency is high, suspect provider-side delays (queue wait, Rate Limit, etc.).

Reproducing Failed LLM Calls

1. Click the LLM HTTPC step with an error in the transaction profile.
2. Check the error type from Error Class and Error Message in the Drawer.
3. Review the SYSTEM and USER message originals in the Prompt Input (Input) section.
4. Use the collected prompt originals and model/parameter information to reproduce the same call and analyze the issue.

Analyzing High-Cost LLM Calls

1. Identify high-cost time periods on the Cost Analysis page.
2. Search transactions for that time period, open the profile, and click the LLM HTTPC step.
3. Check the Token count and Cost in the Drawer.
4. Check the Input Tokens badge in the Prompt Input section and review whether prompts are unnecessarily long.
5. If the Cached Tokens badge shows 0, caching was not applied to this call -- review whether the prompt is a caching candidate.

Reference

Data Collection Architecture

The LLM API Detail Drawer combines two data sources.

Data Source	Description	Usage
APM Trace	HTTP call information (URL, host, port, elapsed, error class/message), Step ID	Call identification in transaction profile and HTTP-level information display
#LlmCallLog Log	Model, provider, tokens, cost, success status, message content	Prompt original restoration and LLM-level metadata display

The two data sources are linked by Step ID.

Message Chunk Assembly

LLM prompts and responses can be long, so the server collects them split into multiple chunks. The Drawer assembles chunks in the correct order based on the chunk_index field to restore the complete message. Up to 100 chunks per type are supported.