LLM API Analysis (Transaction Linkage)
Home > Select Project > Analysis > LLM API Analysis
AI Agent Observability integrates with WhaTap APM to trace LLM API calls as part of application transactions. Clicking an LLM HTTPC step in a transaction profile opens the LLM API Detail Drawer, where you can view prompt input/output, token usage, cost, error information, and GPU infrastructure correlation all in one screen.
Viewing prompt input/output requires log read permission.
How to Access
- Select a transaction from the hitmap or Transaction Search on the LLM Dashboard.
- In the transaction profile, click the LLM HTTPC step (HTTP external call made to an LLM provider URL).
- The LLM API Detail Drawer opens on the right.
Summary and Prompt
This area shows key information about the LLM call at a glance and allows you to view the original prompt input/output.
Identity Tags
Displays the target model and provider information as tags.
| Tag | Description | Example |
|---|---|---|
Model | LLM model used | gpt-4o, claude-sonnet-4-20250514 |
Provider | LLM API provider | api.openai.com, api.anthropic.com |
Operation | Operation type of the call | chat, completion |
If the call failed (success=false), a red Error tag is additionally displayed.
Key Metrics
Displayed inline next to the Identity tags.
| Metric | Description |
|---|---|
Token | Total token count (Input + Output) |
Cost | Cost of the call ($) |
Latency | Time from request start to response completion (ms) |
HTTP Information
| Property | Description |
|---|---|
URL | Call endpoint (host:port + path) |
Elapsed | HTTP call elapsed time (ms) |
Step ID | Unique identifier for this LLM call. Used as the key for navigating to log search. |
Click the Log Search button next to Step ID to open the log explorer filtered by that Step ID in a new window.
Error Information
Displayed only when an error occurred in the call.
| Property | Description |
|---|---|
Error Class | Error class name (e.g., RateLimitError, TimeoutError) |
Error Message | Detailed error message |
Prompt Input (Input)
A collapsible section that displays the input messages sent to the LLM.
Header Badges
| Badge | Description |
|---|---|
Input Tokens | Input token count |
Cached Tokens | Token count retrieved from cache (displayed only when applicable) |
Input Cost ($) | Input cost |
Message Types
| Label | Description |
|---|---|
| SYSTEM | System message. Instructions defining the model's role and behavior. |
| USER | User input message. The actual prompt content. |
When messages are collected in chunks, they are automatically assembled in the correct order to display the complete text.
Model Response (Output)
A collapsible section that displays the response generated by the model.
Header Badges
| Badge | Description |
|---|---|
Output Tokens | Output token count |
Reasoning Tokens | Reasoning token count (displayed only when applicable) |
Output Cost ($) | Output cost |
Message Types
| Label | Description |
|---|---|
| ASSISTANT | Text response from the model. |
| TOOL CALL | Tool call (Function Calling) requested by the model. |
| TOOL RESULT | Execution result of the tool call. |
GPU Correlation
Displayed on the right side of the Drawer when a multi-transaction (mtid exists) is present. Shows real-time charts of GPU infrastructure status for the Pod that processed the LLM call, helping determine whether model response delays are caused by GPU resource shortages.
GPU Information Header
| Property | Description |
|---|---|
Pod | Kubernetes Pod name that performed the LLM inference |
| Pod Detail button | Click to open the Pod detail page in Kubernetes monitoring in a new window. |
GPU Charts
Displays GPU metrics as line charts for a 5-minute window around the LLM call time.
| Chart | Metric | Unit | Description |
|---|---|---|---|
| GPU Utilization | DCGM_FI_DEV_WEIGHTED_GPU_UTIL | % | GPU compute resource utilization. Near 100% indicates GPU saturation, which may cause LLM response delays. |
| VRAM Usage | DCGM_FI_DEV_FB_USED | MiB | GPU memory (Video RAM) usage. Used for model loading and inference; insufficient memory causes OOM or swapping. |
| GPU Temperature | DCGM_FI_DEV_GPU_TEMP | °C | GPU temperature. Overheating triggers automatic throttling, degrading performance. |
| Power Usage | DCGM_FI_DEV_POWER_USAGE | W | GPU power consumption. Viewed alongside Utilization to gauge actual compute load. |
GPU correlation is displayed when all of the following conditions are met:
- The transaction is a multi-transaction (mtid exists)
- The Pod name that performed LLM inference is identified
- DCGM GPU metrics are being collected in the linked Kubernetes project
Analysis Scenarios
Identifying LLM Response Delay Causes
- Click a transaction with long response time from the hitmap on the LLM Dashboard.
- Check the Elapsed time of the LLM HTTPC step in the transaction profile.
- Open the LLM API Detail Drawer and compare the Latency value with the GPU Utilization chart.
- If GPU Utilization is high when Latency spikes, GPU resource shortage is the cause.
- If GPU Utilization is low but Latency is high, suspect provider-side delays (queue wait, Rate Limit, etc.).
Reproducing Failed LLM Calls
- Click the LLM HTTPC step with an error in the transaction profile.
- Check the error type from Error Class and Error Message in the Drawer.
- Review the SYSTEM and USER message originals in the Prompt Input (Input) section.
- Use the collected prompt originals and model/parameter information to reproduce the same call and analyze the issue.
Analyzing High-Cost LLM Calls
- Identify high-cost time periods on the Cost Analysis page.
- Search transactions for that time period, open the profile, and click the LLM HTTPC step.
- Check the Token count and Cost in the Drawer.
- Check the Input Tokens badge in the Prompt Input section and review whether prompts are unnecessarily long.
- If the Cached Tokens badge shows 0, caching was not applied to this call -- review whether the prompt is a caching candidate.
Reference
Data Collection Architecture
The LLM API Detail Drawer combines two data sources.
| Data Source | Description | Usage |
|---|---|---|
| APM Trace | HTTP call information (URL, host, port, elapsed, error class/message), Step ID | Call identification in transaction profile and HTTP-level information display |
#LlmCallLog Log | Model, provider, tokens, cost, success status, message content | Prompt original restoration and LLM-level metadata display |
The two data sources are linked by Step ID.
Message Chunk Assembly
LLM prompts and responses can be long, so the server collects them split into multiple chunks. The Drawer assembles chunks in the correct order based on the chunk_index field to restore the complete message. Up to 100 chunks per type are supported.