Skip to main content

Server Metrics V2

Home > Select Project > Analysis > Metrics Search

Metrics available in the Metrics Search menu. Each metric contains the following information. Some metrics may not have tag information.

  • Tags: Information to identify the collection target (agent name, IP, etc.)
  • Fields: Actual measured values (CPU usage, response time, etc.)

The CategoryV2 metrics of WhaTap server monitoring are a metric system that standardizes server resources in operational environments by domain. From core system resources such as CPU, memory, Swap, and kernel to major infrastructure indicators including disk device I/O, filesystem, network traffic, NIC configuration, NTP synchronization, processes, Windows services, InfiniBand, ZFS, and Veritas LV, they are provided in a consistent structure across on-premises and cloud server environments based on Linux, Windows, and Unix (HP-UX, AIX, Solaris).

CategoryV2 provides both categories that compare resource usage across all servers and categories that analyze each server's components at the detailed instance level. With these, operators can quickly grasp the resource distribution and anomalies of the entire infrastructure, and progressively trace down to the detailed components of a server where a problem occurred.

Whereas the existing CategoryV1 grouped indicators from multiple domains into a few categories, CategoryV2 improves the analysis structure and collection efficiency by separating categories per domain and differentiating collection intervals according to indicator characteristics. The collected CategoryV2 metrics integrate with the Metrics Explorer, so users can directly query the indicators they want and perform detailed analysis under various conditions.

Note

CategoryV2 can be used by enabling the categoryv2.enabled=true option in the agent settings.

CategoryV2 metric details

infra_cpu

A category that collects the usage state of the server's overall CPU resources. It provides indicators for checking CPU usage and load level across the board, such as user mode, kernel mode, I/O wait, idle, interrupt, steal, and load average. Use it to identify the server's overall CPU bottlenecks and system load state.

  • Category: CPU
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
cpuCPU%It refers to the overall CPU utilization excluding the idle percentage at 100%.HP-UX, AIX, Solaris, Linux, Windows
userUser%It refers to the ratio of total CPU time spent executing general applications/processes in user mode.HP-UX, AIX, Solaris, Linux, Windows
sysSys%The ratio of total CPU time spent in kernel mode (System mode) for handling system calls, kernel tasks, and driver processing.HP-UX, AIX, Solaris, Linux, Windows
niceNice%It refers to the percentage of total CPU time used for executing low-priority (niced) user processes. The utilization of user processes with normal priority is typically included in user (%).HP-UX, AIX, Solaris, Linux
waitWait%The ratio of total CPU time during which the CPU was idle waiting for I/O completion (I/O wait). This is interpreted as the time spent waiting for responses from external resources such as disk/network, rather than the time the CPU actually performed computations.HP-UX, AIX, Solaris, Linux
totalUser+Sys%The sum of the ratios used in user mode and kernel mode out of the total CPU utilization. Generally interpreted as the pure CPU execution utilization, excluding I/O wait, irq, softirq, and steal.HP-UX, AIX, Solaris, Linux, Windows
total_waitUser+Sys+Wait%The sum of user, system, and wait time out of the total CPU time. This is a usage metric that includes user processing, kernel processing, and I/O waiting.HP-UX, AIX, Solaris, Linux
idleIdle%The ratio of total CPU time during which the CPU was idle and not performing any tasks. A higher value indicates a greater availability of CPU resources.HP-UX, AIX, Solaris, Linux, Windows
lCoresLogical CorescountThis refers to the number of logical CPUs (logical cores) recognized by the OS. It may differ from the number of physical cores, and when Hyper-Threading/SMT is enabled, the number of logical cores may appear larger.HP-UX, AIX, Solaris, Linux, Windows
irqIrq%The ratio of total CPU time spent handling hardware interrupts (IRQ). It may increase when the load of processing network cards, disks, and other hardware events is high.Linux
softirqSoftirq%It refers to the ratio of total CPU time spent handling Software interrupts (SoftIRQ). This can increase when there are many deferred processing tasks in the kernel, such as network packet processing and block I/O post-processing.Linux
stealSteal%In a virtualized environment, the ratio of time that the current virtual CPU is unable to execute due to the hypervisor prioritizing other virtual CPUs is referred to as CPU steal time. A high value may indicate insufficient CPU resources allocated to the VM or a contention state.Linux
procUtilProcessor Util%A Windows-specific metric that refers to the overall CPU utilization of the entire system's processors. It is the same as the total CPU utilization seen in the Windows Task Manager.Windows
loadavg_1mLoadAvg 1mcountThe average system load (load average) over the last 1 minute. It reflects tasks that are currently running or waiting to run, as well as tasks in an uninterruptible wait state, and is a different concept from CPU utilization.HP-UX, AIX, Solaris, Linux
loadavg_5mLoadAvg 5mcountThe average system load over the last 5 minutes. It is used to observe a trend that is slightly smoother than short-term fluctuations.HP-UX, AIX, Solaris, Linux
loadavg_15mLoadAvg 15mcountThe average system load over the last 15 minutes. It is used to check the long-term trend of system load.HP-UX, AIX, Solaris, Linux

infra_mem

A category that collects the server's overall memory usage state. It provides indicators for checking memory usage and pressure across the board, such as total memory capacity, usage, usage rate, cache, buffer, available memory, slab, page fault, and swap in/out. Use it to identify memory shortage, reclaim, paging, and cache utilization state.

  • Category: MEMORY
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
pUsedUsed%This refers to the ratio of currently used memory to the total memory. Linux typically interprets used as total - available based on MemAvailable, considering page cache and reclaimable memory as free memory. Windows generally interprets usage based on the amount excluding Available. AIX, HP-UX, and Solaris do not have a concept equivalent to Linux's MemAvailable, so they usually calculate based on the used value provided by the OS.AIX, HP-UX, Solaris, Linux, Windows
usedUsedbyteThis refers to the current memory usage. Linux typically calculates this based on the value of MemTotal - MemAvailable, converted to bytes. Windows generally interprets this based on usage excluding Available. AIX, HP-UX, and Solaris do not have a MemAvailable criterion like Linux, so they derive this from the used value provided by the OS.AIX, HP-UX, Solaris, Linux, Windows
pCacheCache%The ratio of the cache area to the total memory. In Linux, it is generally based on the Cached value from /proc/meminfo, referring to memory of the page cache nature, but does not include Buffers or SReclaimable. AIX, HP-UX, and Solaris do not have a concept equivalent to Linux's MemAvailable, so the cache value provided by the OS is typically used for calculation.HP-UX, AIX, Solaris, Linux
cacheCachebyteThe size of memory used as cache. In Linux, it is generally based on the Cached value from /proc/meminfo, referring to memory of the page cache nature, but does not include Buffers or SReclaimable. AIX, HP-UX, and Solaris do not have a concept equivalent to Linux's MemAvailable, so the cache value provided by the OS is typically used for calculation.HP-UX, AIX, Solaris, Linux
bufferBufferbyteThis value represents the amount of memory used as I/O buffers, expressed in bytes. It is a Linux-specific metric based on the Buffers value from /proc/meminfo.Linux
pUsed_pCacheUsed+Cache%This is an indicator that represents the current usage rate (pUsed) and cache ratio (pCache) together. It serves as a reference metric to view the proportion of memory actually in use by users/kernel and the cache area, and is more of a composite indicator for operational interpretation rather than a standard memory metric.HP-UX, AIX, Solaris, Linux
sharedShared SizebyteThis is the value representing the size of shared memory used across the entire server in bytes. It is a Linux-specific metric and may include tmpfs, shared memory, and some shared mapping areas.Linux
sizeTotal SizebyteThe value representing the total physical memory size allocated to the system in bytes.HP-UX, AIX, Solaris, Linux, Windows
availableAvailablebyteThis value represents the currently available memory size in bytes. In Linux, it refers to MemAvailable from /proc/meminfo, which is closer to an 'immediate usable estimate' that reflects not only free memory but also reclaimable cache. While instantaneous values are important, a continuous decrease over time, along with an increase in pageMajFaults, swapIn, and swapOut, may indicate that actual memory pressure is increasing.Linux
sreclaimableSreclaimablebyteThis value represents the size of the reclaimable area in the kernel slab cache in bytes. It may be reclaimed under memory pressure. A larger value does not necessarily indicate a problem; it also reflects the current level of cache utilization. However, if it excessively increases over time or if there is a decrease in available memory without proper reclamation, further investigation may be necessary.Linux
sunreclaimSunreclaimbyteThis is the value representing the size of the area in the kernel slab cache that cannot be reclaimed, expressed in bytes. Since this memory must be retained by the kernel, it may have a more direct implication in terms of memory pressure compared to SReclaimable. A pattern of continuous increase in this value can help interpret the expansion of kernel memory usage or the possibility of abnormal accumulation.Linux
slabSlabbyteThis value represents the total size of slab memory used by the kernel for object caching, expressed in bytes. It generally refers to the Slab in /proc/meminfo and is interpreted as the sum of SReclaimable and SUnreclaim. While the current value is important, a continuous increase over time may indicate an expansion of kernel memory usage. Particularly, if there is an accompanying increase in SUnreclaim, it is necessary to consider the possibility of actual kernel memory pressure rather than just an increase in reclaimable cache.Linux
pageFaultsPage Faultscounts/sThis metric represents the total number of page faults that occurred during memory access across the entire system, expressed as a per-second value. It includes both minor faults and major faults, and does not necessarily involve disk I/O. While a momentary increase alone may not indicate a problem, a sustained rise in value, especially when accompanied by increases in pageMajFaults, scanned, and swapIn/Out, can help interpret the possibility of memory pressure or an increase in page cache misses.Linux
pageMajFaultsPage Major Faultscounts/sThis metric represents the number of major page faults per second, which indicates the instances where the system had to actually read the required pages from disk/storage. The source of the reads can be file-backed pages from the filesystem or the swap area. If the value remains consistently high or shows an increasing trend, it suggests potential memory pressure that involves actual I/O, rather than just an increase in memory access, and can be directly linked to response latency. It is important to observe this alongside swapIn, fsIn, and available metrics.Linux
scannedPage Scanpages/sThis refers to the number of pages scanned by the system to reclaim memory. A higher value indicates more active reclaim activities for memory, and consistently high levels may suggest increasing memory pressure. It is important to interpret the duration of sustained increases rather than one-time spikes, along with the relationship to freed, steal, and swapIn/Out.HP-UX, AIX, Solaris, Linux
freedPage Freepages/sThe number of free pages secured through page reclamation, page release, etc. This is an indicator that shows the results of the system's reclaim or memory return activities. It is important to observe the relationship with scanned as well as the continuous increase of the value itself. If scanned is high but freed does not keep up sufficiently, it may indicate low memory reclamation efficiency.HP-UX, AIX, Solaris, Linux
stealPage Stealpages/sIt refers to the actual number of pages reclaimed during the page reclaim process in Linux. This is associated with the process of securing page cache or inactive pages in situations of memory shortage. A temporary increase in this value may indicate normal reclaim behavior; however, if it remains consistently high, it may suggest that the system is continuously reclaiming pages under memory pressure.Linux
stealRatioPage Steal Ratio%This refers to the ratio of actual pages retrieved from the pages scanned in Linux. It is generally calculated as steal / scanned * 100, where a higher value indicates better retrieval efficiency compared to scanning, and a lower value may suggest a higher proportion of active pages, indicating a situation where retrieval is not effective. (0 has no meaning) However, the interpretation may vary depending on the workload characteristics, so it is important to consider it alongside scanned, freed, and available.Linux
swapInSwap Inpages/sIt refers to the number of pages read back into memory per second from the swap (paging space) area of the disk. If the value consistently occurs or shows an increasing trend, it may indicate a higher dependency on swap due to memory shortages, which could be directly linked to response delays. It is advisable to prioritize the persistence of occurrences over temporary spikes.HP-UX, AIX, Solaris, Linux, Windows
swapOutSwap Outpages/sIt refers to the number of pages exported per second from memory to the disk's swap (paging space) area. A continuous occurrence of this value indicates that the system is pushing pages to swap to free up memory. A consistently high pattern observed alongside swapIn may suggest a memory pressure state with a high potential for performance degradation.HP-UX, AIX, Solaris, Linux, Windows
fsInFilesystem Inpages/sThis refers to the number of pages that have been brought into memory by file system read operations. Unlike Swap In, it specifically refers to pages loaded by file system I/O. While the current value is significant, the increasing pattern along with pageMajFaults, the process's rdiskIo, and available can help interpret that actual file-based page-in activity is increasing.HP-UX, AIX, Solaris, Linux, Windows
fsOutFilesystem Outpages/sThis refers to the number of pages written to disk as a result of filesystem write operations. Unlike Swap Out, it is more appropriate to interpret this as the volume of page writes of the filesystem writeback/flush nature. Continuous increases or periodic spikes in the value can be correlated with the characteristics of operations such as flush/writeback/checkpoint.HP-UX, AIX, Solaris, Linux, Windows
poolPagedPool PagedbyteA Windows-specific metric that represents the size in bytes of the paged pool. It refers to a kernel memory area that can be paged to disk if necessary.Windows
poolNonPagedPool NonpagedbyteA Windows-specific metric that indicates the byte size of the nonpaged pool. It refers to a kernel memory area that must always reside in physical memory.Windows
systemCacheResidentSystem Cache ResidentbyteA Windows-specific metric that indicates the size in bytes of the system cache memory currently residing in physical memory. It reflects the kernel memory usage related to the file system cache.Windows

infra_swap

A category that collects the server's overall swap or paging space usage state. With total swap size, usage, and usage rate, you can grasp the level of disk-based virtual memory use. Use it to check whether swap is being used due to memory shortage and the overall virtual memory pressure state.

  • Category: SWAP
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
sizeTotal SizebyteThis is the value representing the total size of the swap area configured in the system, expressed in bytes. In Unix/Linux systems, it refers to the total size of the swap/paging space, while in Windows, it corresponds to the total size of the page file-based virtual memory area.HP-UX, AIX, Solaris, Linux, Windows
pUsedUsed%This refers to the current percentage of swap used compared to the total swap space. It is calculated as used / size * 100. A higher value may indicate an increased reliance on disk-based virtual memory due to memory shortages.HP-UX, AIX, Solaris, Linux, Windows
usedUsedbyteThis is the value representing the size of the currently used swap space in bytes. In Unix/Linux systems, it refers to the actual occupied space in the swap/paging space, while in Windows, it corresponds to the page file usage.HP-UX, AIX, Solaris, Linux, Windows

infra_kernel

A category that collects the OS kernel and system behavior state. It provides operational indicators from the kernel and scheduler perspective, such as run queue, block queue, context switch, interrupt, syscall, fork, and exec. Use it to identify CPU contention, I/O wait, scheduling load, and kernel entry frequency.

  • Category: System Kernel
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
runqRunQueuecountIt refers to the number of processes or threads that are ready to run but have not yet been allocated CPU time, sitting in the run queue. A higher value may indicate CPU contention or that there are more runnable tasks than available CPU resources. It is important to view the run queue alongside CPU utilization. If CPU utilization is high and the run queue is also high, there is a significant possibility of a CPU bottleneck. Conversely, if CPU utilization is low but the run queue is high, issues such as locks, scheduling, or priority problems may be suspected.HP-UX, AIX, Solaris, Linux, Windows
blockqBlockQueuecountThe number of processes or threads that are in a waiting state and cannot be executed immediately due to I/O, paging, or resource contention. This number can generally increase when there is a high volume of disk I/O, memory waits, or resource contention. A high value suggests a potential bottleneck related to I/O, paging, or resource waits rather than a CPU shortage. It is advisable to monitor disk performance metrics alongside swap metrics.HP-UX, AIX, Solaris, Linux
pendingqPendingQueuecountThis is a metric specific to AIX, generally referring to the number of threads waiting for raw device I/O requests to complete. It may increase during periods of storage response delays or high raw I/O contention.AIX
waitqWaitQueuecountIn the context of HP-UX / Solaris systems, this refers to the number of processes or threads that have been swapped out or are temporarily removed from scheduling target and are waiting. Unlike the simple run queue, this is disctinct from being immediately runnable state that can receive CPU.HP-UX, Solaris
forkForkcounts/sThe number of fork() calls per second. Fork is a system call that creates a new child process by duplicating the current process. A high value may indicate an environment where short-lived processes are frequently created. It can naturally be high in environments that frequently generate processes, such as shell scripts, batch jobs, and CGI/process-oriented applications, so it is more important to determine if there has been a sudden increase compared to usual levels.HP-UX, AIX, Solaris, Linux
execExeccounts/sThe number of exec() calls per second. exec is a system call that replaces the memory image and execution context of the current process with a new program while maintaining the process's PID. Unlike fork, it does not create a new process but substitutes the existing process with another program. In environments where program replacement occurs frequently, such as shell scripts, batch jobs, and CGI/process-based applications, it can naturally be high, so it is more important to determine if there has been a sudden increase compared to usual levels.HP-UX, AIX, Solaris
interruptInterruptcounts/sThe number of hardware interrupts processed by the CPU per second. This can increase with a higher number of network, disk, timer, and other device events. A high value indicates that it is advisable to analyze the increase in network packet throughput, storage I/O, and whether there is a concentration of interrupts from specific devices.HP-UX, AIX, Solaris, Linux, Windows
syscallSystemCallcounts/sIt refers to the number of system calls processed by the kernel per second. This number increases when user processes request kernel services such as file I/O, networking, and process control. A high value does not necessarily indicate an anomaly, but it may reflect changes in workload characteristics or an increase in kernel entry frequency. It can be high in applications with significant file/socket I/O.HP-UX, AIX, Solaris, Windows
ctxswitchContextSwitchcounts/sIt refers to the number of context switches per second. It increases when the CPU switches from one running process or thread to another task. A higher value may indicate more multitasking, thread contention, lock contention, and frequent short-duration tasks. If the context switch count is very high, it may suggest that the CPU is spending more time on scheduling overhead than on actual work, so it is advisable to analyze it alongside run queue, system calls, interrupts, and lock contention situations.HP-UX, AIX, Solaris, Linux, Windows

infra_disk

A category that collects the aggregate performance of all physical disks connected to the server. It provides indicators for checking the server's overall disk I/O scale, such as read/write throughput (Byte/s) and IOPS. Use it to identify the server's overall disk load level.

  • Category: DISK
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
readReadbytes/sThe sum of the data read per second from all physical disks of the server, expressed in Byte/s. In other words, it is the aggregated value of the read throughput of all physical disks in the server.HP-UX, AIX, Solaris, Linux, Windows
writeWritebytes/sThe sum of the data written per second across all physical disks of the server, expressed in Byte/s. In other words, it is the aggregated value of the write throughput for all physical disks of the server.HP-UX, AIX, Solaris, Linux, Windows
totalIO Totalbytes/sThe total data throughput, expressed in Byte/s, is the sum of the read and write throughput from all physical disks of the server. This is the aggregated value of the server's overall disk I/O throughput corresponding to read + write.HP-UX, AIX, Solaris, Linux, Windows
readIopsRead IOPScounts/sThis value represents the sum of the number of read I/O operations processed per second across all physical disks of the server, expressed in IOPS. In other words, it is the aggregated read IOPS of all physical disks in the server. IOPS is a metric that indicates the number of I/O requests processed by the disk per second, without reflecting the size of the data itself. Therefore, in environments with a high number of small block I/O, IOPS may be high while throughput (Byte/s) may be relatively low, whereas in environments with a high number of large block I/O, IOPS may be low but throughput (Byte/s) may appear high.HP-UX, AIX, Solaris, Linux, Windows
writeIopsWrite IOPScounts/sThis value represents the sum of the number of write I/O operations processed per second across all physical disks of the server, expressed in IOPS. In other words, it is the aggregated write IOPS of all physical disks in the server. IOPS is a metric that indicates the number of I/O requests processed by the disk per second, without reflecting the size of the data itself. Therefore, in environments with a high number of small block I/O, IOPS may be high while throughput (Byte/s) may be relatively low, whereas in environments with a high number of large block I/O, IOPS may be low but throughput (Byte/s) may appear high.HP-UX, AIX, Solaris, Linux, Windows
totalIopsTotal IOPScounts/sThe total number of I/O operations, expressed in IOPS, is the sum of the read and write I/O operations processed per second across all physical disks of the server. It corresponds to the aggregate value of the server's total disk I/O operations, which is readIops + writeIops. IOPS is a metric that indicates the number of I/O requests processed by the disk per second, without reflecting the size of the data itself. Therefore, in environments with a high number of small block I/O operations, IOPS may be high while throughput (Byte/s) may be relatively low, whereas in environments with a high number of large block I/O operations, IOPS may be low but throughput (Byte/s) may appear high.HP-UX, AIX, Solaris, Linux, Windows
busyMaxBusy Max%Among all physical disks connected to the server, this indicates the utilization of the disk with the highest busy (%) value during the collection interval. In other words, it represents the busy level of the disk that operated the most actively among all disks on the server. A higher value may indicate that I/O load is concentrated on a specific disk or that there is a potential bottleneck. However, this value is collected at a default interval of 2 seconds, while the busy values for individual disks are collected at a default interval of 5 seconds, so individual disk busy values may not match exactly, even on charts at the same timestamp. Particularly during low busy intervals or moments of sudden load increases, the gap can look larger because of the mismatch between the collection interval and the sampling point. Therefore, this value can be used as a representative indicator to quickly check for signs of disk load at the server level, and it is advisable to examine detailed metrics for individual disks to confirm actual bottleneck disks.HP-UX, AIX, Solaris, Linux, Windows
diskNameBusy DiskstringThis is the name of the disk with the highest busy (%) value among all physical disks connected to the server.HP-UX, AIX, Solaris, Linux, Windows

infra_network

A category that aggregates and provides the server's overall network usage and per-interface usage level. It collects Tx/Rx bps, Tx/Rx pps, total traffic, maximum NIC usage rate, and the corresponding NIC name for NICs excluding loopback. With these, you can check the trend of the server's overall network load and identify the interface with the highest network usage rate.

  • Category: NETWORK
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
rxbpsNet RxbpsThis value represents the total amount of data received (Rx) per second through all NICs of the server (excluding loopback) in bps (bits per second). In other words, it is the aggregated network input bandwidth of the server.HP-UX, AIX, Solaris, Linux, Windows
txbpsNet TxbpsThis value represents the total amount of data transmitted (Tx) per second through all NICs of the server (excluding loopback), expressed in bps (bits per second). In other words, it is the aggregated network transmission bandwidth of the entire server.HP-UX, AIX, Solaris, Linux, Windows
totalbpsNet TotalbpsThis value represents the total data transfer amount received (Rx) and transmitted (Tx) per second through all NICs of the server (excluding loopback), expressed in bps (bits per second). It corresponds to the aggregate value of the server's overall network bandwidth, which is the sum of rxbps and txbps.HP-UX, AIX, Solaris, Linux, Windows
rxppsRx PPScounts/sThis is the sum of the packets received (Rx) per second through all NICs of the server (excluding loopback), expressed in pps (packets per second) or count/s. In other words, it is the aggregate value of the server's total network received packet throughput. Since pps is based on the number of packets regardless of the size of the transmitted data, a high number of small packets can result in a high pps while the bps may be relatively low.HP-UX, AIX, Solaris, Linux, Windows
txppsTx PPScounts/sThis value represents the sum of packets transmitted (Tx) per second through all NICs of the server (excluding loopback) in pps (packets per second) or count/s units. In other words, it is the aggregate value of the server's overall network transmission packet throughput. Since pps is based on the number of packets regardless of the size of the transmission, a high number of small packets can result in a high pps while the bps may be relatively low.HP-UX, AIX, Solaris, Linux, Windows
totalppsTotal PPScounts/sThe total number of packets received (Rx) and transmitted (Tx) per second through all NICs of the server (excluding loopback) is represented in pps (packets per second) or count/s units. This is the aggregate value corresponding to rxpps + txpps for the entire network packet throughput of the server. Since pps is based on the number of packets regardless of the size of the transmitted data, a high number of small packets can result in a high pps while the bps may be relatively low.HP-UX, AIX, Solaris, Linux, Windows
linkUsageMaxNIC Max Usage%This is the percentage value of the utilization of the NIC with the highest link usage (excluding loopback) among all NICs present on the server. It serves as an indicator to quickly identify which NIC among the server's network interfaces is under the highest load. Generally, it corresponds to the maximum value of the link usage of each NIC, and NICs for which link speed information cannot be obtained may be excluded from the calculation or may not reflect a value.HP-UX, AIX, Solaris, Linux, Windows
nicNameMax Usage NIC NamestringThis value indicates the name of the NIC with the highest link utilization (excluding loopback) among all NICs present on the server. It serves as an identifier for which NIC the linkUsageMax value occurred on. For example, it may display names such as eth0, ens192, bond0 on Linux, or adapter names on Windows. If there are no calculable NICs, the value may be empty.HP-UX, AIX, Solaris, Linux, Windows

infra_netstat

Information aggregating the server's TCP network connection states by state.

  • Category: Network State Summary
  • Default collection interval: 60 sec
  • Minimum supported agent: Linux 2.9.18+, Windows 2.9.18+, AIX 1.4.9+, Solaris 1.4.9+, HP-UX 1.4.9+
FieldTitleUnitDescriptionOS
netstat_ListenListencountThe number of TCP sockets in LISTEN state waiting to receive external connection requests on the current server. This is primarily used to verify whether the service port is properly opened.HP-UX, AIX, Solaris, Linux
netstat_EstablishedEstablishedcountThe number of TCP connections in the ESTABLISHED state that are currently successfully established on the server and capable of data transmission. This indicates the actual number of active sessions.HP-UX, AIX, Solaris, Linux
netstat_Close_WaitClose_WaitcountThe number of TCP connections in the CLOSE_WAIT state, where a remote request to terminate the connection has been made, but the local application has not yet fully closed the socket. If this value remains consistently high, it is necessary to check for potential socket cleanup delays or leaks in the application.HP-UX, AIX, Solaris, Linux
netstat_ClosingClosingcountThe number of TCP connections in the CLOSING state, where both sides are in the process of terminating the connection. Typically represented briefly, and if it persists or increases excessively, it is necessary to check for abnormal termination handling.HP-UX, AIX, Solaris, Linux
netstat_Fin_Wait1Fin_Wait1countThe number of TCP connections in the FIN_WAIT1 state, where the local side has initiated the connection termination and sent a FIN, waiting for a response from the remote side. This corresponds to the initial stage of the connection termination procedure.HP-UX, AIX, Solaris, Linux
netstat_Fin_Wait2Fin_Wait2countThe number of TCP connections in the FIN_WAIT2 state, which are waiting for the final termination signal from the remote side after receiving a connection termination request from the local side. If this state persists for a long time, it may indicate a termination delay or issues on the peer side.HP-UX, AIX, Solaris, Linux
netstat_Time_WaitTime_WaitcountThe number of TCP connections in the TIME_WAIT state, which is maintained for a certain period to prevent socket reuse conflicts similar to delayed packet processing after connection termination. In environments with a very high number of short connections, this value can increase significantly.HP-UX, AIX, Solaris, Linux
netstat_Syn_RecvSyn_RecvcountThe number of TCP connections in the SYN_RECV state that have received a connection request (SYN) from the remote side, responded to it, and are now waiting for final confirmation. If the value increases sharply, it is necessary to check for a surge in connection requests or the possibility of a SYN Flood attack.HP-UX, AIX, Solaris, Linux
netstat_Syn_SentSyn_SentcountThe number of TCP connections in the SYN_SENT state after the local side has sent a connection request (SYN) and is waiting for a response from the remote side. This can increase due to remote server response delays, network path issues, or firewall problems.HP-UX, AIX, Solaris, Linux
netstat_Last_AckLast_AckcountThe number of TCP connections in the LAST_ACK state after receiving a termination request from the remote side, where the local side also sends a termination response and waits for the final confirmation. This state is generally short-lived, and if it persists at a high level, it is necessary to check for any delays in the termination process.HP-UX, AIX, Solaris, Linux
netstat_UnknownUnknowncountThe number of connections that are not classified as defined major TCP states or whose states could not be clearly identified at the time of collection. Generally, this value should not be large, and if it continues to increase, it is necessary to check the collection logic or the state mapping criteria specific to the OS.HP-UX, AIX, Solaris, Linux

infra_ntp_info

A category that collects the server's time synchronization state. It provides NTP usage, synchronization daemon, synchronization state, the NTP server in use, offset, offset direction, time zone, etc. Use it to check how much the system time differs from the reference time and whether NTP synchronization is normal.

  • Category: NTP Information
  • Default collection interval: 60 sec
  • Minimum supported agent: Linux 2.9.18+, Windows 2.9.18+, AIX 1.4.9+, Solaris 1.4.9+, HP-UX 1.4.9+
FieldTitleUnitDescriptionOS
checkTimeCheck TimestringThis refers to the time when NTP information collection or inspection was performed. It indicates the local system time of the server where the Agent is installed at the moment the result was generated.HP-UX, AIX, Solaris, Linux, Windows
ntpInUseNTP In UsestringIndicates whether the NTP-based time synchronization feature is in use on the system. 'Yes' means that NTP or a time synchronization daemon is configured and operational, while 'No' indicates that it is not in use or the related daemon is not running.HP-UX, AIX, Solaris, Linux, Windows
ntpDaemonNTP DaemonstringThis indicates the name of the time synchronization daemon or service used in the system. For example, in Linux/Unix systems, it may display chrony, ntpd, xntpd, and in Windows, it may show w32time.HP-UX, AIX, Solaris, Linux, Windows
timeSyncedTime SyncedstringIndicates whether the current system is properly synchronized with the NTP server. 'Yes' means that it is currently synchronized with a valid NTP server, while 'No' indicates that NTP is in use but has not yet synchronized or has not secured a valid synchronization target.HP-UX, AIX, Solaris, Linux, Windows
ntpServerNTP ServerstringIndicates the hostname or IP address of the NTP server currently used for time synchronization. If synchronization is not functioning properly, the value may be empty.HP-UX, AIX, Solaris, Linux, Windows
offsetOffset (ms)msThis value represents the absolute difference in milliseconds (ms) between the current system time and the reference NTP server time. The direction information regarding whether it is fast or slow is distinguished in the offsetDirection field.HP-UX, AIX, Solaris, Linux, Windows
offsetDirectionOffset DirectionstringIndicates whether the current system time is faster or slower than the reference NTP server time. For example, FAST (+) means the local system time is ahead, while SLOW (-) means the local system time is behind.HP-UX, AIX, Solaris, Linux, Windows
timeZoneTime ZonestringThis indicates the time zone information set in the system. Depending on the operating system, it may be displayed in different formats such as Asia/Seoul, KST, or Korea Standard Time.HP-UX, AIX, Solaris, Linux, Windows

infra_process

A category that aggregates and shows the server's overall process state. You can check the overall process/thread state, such as total process count, thread count, zombie count, runnable/sleeping counts, and file descriptor usage. Use it to identify system operational state such as process count spikes, zombie accumulation, and FD exhaustion.

  • Category: Process State Summary
  • Default collection interval: 2 sec
FieldTitleUnitDescriptionOS
procTotalProcess TotalcountIt refers to the total number of processes currently existing in the OS. A high process count does not necessarily indicate an issue. In environments with batch processing, web server prefork structures, and many short job units, it can naturally be high.HP-UX, AIX, Solaris, Linux, Windows
threadTotalThread TotalcountThis refers to the total number of threads currently existing in the OS. It can increase with the number of multithreaded applications and may reflect system load more directly than the number of processes. The thread count can better represent the actual level of concurrency than the process count. In Java, WAS, DB, and multithreaded application environments, the trend of threadTotal is important.HP-UX, AIX, Solaris, Linux, Windows
handleTotalHandle TotalcountThe total number of handles currently open or in use in the Windows OS. This includes references to various kernel objects such as files, registry keys, processes, threads, events, and mutexes.Windows
runnableTotalRunnable TotalcountThe number of processes with a state of R in Linux refers to those that are currently running or are runnable but waiting for CPU in the run queue.Linux
sleepingTotalSleeping TotalcountIt refers to the number of processes in the S state on Linux. This state is interruptible sleep, which includes processes that can wake up due to signals while waiting for events or resource responses. It is the most common range for processes in a waiting state.Linux
idleTotalIdle TotalcountIt refers to the number of processes in the I state on Linux. From the perspective of the kernel scheduler, it indicates the number of tasks marked as idle, which may be observed more in kernel internal workers or idle-type tasks rather than in general user processes.Linux
disksleepTotalDisk Sleep TotalcountIt refers to the number of processes in the D state on Linux. This state is known as uninterruptible sleep, primarily including processes waiting for disk I/O, filesystem, devices, or kernel resources. A high and persistent value may indicate a storage or kernel wait bottleneck.Linux
zombieTotalZombie TotalcountIt refers to the number of processes that have terminated but remain in a zombie (Z) state because they have not yet been collected by the parent process. A continuous increase may indicate issues with the parent process's wait handling or delays in process collection.HP-UX, AIX, Solaris, Linux
stoppedTotalStopped TotalcountIt refers to the number of processes in the T state on Linux. Generally, it indicates processes that have been stopped by job control signals (SIGSTOP, SIGTSTP, etc.).Linux
tracingstopTotalTracing Stop TotalcountThe number of processes in the process state t in Linux. Generally, it indicates processes that are stopped in a tracing stop state by a debugger (ptrace).Linux
deadTotalDead TotalcountIt refers to the number of processes in state X on Linux. This indicates that the processes are in the process of termination or have completely terminated, but it is rarely observed in typical operating environments.Linux
wakekillTotalWake Kill TotalcountThe number of processes in the K state in Linux. This may be classified as an internal kernel state and is rarely observed in typical operating environments, or its meaning may vary depending on the kernel version.Linux
wakingTotalWaking TotalcountIt refers to the number of processes in the W state on Linux. This can be viewed as a transitional state during waking processing, and in the latest kernels, it may be rarely observed or only environment-dependent.Linux
otherTotalOther TotalcountIt refers to the number of processes with other process states than the explicitly separated R/S/I/D/Z/T/t/X/K/W in Linux. The inclusion criteria may vary depending on kernel version differences, transient states at the moment of observation, and implementation differences.Linux
openfdCurrentOpen fd UsedcountThe number of currently used file descriptors across the Linux OS. This includes the number of open objects managed by the kernel as file descriptors, such as files, sockets, pipes, and devices.Linux
openfdMaxOpen fd MaxcountIt refers to the maximum number of file descriptors set across the entire Linux OS. Generally, it is interpreted based on the system-wide file descriptor limit (/proc/sys/fs/file-max).Linux
openfdPUsedOpen fd Used%This refers to the overall file descriptor usage of the Linux OS. It is generally calculated as openfdCurrent / openfdMax * 100, and a higher value indicates an increased risk of global FD exhaustion in the system.Linux

infra_nic_info

A category that collects the configuration information of each network interface in the server. It provides static or semi-static information such as NIC name, IP, MAC, link, state, duplex, speed, mode, and driver. Use it for monitoring NIC configuration changes, checking redundancy structures, and identifying interfaces.

  • Category: NIC Information
  • Default collection interval: 60 sec
  • Minimum supported agent: Linux 2.9.18+, Windows 2.9.18+, AIX 1.4.9+, Solaris 1.4.9+, HP-UX 1.4.9+
FieldTitleUnitDescriptionOS
nicNameNIC NamestringThe name of the network interface recognized by the operating system, which can display not only physical NICs but also alias interfaces, bond/team/bridge, EtherChannel, APA, AGGR, IPMP, and their member NICs together.HP-UX, AIX, Solaris, Linux, Windows
ipAddressIP AddressstringThis is the IPv4 address configured on the NIC, which may be displayed together with multiple addresses separated by commas (,), and if there is no IP, it may be indicated as none.HP-UX, AIX, Solaris, Linux, Windows
macAddressMAC AddressstringThe MAC address assigned to the NIC, and the logical/aggregate interface may display the representative MAC reported by the operating system, and if it cannot be verified, it may be displayed as none.HP-UX, AIX, Solaris, Linux, Windows
linkLinkstringThe link connection status determined by the operating system or driver is indicated, where 'yes' means connected and 'no' means disconnected or inactive, which may not fully align with the actual cable status.HP-UX, AIX, Solaris, Linux, Windows
stateStatestringThe current interface status of the NIC is indicated, where UP means available, DOWN means inactive or unavailable, and Unknown/Unknown! means status cannot be determined.HP-UX, AIX, Solaris, Linux, Windows
duplexDuplexstringThe Duplex mode of the NIC is indicated, where Full means full duplex, Half means half duplex, and unknown refers to a state that cannot be determined due to the characteristics of the operating system, driver, or device.HP-UX, AIX, Solaris, Linux, Windows
speedSpeedstringThe current link speed of the NIC or the operational speed reported by the operating system, which refers to the speed currently negotiated or recognized rather than the maximum supported speed, and some aggregated interfaces may be displayed as a sum.HP-UX, AIX, Solaris, Linux, Windows
modeModestringAdditional information regarding the NIC aggregation, redundancy, or teaming modes is indicated, for example, EtherChannel:<mode> in AIX, APA in HP-UX, Teaming Mode/Load Balancing in Windows, or AGGR or IPMP in Solaris.HP-UX, AIX, Solaris, Linux, Windows
windowsDriverWindows DriverstringThis is the driver or adapter description information used by the NIC in the Windows environment.Windows
Redundancy configuration display

The nicName and mode fields are displayed as follows depending on the redundancy configuration.

ItemDescription
Linux bondingMember interfaces belonging to a bond are displayed as bond0_eth1, in the bondname_membername form, and mode shows the mode of that bond
Windows team interfacemode is displayed in the LBFO/<team mode>/<load balancing method> form (for example, LBFO/SwitchIndependent/TransportPorts)
Windows team member NICmode is displayed in the LBFO_MEMBER/<configured role>/<operational status> form (for example, LBFO_MEMBER/Active/Active)

A Windows team member NIC is a hidden adapter without its own IP address, so it did not appear in the list before; now member NICs are collected as well. On Linux, the issue where the control entry of the bonding driver (bonding_masters) was incorrectly displayed as a network interface has been fixed.

infra_process_group

A category that targets all processes collected from the server without exclusion and, by default, groups processes run with the same cmd by the same User even if their run arguments differ, providing resource usage by group. It sums and displays usage such as CPU, memory, I/O, and page fault per group, so users can grasp at a glance how a given program family affects server resources. In addition, through the user-defined process group management feature, you can directly define groups by specific server/owner/argsPattern conditions and manage them with a user-specified process group name.

  • Category: ProcessGroup
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
cmdCommandstringThis is the execution command (cmd) of the processes belonging to the Process group.HP-UX, AIX, Solaris, Linux, Windows
nameNamestringBy default, it displays cmd, but if it is a Process Group defined by the user through the process group management menu, it displays the name of the defined Process Group.HP-UX, AIX, Solaris, Linux, Windows
ownerOwnerstringThe execution account (user) of the processes belonging to the Process group. Since they are grouped by the same user, the processes within the group have the same owner.HP-UX, AIX, Solaris, Linux, Windows
pidListProcess ID ListstringOnly the PIDs collected from the Process TopN are provided in the list. Therefore, the number of displayed PIDs may differ from the Process count.HP-UX, AIX, Solaris, Linux, Windows
customCustomstringIndicates whether the user-defined process group is True/False.HP-UX, AIX, Solaris, Linux, Windows
cpuCPU%The total CPU utilization used by the processes belonging to the Process Group. This value includes both user mode and system mode utilization, and observing sustained increases over time, prolonged high utilization, and periodic spikes can be useful for interpreting the operational load characteristics of the group or identifying abnormal CPU occupancy.HP-UX, AIX, Solaris, Linux, Windows
cpuUserCPU User%The CPU utilization of processes belonging to the Process Group in user mode indicates the proportion of CPU used for executing application logic. This metric helps to interpret whether the observed values, including sustained increases over time, repetitive spikes, and correlations with procCnt/threadCnt changes, reflect normal load due to increased throughput or excessive user-space operations.HP-UX, AIX, Solaris, Linux, Windows
cpuSysCPU System%The CPU utilization of processes belonging to the Process Group in kernel/system mode refers to the CPU usage associated with system calls, I/O processing, and kernel resource usage. A consistently high value or an excessively large proportion of system mode compared to user mode can indicate increased I/O processing, a rise in kernel calls, or lock contention.HP-UX, AIX, Solaris, Linux, Windows
rssRSSbyteThe sum of the Resident Set Size (RSS) of the processes belonging to the Process Group. This refers to the size of the memory loaded in actual physical memory. Since this value may include shared memory, in servers with many processes that use shared memory extensively, the sum of the RSS of the processes may exceed the total memory size. Observing not only the current occupancy but also the sustained increase over time and any sharp fluctuation patterns can help determine whether there is a memory leak or abnormal occupancy.HP-UX, AIX, Solaris, Linux
pRssRSS%The total RSS of the process group represents the proportion it occupies in the total physical memory of the system. It is calculated as rss / total physical memory * 100. The total RSS of the process group may include shared memory redundantly, which can make it appear to exceed 100% of the total physical memory of the system. Observing not only the absolute utilization but also the long-term upward trend and peak maintenance can help assess the impact of memory pressure from the process group.HP-UX, AIX, Solaris, Linux
vszVSZbyteThe sum of the total virtual memory address space size acquired by the processes belonging to the Process Group. This refers to the size of the allocated or reserved address space, not the actual physical memory usage, and may include file mappings, shared libraries, and swap reservations. Therefore, it is important to observe the long-term trend of the current value itself, along with the differences from other actual memory metrics (RSS/PSS/USS).HP-UX, AIX, Solaris, Linux, Windows
pssPSSbyteThe sum of the PSS (Proportional Set Size) of processes belonging to a Process Group in Linux. This reflects the physical memory usage based on the proportion divided by the number of processes sharing shared memory, making it useful for understanding the actual memory burden closer to the occupancy level than RSS. Observing the usage trends of shared memory across the entire Process Group along with the increase over time helps interpret changes in memory usage structure. In Linux, collection and aggregation are performed only when the /proc/[pid]/smaps_rollup file is available for each PID, considering the collection cost. From Agent 2.9.20 onward, calculating PSS for a process whose RSS exceeds a certain size (4 GiB by default) can put load on the target process, so the agent skips PSS collection by default and the value may be empty. Therefore, when multiple processes are aggregated into a single group, processes with no collected PSS are excluded from the group total, so the group's total PSS may appear smaller than the actual memory usage. You can change the threshold in the settings.Linux: smaps_rollup supported environment
ussUSSbyteThe sum of the Unique Set Size (USS) of processes belonging to a Process Group in Linux. This represents the total amount of non-shared physical memory used exclusively by each process, which is useful for understanding the scale of memory that is likely to be returned upon the termination of the Process Group. Observing the trend of pure occupied memory, excluding shared memory, helps in assessing the potential for memory leaks within a specific service group. In Linux, data is collected and aggregated only when the /proc/[pid]/smaps_rollup file is available for each PID, considering the cost of collection. From Agent 2.9.20 onward, calculating USS for a process whose RSS exceeds a certain size (4 GiB by default) can put load on the target process, so the agent skips USS collection by default and the value may be empty. Therefore, when multiple processes are aggregated into a single group, processes with no collected USS are excluded from the group total, so the group's total USS may appear smaller than the actual memory usage. You can change the threshold in the settings.Linux: smaps_rollup supported environment
minFltPage Minor Faultcounts/sThe sum of the minor page faults per second for the processes belonging to the Process Group. This refers to faults that occurred but were resolved internally in memory without disk I/O. While the value itself is significant, a gradual increase in conjunction with RSS growth or workload increase may indicate normal memory access expansion, whereas a sudden spike may suggest a change in access patterns or increased memory reclamation.Linux
majFltPage Major Faultcounts/sThe sum of the major page faults per second for the processes belonging to the Process Group. This refers to faults that required reading the necessary pages from disk or swap space. Typically, this includes reads from file-backed pages or the swap area. A consistently high or increasing value serves as an important indicator for interpreting memory pressure, increased page-ins, and potential response latency.Linux
workingSetWorking SetbyteThis is a Windows-specific metric that represents the sum of the Working Sets of processes belonging to the Process Group. It indicates the size of the memory currently resident in physical memory. While instantaneous values are important, observing long-term trends, recovery after sudden spikes, can help identify memory pressure or abnormal occupancy patterns.Windows
privatePrivatebyteThis is a Windows-specific metric that represents the total private memory exclusively used by processes belonging to the Process Group. It indicates the usage of non-shared private memory and serves as a metric to observe the memory growth unique to the process. A pattern of continuous increase over time that is not reclaimed can be useful for interpreting signs of memory leaks.Windows
poolPagedPool PagedbyteThis is a Windows-specific metric that represents the total paged pool memory usage associated with processes belonging to the Process Group. It refers to the kernel memory area that can be paged to disk if necessary. Observing not only the current value but also the long-term trend of increase and recovery can help identify increases in kernel resource usage or abnormal occupancy patterns.Windows
totalCharIoProcess IO Totalbytes/sThe total character-based I/O (char I/O) throughput generated per second by the processes belonging to the Process Group. This may include not only disk but also pipes, terminals, caches, and some buffer-based I/O, so there may be differences from actual physical disk I/O. While the current throughput is important, observing the sustained increase over time, spikes, and repetitive work patterns can help interpret the overall I/O activity characteristics of the group.HP-UX, AIX, Solaris, Linux, Windows
rcharIoRead Character IObytes/sThe total throughput of read character I/O operations generated per second by the processes belonging to the Process Group. This includes not only disk reads but also read requests through buffers, caches, terminals, etc. Therefore, it is important to observe not only instantaneous values but also repetitive patterns associated with the workload cycle, peak moments, and their relationship with CPU and Disk I/O metrics.Linux, Windows
wcharIoWrite Character IObytes/sThe total write character I/O throughput generated per second by the processes belonging to the Process Group. This includes write requests not only to the disk but also through buffers, caches, terminals, etc. A sustained increase or recurring spike pattern in values can help interpret the correlation with log output increases, buffer flushes, and data processing tasks.Linux, Windows
rdiskIoRead Disk IObytes/sThis is a Linux-specific metric that represents the total throughput of read I/O operations actually performed by processes belonging to a Process Group per second from the disk. Since reads processed from the filesystem cache are excluded, this value is closer to the actual storage device reads than rcharIo. Therefore, in environments with a high cache hit rate, rcharIo may be high while rdiskIo may be low, making it appropriate to interpret this metric as an indicator of the actual disk-dependent read load.Linux
wdiskIoWrite Disk IObytes/sThis is a Linux-specific metric that represents the total write I/O throughput to the disk generated per second by processes belonging to a Process Group. This value may differ from the total amount of write requests generated by the processes (wcharIo) and can be reflected with a time lag due to the effects of page cache/buffer cache and writeback operations. Therefore, it is appropriate to interpret this metric as an indicator of the write load actually recorded on the storage.Linux
threadCntThreadcountThe current total number of threads belonging to the Process group. This is the sum of the thread counts of all processes within the group.HP-UX, AIX, Solaris, Linux, Windows
procCntProcesscountThis refers to the total number of processes belonging to the current Process group. It can be interpreted as the number of processes executed by the same user with the same command.HP-UX, AIX, Solaris, Linux, Windows
fdCntFile DescriptorcountA Linux-specific metric that represents the total number of file descriptors currently opened by processes belonging to a Process Group. This includes open file descriptors for files, sockets, pipes, devices, etc.Linux
handleCntHandlecountA Windows-specific metric that represents the total number of handles currently held by the processes belonging to a Process group. It includes references to various kernel objects such as files, processes, threads, events, and registry keys.Windows

infra_process_topn

A category that selects and stores only the individual processes with high resource usage on the server at collection time. By default, it preferentially collects the top 30 processes with individual CPU usage of 0.1% or higher, and separately from this condition, additionally collects up to the top 10 processes with RSS of 20 MB or more even if CPU usage is 0, storing up to about 40 processes. Through this category, users can check, from a detailed analysis perspective, which PIDs are actually using a lot of CPU or memory. In other words, while Process Group is for viewing the broad flow of resource usage, Process TopN is for concretely tracing the individual processes that are the cause.

  • Category: ProcessTopN
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
cmdCommandstringThis refers to the value extracted from the execution command of the process. It is parsed based on cmdline or executable file name.HP-UX, AIX, Solaris, Linux, Windows
ownerOwnerstringThis value represents the user account that executed the process.HP-UX, AIX, Solaris, Linux, Windows
argsArgumentsstringThis value represents the execution arguments (command line parameters) of the process.HP-UX, AIX, Solaris, Linux, Windows
stimeStartTimestringThis value represents the start time of the process in epoch time.HP-UX, AIX, Solaris, Linux, Windows
pidPIDstringThis value represents the Process ID (PID) of the corresponding process.HP-UX, AIX, Solaris, Linux, Windows
ppidPPIDstringThis value represents the PPID (Parent Process ID) of the process, which is the PID of the parent process that created this process.HP-UX, AIX, Solaris, Linux, Windows
procInfoProcess InformationstringThis is an informative field used for process identification and display. It consists of a combination of owner, pid, ppid, stime, cmd(args), etc.HP-UX, AIX, Solaris, Linux, Windows
cpuCPU%This refers to the total CPU utilization used by the process. It includes the values for user mode and system mode utilization, and in a multi-core environment, it is adjusted based on the total number of logical cores, displayed within a maximum range of 100%. In other words, it can be interpreted as the usage ratio of this process compared to the overall CPU resources of the system. While instantaneous utilization is important, observing whether there is a sustained high utilization or periodic spikes can be useful for interpreting the CPU resource occupancy characteristics.HP-UX, AIX, Solaris, Linux, Windows
cpuUserCPU User%This indicates the CPU utilization used by the process in user mode. It serves as a metric to observe the proportion of CPU used for application logic execution, and analyzing sustained increases or repetitive spikes over time can help identify the characteristics of user-space operational load.HP-UX, AIX, Solaris, Linux, Windows
cpuSysCPU System%This refers to the CPU utilization used by the process in kernel/system mode. It is utilized to interpret the CPU share associated with system calls, I/O processing, and kernel resource usage. A pattern where the value is consistently high or the system share is excessively large compared to user share can help suspect influences such as kernel calls, I/O, and lock contention.HP-UX, AIX, Solaris, Linux, Windows
rssRSSbyteThe RSS (Resident Set Size) value of the process indicates the size of memory currently loaded in physical memory. This value includes shared memory, so in servers with processes that utilize a lot of shared memory, the sum of the RSS of the processes may exceed the total memory size. It is a representative metric showing the current memory occupancy, and if it continuously increases over time, it is useful for determining the possibility of memory leaks or abnormal occupancy.HP-UX, AIX, Solaris, Linux
pRssRSS%The ratio of the RSS of the process to the total physical memory of the system is indicated. Rather than focusing solely on the absolute value, observing the long-term upward trend and whether it maintains a peak can help interpret how much a specific process affects system memory pressure.HP-UX, AIX, Solaris, Linux
vszVSZbyteThis refers to the total size of the virtual memory address space secured by the process. It is based on the size of the mapped or reserved virtual address space, not the actual physical memory usage. Therefore, it is important to look at the differences between this value and RSS/PSS/USS, as well as the long-term trends. There may be cases where VSZ is large, but the actual occupied memory is not significant.HP-UX, AIX, Solaris, Linux, Windows
pssPSSbyteThis is the PSS (Proportional Set Size) value of the process. It represents the size of physical memory based on the proportion divided by the number of processes sharing shared memory, making it useful for understanding memory usage that is closer to the actual occupancy level than RSS. It is particularly useful when comparing memory burdens per process in environments with heavy use of shared libraries or shared memory. In Linux, it is collected only in environments where the /proc/[pid]/smaps_rollup file is available, considering the collection cost. From Agent 2.9.20 onward, calculating PSS for a process whose RSS exceeds a certain size (4 GiB by default) can put load on the target process, so the agent skips PSS collection by default and the value may be empty. You can change the threshold in the settings.Linux: smaps_rollup supported environment
ussUSSbyteThis is the USS (Unique Set Size) value of the process. It indicates the size of the non-shared physical memory that this process uses exclusively, and is useful for understanding the scale of memory that is likely to be returned upon process termination. It serves as an indicator for viewing the pure occupied memory, excluding shared memory, and a pattern of continuous increase and non-recovery over time is particularly useful for assessing the potential memory leak of the process itself. In Linux, it is collected only in environments where the /proc/[pid]/smaps_rollup file is available, considering the collection cost. From Agent 2.9.20 onward, calculating USS for a process whose RSS exceeds a certain size (4 GiB by default) can put load on the target process, so the agent skips USS collection by default and the value may be empty. You can change the threshold in the settings.Linux: smaps_rollup supported environment
minFltPage Minor Faultcounts/sThe number of minor page faults per second for the process. This refers to faults that occurred but were resolved internally in memory without disk I/O. While the value itself is significant, a gradual increase in conjunction with RSS growth or workload increase may indicate normal memory access expansion, whereas a sudden spike may suggest a change in access patterns or increased memory recycling.Linux
majFltPage Major Faultcounts/sThis refers to the number of major page faults per second for the process. It indicates faults where the required pages had to be read from disk/swap space. Typically, this may include reads from file-backed pages or swap space. If the value remains consistently high or shows an increasing trend, it becomes an important indicator for interpreting memory pressure, increased page-ins, and potential response delays.Linux
workingSetWorking SetbyteThis is a Windows-specific metric that indicates the size of memory currently being used by the process in physical memory. It serves as a representative metric for observing the actual memory usage of a process. While the current value is important, analyzing long-term trends, recovery after sudden spikes, and other patterns can help identify memory pressure or abnormal occupancy patterns.Windows
privatePrivatebyteA Windows-specific metric that indicates the size of private memory exclusively used by the process. It represents the amount of dedicated memory that is not shared with other processes, serving as an indicator for observing the memory increase unique to the process. A pattern of continuous increase over time that is not reclaimed can be useful for interpreting signs of memory leaks.Windows
poolPagedPool PagedbyteThis is a Windows-specific metric that indicates the paged pool memory usage associated with the process. It relates to the usage of kernel memory areas that can be paged to disk if necessary. Observing not only the current value but also the long-term trend of increase and recovery can help identify increases in kernel resources or abnormal memory usage patterns.Windows
totalCharIoProcess IO Totalbytes/sThis refers to the total character-based I/O (char I/O) throughput generated by the process per second. It may include I/O through not only disks but also pipes, terminals, buffers, caches, and file system layers, so there may be differences from actual physical disk I/O. Observing both the current throughput and the spikes over time, as well as sustained high usage patterns, can help interpret the I/O activity characteristics of the process.HP-UX, AIX, Solaris, Linux, Windows
rcharIoRead Character IObytes/sThis refers to the read character I/O throughput generated by the process per second. It may include read requests not only from disk reads but also from buffers, caches, terminals, etc. Therefore, it is important to consider not only instantaneous values but also the repetitive patterns associated with the workload cycle and the points of spikes.Linux, Windows
wcharIoWrite Character IObytes/sThis refers to the write character I/O throughput generated by the process per second. It may include write requests through disk writes, as well as buffers, caches, and terminals. A sustained increase in values or repetitive spike patterns can help interpret the correlation with log outputs, buffer flushes, and data processing tasks.Linux, Windows
rdiskIoRead Disk IObytes/sThis is a Linux-specific metric that represents the total throughput of read I/O operations actually performed on the disk by the process per second. Since reads processed from the filesystem cache are excluded, this value is closer to the actual storage device reads than rcharIo. Therefore, in environments with a high cache hit rate, rcharIo may be high while rdiskIo may be low, making it appropriate to interpret this metric as an indicator of the actual disk-dependent read load.Linux
wdiskIoWrite Disk IObytes/sThis is a Linux-specific metric that represents the total write I/O throughput actually generated by the process to the disk per second. This value may differ from the total amount of write requests generated by the process (wcharIo) and can be reflected with a time lag due to the effects of page cache/buffer cache and writeback operations. Therefore, it is appropriate to interpret this as a metric for observing the write load recorded on the actual storage.Linux
threadCntThreadcountThe number of threads currently held by the process. It is useful for understanding the concurrency level of multi-threaded applications.HP-UX, AIX, Solaris, Linux, Windows
fdCntFile DescriptorcountA Linux-specific metric that indicates the number of file descriptors currently opened by the process. This includes open file descriptors for files, sockets, pipes, and devices.Linux
handleCntHandlecountA Windows-specific metric that indicates the number of handles currently held by the process. It includes references to various kernel objects such as files, processes, threads, events, and registry keys.Windows

Kubernetes metadata fields

On Linux servers where a container runtime is running, the Pod information the process belongs to is collected as well. After finding a process that uses a lot of resources on the server, you can immediately check which container of which Pod that process is.

FieldTitleUnitDescriptionOS
namespaceNamespacestringKubernetes namespace of the container the process belongs to. Empty for a non-container processLinux
podNamePod NamestringName of the Pod the process belongs toLinux
containerNameContainer NamestringName of the container the process belongs toLinux

The three fields are collected by default. If you set process.topn.k8s.enabled=false, collection stops and the three fields are not sent. On servers without a container runtime, the values are empty.

The supported agent version is Linux 2.9.23 or later.

infra_win_service

A category that collects state information per Windows service. It provides service name, PID, state, status, start mode, exit code, etc. Use it to check the run state, startup configuration, and abnormal termination of Windows services.

  • Category: Windows Service
  • Default collection interval: 60 sec
  • Minimum supported agent: Windows 2.9.19+
FieldTitleUnitDescriptionOS
svcNameService NamestringThis value represents the unique service name of the Windows service. It corresponds to the internal identifier in the Service Management tool (services.msc) or the system service name, and may differ from the Display Name.Windows
pidPIDstringThis value represents the PID (Process ID) of the process currently running the service. If the service is not running, it may be displayed as empty or 0. In cases where multiple services operate within a single process, such as svchost.exe, they may share the same PID.Windows
stateStatestringThis value represents the current service status according to the Service Control Manager (SCM). For example, states such as Running, Stopped, Start Pending, Stop Pending, Paused, Pause Pending, and Continue Pending may be displayed.Windows
startModeStart ModestringThis value indicates how this service is configured to start during system boot or service initiation. It can display as Automatic, Manual, or Disabled, and may be differentiated as Automatic (Delayed Start) depending on the environment.Windows
exitCodeExit CodestringThis value represents the exit code returned during the last termination or start/stop operation of the service. Generally, 0 is interpreted as a normal termination or no error, while a non-zero value can be referenced when tracing causes such as service start failure, execution errors, or abnormal termination.Windows

infra_nic_perf

A category that collects performance indicators per individual NIC. It provides Tx/Rx bandwidth, packet count, error, drop, collision, etc. per interface. Use it to check a specific NIC's bottleneck, packet loss, and concentration of interrupts/errors.

  • Category: by NIC
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
nameNIC NamestringThis value represents the name of the network interface. For example, it displays eth0, ens192, bond0 in Linux, and adapter names in Windows.HP-UX, AIX, Solaris, Linux, Windows
totalbpsNIC TotalbpsThis is the total amount of data transmitted (Tx) and received (Rx) through the specified NIC, expressed in bps (bits per second).HP-UX, AIX, Solaris, Linux, Windows
rxbpsNIC RXbpsThis value represents the amount of data received (Rx) per second through the specified NIC, measured in bps (bits per second).HP-UX, AIX, Solaris, Linux, Windows
txbpsNIC TxbpsThe value representing the amount of data transmitted per second (Tx) through the specified NIC, measured in bps (bits per second).HP-UX, AIX, Solaris, Linux, Windows
totalppsNIC Total PPScounts/sThe total number of packets received (Rx) and transmitted (Tx) through the specified NIC per second is represented in pps (packets per second) or count/s. This corresponds to rxpps + txpps.HP-UX, AIX, Solaris, Linux, Windows
rxppsNIC Rx PPScounts/sThe number of packets received per second (Rx) through the specified NIC is represented in pps (packets per second) or count/s.HP-UX, AIX, Solaris, Linux, Windows
txppsNIC Tx PPScounts/sThe number of packets transmitted (Tx) per second through the specified NIC is represented in pps (packets per second) or count/s.HP-UX, AIX, Solaris, Linux, Windows
collisionNIC Collisioncounts/sThis is the value converted to an average rate per second of the number of packet collisions detected during the transmission (Tx) process of the corresponding NIC. It is primarily significant in half-duplex Ethernet environments, while in modern full-duplex environments, it is generally normal for this value to be close to 0.HP-UX, AIX, Solaris, Linux
errInNIC Error Incounts/sThis is the value converted to the average rate of change per second of the number of packets received (Rx) through the specified NIC that encountered errors. It may include received errors such as CRC errors, frame errors, and overruns, and the scope of inclusion may vary depending on the OS/driver.HP-UX, AIX, Solaris, Linux, Windows
errOutNIC Error Outcounts/sThis is the value converted to the average rate of packets with errors that were transmitted (Tx) through the corresponding NIC per second. It may include transmission errors such as carrier errors, aborts, and transmit queue issues, and the scope of inclusion may vary depending on the OS/driver.HP-UX, AIX, Solaris, Linux, Windows
dropInNIC Drop Incounts/sThis is the average rate of packets dropped that were received (Rx) through the specified NIC and not processed at the kernel or NIC layer. It can increase due to buffer shortages, queue overflow, driver processing delays, etc.HP-UX, AIX, Solaris, Linux, Windows
dropOutNIC Drop Outcounts/sThis is the average rate of packets dropped before actual transmission among the packets transmitted (Tx) through the specified NIC, expressed as a value per second. It can increase due to factors such as transmission queue overflow, driver issues, or interface status anomalies.HP-UX, AIX, Solaris, Linux, Windows
linkSpeedNIC Link SpeedbpsThis is the value representing the link speed of the NIC in Mbps. It refers to the transmission bandwidth capacity of the currently negotiated or configured interface, not the actual usage. However, if the link speed cannot be determined due to the OS, driver, or type of virtual interface, it may be collected as unknown or null, and in such cases, the value may not be displayed.HP-UX, AIX, Solaris, Linux, Windows
linkUsageNIC Utilization%This value represents the total transmission and reception throughput of the NIC (NIC Total) expressed as a percentage of the link speed (NIC Link Speed). It indicates the actual traffic usage level compared to the available link bandwidth of the NIC, with higher values signifying that the link is being utilized more heavily. However, if the link speed (NIC Link Speed) is unknown or null, the utilization cannot be calculated, and thus no value may be displayed.HP-UX, AIX, Solaris, Linux, Windows

infra_infiniband

A category that collects state and performance indicators per InfiniBand or HCA port. It provides port state, link layer, speed, LID, Tx/Rx throughput, packet count, symbol error, link recovery, constraint error, etc. Use it to check IB fabric connection state, link quality, and per-port bottlenecks/errors.

  • Category: by Infiniband
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
interfaceNameInterface NamestringThis is an identifier that represents the HCA device name along with the port number. For example, mlx5_0:1 refers to port 1 of the mlx5_0 device.Linux
linkLayerLink LayerstringIndicates the link layer type of the port. It displays either InfiniBand or Ethernet and is used to check which link mode the port is operating in.Linux
stateStatestringIndicates the logical state of the port. For example, Down, Init, Active, etc. can be displayed, which serves as a criterion for determining whether the port is participating normally in the IB fabric.Linux
physStatePhysical StatestringIndicates the physical link status of the port. For example, Polling, LinkUp, Disabled, etc. are displayed and are used to determine the cable connection and physical layer negotiation status.Linux
rateLink SpeedstringIndicates the current link speed of the port. For example, values such as 100Gb/s and 200Gb/s are displayed, which are used to verify the results of link negotiation or the maximum operational speed of the port.Linux
lidLocal IdentifierstringThis represents the LID (Local Identifier) assigned to the corresponding port in the InfiniBand environment. It serves as the logical address of the port within the IB fabric, and may be meaningless or empty in Ethernet mode.Linux
portXmitDataTransmit Databytes/sThis value represents the data throughput transmitted (Tx) per second on the specified port, measured in Bytes/s. It indicates the transmission usage based on the port.Linux
portRcvDataReceive Databytes/sThis value represents the throughput of data received (Rx) per second on the specified port, measured in Bytes/s. It indicates the port-based receive usage.Linux
portXmitPacketsTransmit Packetscounts/sThis value represents the total number of packets transmitted per second on the specified port. It is a packet rate metric independent of packet size.Linux
portRcvPacketsReceive Packetscounts/sThis value represents the total number of packets received per second on the specified port. It is a packet rate metric that is independent of packet size.Linux
unicastRcvPacketsUnicast Receive Packetscounts/sThis value indicates the number of unicast packets received per second on the specified port. It is useful for monitoring the general traffic flow with a single destination.Linux
unicastXmitPacketsUnicast Transmit Packetscounts/sThe value representing the number of unicast packets transmitted per second on the specified port.Linux
multicastRcvPacketsMulticast Receive Packetscounts/sThis value represents the number of multicast packets received per second on the specified port. It can be referenced when analyzing the impact of multicast-based communication or IB fabric control traffic.Linux
multicastXmitPacketsMulticast Transmit Packetscounts/sThe value representing the number of multicast packets transmitted per second on the specified port.Linux
symbolErrorSymbol Errorscounts/sThis value represents the rate of symbol errors occurring per second at the physical link layer. Generally, it may indicate bit errors, cable quality issues, or instability in optical modules/ports.Linux
portRcvErrorsPort Receive Errorscounts/sThis value represents the per-second change rate of the total error counter recorded during the reception process for the port. It may include received frame anomalies, integrity issues, protocol errors, etc.Linux
portRcvRemotePhysicalErrorsRemote Physical Errorscounts/sThis value indicates the rate of change per second of the error counter received by this port due to a physical layer issue occurring at the opposite port. It may suggest a problem with the remote port or the cable/module status on the other side of the link.Linux
portRcvSwitchRelayErrorsSwitch Relay Errorscounts/sThis value represents the rate of change per second of the error counter recorded when the switch fails to properly relay packets. It may indicate issues with the switch's forwarding path or anomalies in the fabric.Linux
linkErrorRecoveryLink Error Recoverycounts/sThis value represents the rate of change per second of link error recovery events. It may indicate that link reinitialization or error recovery is occurring frequently. If the value continues to increase, link instability may be suspected.Linux
portXmitConstraintErrorsTransmit Constraint Errorscounts/sThis value represents the per-second change rate of errors recorded due to violations of transmission-side constraints. For example, inconsistencies in fabric policies such as SL, VL, QoS, and partition key can be the cause.Linux
portRcvConstraintErrorsReceive Constraint Errorscounts/sThis value represents the rate of change per second of errors recorded due to violations of receiving-side constraints. It may increase in cases of mismatches in fabric policies, partitions, and QoS between the transmitting and receiving sides.Linux
localLinkIntegrityErrorsLocal Link Integrity Errorscounts/sThis value represents the per-second change rate of link integrity errors based on the local port. It may be associated with CRC/bit errors, signal quality degradation, or issues with cable or port hardware.Linux
excessiveBufferOverrunErrorsBuffer Overrun Errorscounts/sThis value indicates the rate of errors that could not be processed normally due to a receive buffer overrun, which may increase when burst traffic is high relative to port processing capacity or when there are delays in system/driver/firmware processing.Linux
linkDownedLink Down Eventscounts/sThis value represents the rate of change per second for events where the link transitions to a Down state. It may increase in situations such as cable disconnection, disabling the relative port, link instability, and equipment rebooting.Linux
portXmitDiscardsTransmit Discardscounts/sThis value represents the rate of change per second of packets discarded before actual transmission in the transmission queue. Causes may include buffer shortages, congestion, queue constraints, and port status anomalies.Linux
vl15DroppedVL15 Dropped Packetscounts/sThis value indicates the rate of change per second of the number of discarded packets in the VL15 (Virtual Lane dedicated to Subnet Management) queue. It may suggest congestion in Subnet Manager-related control traffic or anomalies in the fabric management path.Linux
FieldTitleUnitDescriptionOS
------:---:------
rdmaNetDeviceRDMA Net DevicestringName of the operating system network interface connected to that RDMA device and port. Multiple names are separated by | (for example, ib0, enp175s0f0). Empty when there is no mapping informationLinux

You can view the RDMA port together with the operating system network interface, so use it when checking the error metrics of an InfiniBand port and the traffic metrics of the corresponding interface at the same time. When port-level mapping is not provided, such as for a pure InfiniBand port, device-level mapping is displayed instead.

The supported agent version is Linux 2.9.23 or later.

infra_disk_perf

A category that collects performance indicators per individual physical disk. It provides detailed I/O performance such as per-disk throughput, IOPS, busy, service time, and wait time. Use it to check a specific disk's bottleneck, latency, and queue growth.

  • Category: by Disk
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
nameDisk NamestringValue displaying the name of the physical disk.HP-UX, AIX, Solaris, Linux, Windows
busyDisk Busy%This value represents the ratio of time the physical disk was busy during the observation period.HP-UX, AIX, Solaris, Linux, Windows
totalDisk IO Totalbytes/sThis value represents the total throughput of the physical disk per second, which is the sum of the read and write byte throughput.HP-UX, AIX, Solaris, Linux, Windows
readDisk Readbytes/sThis value indicates the read throughput of the physical disk in bytes per second. It is interpreted based on Bytes/sec.HP-UX, AIX, Solaris, Linux, Windows
writeDisk Writebytes/sThis value represents the write throughput of the physical disk in bytes per second. It is interpreted based on Bytes/sec.HP-UX, AIX, Solaris, Linux, Windows
totalIopsDisk Total IOPScounts/sThis value represents the total I/O operations per second for the physical disk. It is the sum of the read IOPS and write IOPS.HP-UX, AIX, Solaris, Linux, Windows
readIopsDisk Read IOPScounts/sThis value represents the number of read I/O operations per second for the physical disk.HP-UX, AIX, Solaris, Linux, Windows
writeIopsDisk Write IOPScounts/sThis value represents the number of write I/O operations per second for the physical disk.HP-UX, AIX, Solaris, Linux, Windows
readSvcTimeDisk Read Service Timems/ioThe value displayed in milliseconds represents the time taken by the physical disk to process a single Read I/O request.AIX, Solaris, Linux, Windows
writeSvcTimeDisk Write Service Timems/ioThe value displayed in milliseconds represents the time taken by the physical disk to process a single Write IO request.AIX, Solaris, Linux, Windows
avgWaitTimeDisk Average Wait TimemsThis is a metric exclusive to HP-UX that indicates the average time, in milliseconds, that IO processing requests have spent in an idle state in the disk queue.HP-UX
avgSvcTimeDisk Average Service TimemsThis is a value that indicates the average time in milliseconds taken to process each transfer request to the disk (including seek time, rotational latency, and data transfer time) for HP-UX.HP-UX
qLengthDisk Q-LengthcountThis metric indicates the queue length of I/O requests that are either pending or being processed for the specified disk. A low value suggests that requests are being handled smoothly, while a consistently high value may indicate that the disk is unable to process requests in a timely manner. However, the absolute benchmark may vary depending on the type of disk (HDD/SSD/NVMe), system architecture, and workload characteristics, so it should be interpreted comprehensively along with busy (%), wait time, and service time.Linux, Windows

infra_filesystem

A category that collects capacity and performance information per filesystem or mount point. It provides total capacity, usage, free space, usage rate, inode usage rate, read/write throughput, IOPS, queue length, etc. Use it to check a specific filesystem's space shortage, inode exhaustion, and I/O load.

  • Category: by Filesystem
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
mountNameMount NamestringThis value represents the path or identifier name where the file system is mounted. For example, in Linux/Unix, mount points such as /, /var, and /data may be displayed, while in Windows, drive letters or volume paths may be shown.HP-UX, AIX, Solaris, Linux, Windows
fsTypeFileSystem TypestringThis value indicates the file system type. For example, ext4, xfs, zfs, vxfs, ntfs, etc.HP-UX, AIX, Solaris, Linux, Windows
devNameDevice NamestringThis value represents the name of the device or the device identification path to which the file system is actually connected. In Linux, it is displayed based on the KNAME column of the lsblk command, while in Unix-like systems, it is based on the FILESYSTEM column of the df or bdf command results. In Windows, it is displayed based on the device identification path to which the file system is connected, rather than the drive letter.HP-UX, AIX, Solaris, Linux, Windows
pDevNameParent Device NamestringThis value indicates the name of the parent device to which the device belongs. In Linux, it is displayed based on the PKNAME value of lsblk, and can be used to identify which higher disk hierarchy a partition or logical device is connected to. For example, it may be displayed as nvme0n1 for nvme0n1p1, and as sda for sda1. In Unix-like systems and Windows, it is difficult to provide information based on the same criteria as Linux's PKNAME in a consistent manner, so it may be empty or excluded from the collection target.Linux
totalSizeFS Total SizeGiBThe total capacity of the file system expressed in bytes.HP-UX, AIX, Solaris, Linux, Windows
usedFS UsedGiBThe value representing the currently used capacity in bytes for the file system.HP-UX, AIX, Solaris, Linux, Windows
freeFS FreeGiBThe value representing the currently available remaining capacity in the file system, in bytes.HP-UX, AIX, Solaris, Linux, Windows
pUsedFS Used%This value represents the ratio of used capacity to the total capacity of the file system.HP-UX, AIX, Solaris, Linux, Windows
pFreeFS Free%This value represents the percentage of available remaining capacity compared to the total capacity of the file system.HP-UX, AIX, Solaris, Linux, Windows
pInodeUsedFS i-node Used%This value represents the ratio of used inodes to the total number of inodes in the file system. When the number of files is very high, it may not be possible to create new files due to inode exhaustion, even if there is available capacity, so it is necessary to check this as well.HP-UX, AIX, Solaris, Linux
readFS Readbytes/sThis value represents the data throughput read per second from the file system, expressed in Byte/s. In other words, it is the read throughput based on the file system.HHP-UX, AIX, Solaris, Linux, Windows
writeFS Writebytes/sThis value represents the data throughput recorded per second on the file system, expressed in Byte/s. In other words, it is the write throughput based on the file system.HP-UX, AIX, Solaris, Linux, Windows
ioTotalFS IO Totalbytes/sThis value represents the total data throughput processed per second in the file system, measured in Byte/s. It is the sum of the read and write data throughput, indicating the total I/O throughput of the file system.HP-UX, AIX, Solaris, Linux, Windows
readIopsFS Read IOPScounts/sThis value represents the number of read I/O operations processed per second on the file system. It is interpreted as the file system's read IOPS.HP-UX, AIX, Solaris, Linux, Windows
writeIopsFS Write IOPScounts/sThis value represents the number of write I/O operations processed per second on the file system. It is interpreted as file system write IOPS.HP-UX, AIX, Solaris, Linux, Windows
IopsTotalFS Total IOPScounts/sThis value represents the total number of I/O operations processed per second on the file system. It is the sum of read IOPS and write IOPS, indicating the total IOPS for the file system.HP-UX, AIX, Solaris, Linux, Windows
iopsTotalFS Total IOPScounts/sThis value represents the total number of I/O operations processed per second on the file system. It is the sum of read IOPS and write IOPS, indicating the total IOPS for the file system.HP-UX, AIX, Solaris, Linux, Windows
qLengthFS Q-LengthcountThis metric indicates the queue length level of I/O processing requests associated with the file system. It can be interpreted as the average cumulative level of requests that are either waiting or being processed. A temporary spike in the value may be normal due to a momentary increase in load; however, if a high state persists, it may suggest potential processing delays in the file system or underlying storage. It is important to note that absolute benchmark values can vary depending on the type of file system, storage architecture, and workload characteristics, so it should be interpreted comprehensively alongside metrics such as read/write throughput, IOPS, and response time series.Linux, Windows

infra_zpool_perf

A category that collects performance/state information per ZFS zpool and vdev. It provides zpool capacity, read/write throughput, IOPS, queue and disk wait family indicators, and wait times related to scrub/trim/rebuild. Use it to check the overall state of the ZFS storage pool and bottleneck sections of a specific vdev or device.

  • Category: by ZFS
  • Default collection interval: 5 sec
  • Minimum supported agent: Linux 2.9.15+, Solaris 1.4.9+
FieldTitleUnitDescriptionOS
nameZpool Display NamestringThis is the display name of the device. It is an identifier value that combines the pool name and the device name, used as a display name to distinguish between zpool or vdev on the screen.Linux, Solaris
poolNameZpool NamestringThis value represents the name of the ZFS zpool to which the device belongs. It is used when interpreting multiple vdevs or leaf devices grouped within the same pool.Linux, Solaris
devNameZpool Device NamestringThis value represents the original name of the actual vdev or leaf device. It is used to identify devices where performance bottlenecks have occurred.Linux, Solaris
devTypeZpool Device TypestringThis value indicates the ZFS device type. For example, it can be represented as zpool, data, special, log, cache, spare, etc., and is used to distinguish the roles of the devices.Linux, Solaris
totalSizeTotal SizebyteThis is the total size of the zpool or device (Allocated + Free) expressed in bytes.Linux, Solaris
allocatedAllocatedbyteThis value represents the currently allocated capacity for the zpool or device in bytes. It is used to understand the scale of space that has been logically assigned.Linux, Solaris
freeFreebyteThis value represents the currently available remaining capacity in bytes for the specified zpool or device. It is used to check for insufficient available space.Linux, Solaris
pctUsedUsed%This value represents the percentage of used capacity relative to the total capacity of the zpool or device. A higher value indicates less available space.Linux, Solaris
readBytesReadbytes/sThis value represents the read data throughput of the specified zpool or device in bytes/s. It indicates the ZFS read throughput metric.Linux, Solaris
writeBytesWritebytes/sThis value represents the write data throughput of the specified zpool or device in bytes/s. It indicates the ZFS standard write throughput metric.Linux, Solaris
totalBytesRead+Writebytes/sThis value represents the total data throughput of the specified zpool or device in bytes/s. It is the total throughput metric obtained by summing the read and write throughput.Linux, Solaris
readOperationRead IOPScounts/sThis value represents the number of read I/O operations processed per second on the specified zpool or device. It refers to the ZFS standard read IOPS.Linux, Solaris
writeOperationWrite IOPScounts/sThis value represents the number of write I/O operations processed per second on the specified zpool or device. It refers to the ZFS standard write IOPS.Linux, Solaris
totalOperationRead+Write IOPScounts/sThis value represents the total number of I/O operations processed per second on the specified zpool or device. It is the total IOPS metric obtained by summing the read IOPS and write IOPS.Linux, Solaris
asyncqReadWait_nsAsyncq Read WaitnsThe average time in nanoseconds that read I/O requests have waited in the ZFS internal asynchronous queue. A higher value indicates that asynchronous read requests are being delayed in the queue.Linux
asyncqWriteWait_nsAsyncq Write WaitnsThe average time in nanoseconds that write I/O requests wait in the ZFS internal asynchronous queue. A higher value indicates that asynchronous write requests are being delayed in the queue.Linux
totalAsyncqWait_nsAsyncq Total WaitnsThe total average time in nanoseconds that read/write I/O requests waited in the ZFS internal asynchronous queue. This is used to comprehensively check the bottleneck level of the async queue.Linux
syncqReadWait_nsSyncq Read WaitnsThe average time in nanoseconds that read I/O requests waited in the ZFS internal synchronous queue. This is used to assess the latency level of synchronous read request processing.Linux
syncqWriteWait_nsSyncq Write WaitnsThe average time in nanoseconds that write I/O requests waited in the ZFS internal synchronous queue. This is used to assess the level of latency in processing synchronous write requests.Linux
totalSyncqWait_nsSyncq Total WaitnsThe total average time in nanoseconds that read/write I/O requests have waited in the ZFS internal synchronous queue. This is used to comprehensively check for sync queue bottlenecks.Linux
diskReadWait_nsDisk Read WaitnsThe average time taken for read I/O requests to be processed at the actual disk layer, expressed in nanoseconds. This is used to understand the actual device processing delay level after queue waiting.Linux
diskWriteWait_nsDisk Write WaitnsThe value represents the average time taken for write I/O requests to be processed at the actual disk layer, measured in nanoseconds. It is used to understand the actual device processing latency level after queue waiting.Linux
totalDiskWait_nsDisk Total WaitnsThe value represents the total average time in nanoseconds for read/write I/O requests to be processed at the actual disk layer. It is used to comprehensively assess the processing latency level of the device itself.Linux
totalReadWait_nsTotal Read WaitnsThe total average latency of read I/O requests expressed in nanoseconds. It includes both the internal queue wait time of ZFS and the actual disk processing time.Linux
totalWriteWait_nsTotal Write WaitnsThe value representing the overall average latency of write I/O requests in nanoseconds. It includes both the internal queue wait time of ZFS and the actual disk processing time.Linux
totalWait_nsTotal WaitnsThis value represents the average latency of all read/write I/O requests in nanoseconds. It is a consolidated wait metric that includes both queue waiting time and actual disk processing time. A higher value indicates a potential for I/O latency for the corresponding zpool or device.Linux
scrub_nsScrubnsThe average time (in ns) that scrub operations wait in the ZFS internal queue. Scrub is the process of sequentially reading blocks within the pool to verify checksums and check for data anomalies.Linux
trim_nsTrimnsThe average time that trim operations wait in the ZFS internal queue is represented in nanoseconds. Trim is the operation that returns unused blocks from SSDs or thin provisioning devices.Linux
rebuild_nsRebuildnsThe average time (in ns) that the rebuild operation waits in the ZFS internal queue. It indicates the level of delay in the process of reconstructing the necessary data during device replacement or recovery.Linux

infra_vxlv_perf

Data collecting I/O performance information per Logical Volume (vxlv) in a Veritas Volume Manager environment. It provides indicators such as read/write IOPS, throughput, and average latency per disk group and volume, and is used to analyze per-volume performance trends, bottlenecks, and read/write skew.

  • Category: by VxLV
  • Default collection interval: 5 sec
  • Minimum supported agent: Linux 2.9.15+
FieldTitleUnitDescriptionOS
nameVxLV Display NamestringThis is the display name. It can be used as an identifier combining the Disk Group name and the VxLvol name.Linux
diskGroupVxLV Disk GroupstringThe name of the Veritas Disk Group to which the volume belongs. It is a management unit that logically groups multiple volumes and is used to identify or aggregate volumes within the same Disk Group.Linux
vxLvolNameVxLV Lvol NamestringThe name of the logical volume managed by Veritas Volume Manager. It is the volume identifier that is the subject of performance metrics measurement.Linux
blockSizeVxLV Block SizestringThis value represents the I/O block size of the volume. It is expressed in bytes and is used to convert block count-based metrics into byte-based throughput. It is based on the output of the command 'vxprint -F %iosize', and in case of a retrieval failure or abnormal value, it falls back to 512 bytes.Linux
readIopsRead IOPScounts/sThe average number of read operations per second is calculated by dividing the number of read operations that occurred during the collection interval by the duration of the interval. This indicates the frequency of read request processing for the volume.Linux
writeIopsWrite IOPScounts/sThe average number of write operations per second is calculated by dividing the number of write operations that occurred during the collection interval by the duration of the interval. This indicates the frequency of write requests to the volume.Linux
IopsTotalTotal IOPScounts/sThe average number of operations per second is calculated by dividing the total number of I/O operations that occurred during the collection interval by the duration of the interval. It is interpreted as the sum of Read IOPS and Write IOPS.Linux
readReadbytes/sThe average read throughput per second is calculated by multiplying the number of read blocks processed during the collection interval by the I/O block size and then dividing by the interval time. The unit is bytes/s.Linux
writeWritebytes/sThe average write throughput per second is calculated by multiplying the number of write blocks processed during the collection interval by the I/O block size and then dividing by the interval duration. The unit is bytes/s.Linux
totalIO Totalbytes/sThe average throughput processed during the collection interval, calculated by dividing the total I/O data amount by the interval time. It refers to the sum of read throughput and write throughput.Linux
readAvgMsRead Average LatencymsAverage latency of read operations during the collection interval. This is the average time taken for each read request to complete, measured in milliseconds (ms).Linux
writeAvgMsWrite Average LatencymsThe average latency of write operations that occurred during the collection interval. It is the average time taken for each write request to complete, measured in milliseconds (ms).Linux

infra_cpu_core

A category that collects the usage state per logical CPU core. It provides user, sys, wait, idle, irq, softirq, steal, core CPU usage, etc. by coreId. Use it to check core skew, IRQ concentration, and single-thread bottlenecks.

  • Category: by CPU Core
  • Default collection interval: 5 sec
FieldTitleUnitDescriptionOS
coreIdCore IDstringThis is the ID value used to identify CPU cores. It refers to the logical core number recognized by the OS and may not correspond 1:1 with physical cores. When Hyper-Threading/SMT is enabled, multiple logical cores can be mapped to a single physical core.HP-UX, AIX, Solaris, Linux, Windows
coreCpuCPU%This refers to the overall CPU utilization of the core. It is calculated as 100 minus idle and interpreted as the usage ratio that includes all non-idle times such as user mode, kernel mode, nice, I/O wait, irq, softirq, and steal. The included items may vary depending on the operating system.HP-UX, AIX, Solaris, Linux, Windows
userUser%It refers to the ratio of the total CPU time of the core that is used to execute general applications/processes in user mode.HP-UX, AIX, Solaris, Linux, Windows
sysSys%It refers to the ratio of the total CPU time of the core that is used for system call processing, kernel tasks, and driver handling in kernel mode (System mode).HP-UX, AIX, Solaris, Linux, Windows
niceNice%It refers to the ratio of the total CPU time of the core that is used for executing low-priority (niced) user processes. The utilization of user processes with normal priority is typically included in user(%).HP-UX
waitWait%The ratio of the total CPU time of the core during which the CPU was idle waiting for I/O completion (I/O wait). This is interpreted as the time spent waiting for responses from external resources such as disk/network, rather than the time the CPU actually performed computations.HP-UX, AIX, Solaris, Linux
totalUser+Sys%It refers to the sum of the ratios of CPU usage in user mode and kernel mode out of the total CPU utilization of the respective core. This is interpreted as the pure CPU execution utilization, excluding I/O wait, irq, softirq, and steal.HP-UX, AIX, Solaris, Linux, Windows
total_waitUser+Sys+Wait%The sum of user, system, and wait time of the total CPU time for the corresponding core. In other words, it is a usage metric that includes user processing, kernel processing, and I/O waiting. There may be differences in the actual total CPU utilization depending on the operating system.HP-UX, AIX, Solaris, Linux
idleIdle%This refers to the ratio of the total CPU time of the core that was idle without performing any tasks. A higher value indicates that the core has more availability.HP-UX, AIX, Solaris, Linux, Windows
irqIrq%This refers to the ratio of total CPU time of the core that is used to handle hardware interrupts (IRQ). It can increase when network cards, disks, and other hardware events are concentrated on a specific core.Linux
softirqSoftirq%This refers to the ratio of total CPU time of the core used for handling Software interrupts (SoftIRQ). It can increase when delayed processing tasks of the kernel, such as network packet processing and block I/O post-processing, are concentrated on a specific core.Linux
stealSteal%In a virtualized environment, the ratio of time that a core is unable to execute due to the hypervisor prioritizing other virtual CPUs is referred to as CPU steal utilization. A high value may indicate insufficient CPU resources allocated to the VM or a contention state.Linux
FieldTitleUnitDescriptionOS
numaNodeIdNUMA Node IDstringNUMA node number the logical core belongs to. -1 for cores whose NUMA connection information cannot be determinedLinux

The CPU Core chart legend of Metrics Explorer uses this field to display the Core cpu1 (Node 0) form. For cores whose value is -1 and in Windows and Unix environments where the field is not collected, the Core cpu1 form is kept.

infra_os_parameter

A category that collects the kernel and system parameter values of the operating system. It provides the main parameters of the network, file system, and kernel areas along with their current values. Use it to compare configuration differences between servers or to check kernel setting values that can cause performance problems. The collected values can be queried on the OS Parameter information screen in Labs.

  • Category: OS Parameter
  • Default collection interval: 1 hour
FieldTitleUnitDescriptionOS
parameterParameterstringParameter name (for example, net.ipv4.tcp_keepalive_intvl)AIX, HP-UX, Solaris, Linux
categoryCategorystringArea the parameter belongs to (for example, net, fs, kernel)AIX, HP-UX, Solaris, Linux
osOSstringType of the operating system the parameter was collected fromAIX, HP-UX, Solaris, Linux
valueValuestringCurrent value of the parameterAIX, HP-UX, Solaris, Linux
customCustomstringWhether the parameter was added by the user with the os.parameter.custom option instead of being a default collection targetAIX, HP-UX, Solaris, Linux

To add parameters other than the default collection targets, enter the parameter names separated by commas (,) in the os.parameter.custom option. Adjust the collection interval with the infra_os_parameter.interval option. The default value is 3600 seconds.

Windows does not collect this category.

References