GPU metrics

The following NVIDIA GPU metrics are collected through DCGM Exporter. For details about how GPU metrics are collected and the sidecar pattern, see the Supported environment > NVIDIA GPU document.

Metric	Type	Unit	Description
DCGM_FI_DEV_GPU_UTIL	gauge	%	Percentage of time the GPU cores were actively performing computation.
DCGM_FI_PROF_GR_ENGINE_ACTIVE	gauge	ratio	Fraction of time the graphics engine (3D/Compute) was active.
DCGM_FI_PROF_SM_ACTIVE	gauge	ratio	Fraction of cycles in which one or more warps were active.
DCGM_FI_PROF_SM_OCCUPANCY	gauge	ratio	SM (Streaming Multiprocessor) occupancy (resident warps per SM).
DCGM_FI_PROF_PIPE_TENSOR_ACTIVE	gauge	ratio	Fraction of cycles in which the tensor core (HMMA) pipe was active.
DCGM_FI_PROF_DRAM_ACTIVE	gauge	ratio	Fraction of cycles in which the memory interface was active (sending or receiving data).
DCGM_FI_DEV_FB_USED	gauge	MiB	Used framebuffer (VRAM) memory.
DCGM_FI_DEV_FB_FREE	gauge	MiB	Available framebuffer (VRAM) memory.
DCGM_FI_DEV_FB_TOTAL	gauge	MiB	Total framebuffer (VRAM) memory.
DCGM_FI_DEV_FB_USED_PERCENT	gauge	%	Percentage of used framebuffer (VRAM) memory.
DCGM_FI_DEV_FB_RESERVED	gauge	MiB	Reserved framebuffer memory.
DCGM_FI_PROF_PCIE_TX_BYTES	counter	byte	Total bytes transmitted (Host → Device) over the PCIe bus.
DCGM_FI_PROF_PCIE_RX_BYTES	counter	byte	Total bytes received (Device → Host) over the PCIe bus.
DCGM_FI_DEV_POWER_USAGE	gauge	W	Current power consumption of the GPU.
DCGM_FI_DEV_GPU_TEMP	gauge	°C	Current temperature of the GPU chip.
DCGM_FI_DEV_SM_CLOCK	gauge	MHz	SM (Streaming Multiprocessor) clock frequency.
DCGM_FI_DEV_MEM_CLOCK	gauge	MHz	Memory clock frequency.
DCGM_FI_DEV_PSTATE	gauge	P-State	Current performance state of the GPU (P0 is the highest).
DCGM_FI_DEV_ECC_SBE_AGG_TOTAL	counter	count	Aggregated count of single-bit persistent ECC errors.
DCGM_FI_DEV_ECC_DBE_AGG_TOTAL	counter	count	Aggregated count of double-bit persistent ECC errors.

The following metric has been added.

Metric	Type	Unit	Description
DCGM_FI_DEV_WEIGHTED_GPU_UTIL	gauge	ratio	GPU utilization weighted by the slice ratio the device occupies.

This metric lets you aggregate GPU utilization of MIG and non-MIG environments on the same basis. Because a MIG profile occupies only part of a physical GPU, simply adding up utilization values does not match the GPU-level total.

The calculation method is distinguished by the calculation_method label.

Value	Description
weighted_sum	MIG instance. DCGM_FI_PROF_GR_ENGINE_ACTIVE multiplied by the slice ratio (compute_slices / DCGM_FI_DEV_MIG_MAX_SLICES) and then summed
direct	Non-MIG GPU. DCGM_FI_DEV_GPU_UTIL divided by 100
prof_gr_engine_active	Non-MIG GPU of the H100, H200, H800, and H20 series. DCGM_FI_PROF_GR_ENGINE_ACTIVE used as is

• The value ranges from 0 to 1.
• Abnormal DCGM_FI_DEV_GPU_UTIL values over 100 are capped at 100 before the calculation.
• The GPU UUID is provided together in the DCGM_FI_DEV_UUID label.