Monitoring Multiple Instances
Home > Select Project > Dashboard > Monitoring Multiple Instances
While you can monitor only one agent (instance) in Monitoring a Database Instance, Monitoring Multiple Instances allows you to monitor and manage multiple agents (instances).
You can compare and analyze key performance metrics occurring in each instance at a glance. You can effectively perform monitoring by aggregating data such as the number of agents running, the sum of all active sessions, and the number of transactions. Users can easily check the changes in metric values over time and high performance metrics for each instance through graph charts.
Additionally, users can see the longest-lasting active sessions, which can be used as a basis for diagnosing and resolving the system performance bottlenecks.
If you select an agent to compare in Dashboard > Instance List, and then select Multi view, the Monitoring Multiple Instances menu appears.
Viewing the past data
The dashboard basically provides real-time monitoring features in which you can view the past data.
Real-time mode | Non-real-time mode |
---|---|
-
Select in the time selector at the upper left of the screen.
-
Click the date and time to set the desired time zone.
-
Select Apply.
It updates the data in widgets placed on the dashboard based on the set time. To switch to the Realtime mode, select .
Checking the agent
Checking the agent connection status
On the upper left of the screen, the right area of the time selector provides information that allows you to check the status of agents connected to the project. This allows you to immediately check whether the target application server is running.
-
Total: Number of all agents connected to the project
-
Active: Number of active agents
-
Inactive: Number of inactive agents
-
: Can display or hide inactive agents.
Agent-based monitoring
By default, the dashboard displays the metrics collected from all agents in charts and you can also view the data by agent. Select one or more agents under the time selector. The data of the widgets on the dashboard is updated with the metrics of the selected agent(s).
To select all agents again while one or more agents is selected, deselect them or select Total.
If there are numerous agents connected to the project, it is efficient to set the agent names to be short. For more information about the agent name setting, see the following.
Monitoring each classified agents
You can select and monitor agents by each group classified through agent settings.
-
Agent: You can select an individual agent or all agents.
-
Agent by cluster: In case of the database that consists of multiple clusters, agents can be monitored by cluster.
To use Agent by cluster, configure the following options for the DBX agent (whatap.conf). For more information, see the following.
cluster_name={cluster_name}
Editing a dashboard widget
The widgets on the dashboard can be adjusted to the desired size and placed in the desired positions. You can also delete unnecessary widgets or add them again.
Resizing a widget
Click and hold the mouse on the element at the lower right of the widget, and then drag it to the desired size. Grids with a uniform horizontal-to-vertical ratio appear and the size of the widget can be adjusted in each grid.
Moving a widget
When you move the mouse cursor to the top of the widget, the cursor shape changes to . At this time, you can move the widget by dragging with your left mouse button and move to the desired position.
Deleting a widget
Right-click on a widget to delete. If you select Delete, the widget is deleted from the dashboard.
Adding a widget
Move the mouse cursor to an empty space on the dashboard and then right-click on it. Select a widget to add from the pop-up menu. Place the widget to the desired position and then resize it.
-
For more information about the widgets that can be placed on the dashboard, see the following.
-
The widgets that can be added are fixed, but we plan to increase supports through future updates.
Custom widget
The custom widget feature allows users to add their own metrics in addition to the metrics provided by default on the dashboard. If you have metrics to continuously monitor on the dashboard in addition to the metrics provided by default, use the custom widget feature.
Add
-
Right-click an empty space on the dashboard.
-
On the pop-up menu, select Add Custom Widgets.
-
If Custom Widget is created on the dashboard, select Select indicator.
-
When the Add Custom Widgets window appears on the right of the screen, enter the widget name in Title.
-
In the metrics list at the bottom of the screen, select the button on the upper right of the metric to add.
-
After the metric has been changed, select Save.
-
You can add up to four metrics. To add metrics, select .
-
Ti change the metric, select a different metric in the metrics list. If you select , the metric is added.
-
If you do not enter the widget title, it is saved as a custom widget.
Selecting the chart type
In chart type of the Add Custom Widgets window, you can select the following types.
-
Series: Time series chart that displays the metric changes over time.
-
Equalizer: Bar graph chart that can compare the performance for multiple metrics. By visually displaying various metrics on one screen, you can easily see the overall performance status.
-
Pie: Pie chart useful for checking the composition ratio for multiple metrics relative to the whole.
-
Text: It displays the figures for the metrics in text.
Selecting the data merging method
In Merge target of the Add Custom Widgets window, you can select the way to display metrics collected from agents.
-
Sum: It sums up the metric values collected from multiple agents and displays them as a single value. This option is useful for displaying the overall total by merging data from multiple agents.
-
Avg: It averages the metric values collected from multiple agents and displays them as a single value. This option is useful for checking the average performance or status of all agents.
-
Max: By selecting the largest value among the metric values collected from multiple agents, it displays a single value. This option is useful for identifying the highest performance consumption or worst performance cases among agents.
-
No Merge: It displays the metrics for each agent.
The available merge method may differ depending on the chart type.
Changing a metric
You can change or add metrics for the metrics added to the custom widget.
-
Select on the upper right of the custom widget added on the dashboard.
-
When the Add Custom Widgets window appears on the right of the screen, select a metric to change.
-
Select a metric to change from the metric list at the bottom of the screen. To add a metric, select on the upper right of the metric to add.
-
After the metric has been changed, select Save.
The features of the icon buttons in the list for added or changed metrics are as follows:
-
: You can exclude the metric from the custom widget.
-
: You can set the alert for the metric. When the button is selected, the Alert > Event Configuration appears. For more information about the Metrics event setting, see the following.
Using the widget options
The icons displayed on the widget function as follows:
-
: You can see the features and information for major widgets. (multilingual support planned)
-
or : You can enlarge or reduce the range of the vertical axis.
-
: You can view the widget's data on a wider screen.
-
: A detail window appears where you can search the data of the widgets separately by agent.
The provided options may differ depending on the widget.
Setting presets
You can save and load the settings and layouts for widgets on the dashboard. You can create a new preset by adjusting its size and placing it on the desired position.
-
Using Default, the specified preset cannot be changed.
-
You can save or modify the presets with the Edit project role. For more information about the member roles, see the following.
Creating a new preset
-
Place the widgets in the desired format on the dashboard. You can also resize and place only the widgets to be frequently checked.
-
On the upper right of the screen, select .
-
Enter a new preset name.
To save the selected agent data, select Include agent selection history.
-
Select Save.
You can see the newly saved preset from the preset list.
-
If any changes are made to the newly created preset, save the preset again. Select and then save the preset with the same name. Any changes are overwritten in the existing preset.
-
If you go to another menu without saving changes on the dashboard, the changes are not saved.
-
Presets are saved on a project basis and can be shared with other users.
Deleting presets
If you have unused presets, you can delete them from the preset list. Select on the right of the item to delete from the preset list.
The Default preset cannot be deleted.
Learning about the main widgets
The types of widgets that can be placed on the dashboard are as follows.
-
DB Status
You can see the status of agents (instances) in the project. Individual agents are displayed in a hexagon. If you hover your mouse over it, the current status and IP address appear.
The status of the agent can be classified as follows:
-
Normal: Number of normal agents.
-
Inactive Agents: Number of disconnected agents.
-
-
Clusters
You can see the status of Master and Slave instances.
NoteTo use the Clusters widget, set the following options in the DBX agent (whatap.conf). For more information, see the following.
whatap.confcluster_name={cluster_name}
-
Instantaneous Ops Per Sec
It displays the number of commands per second (
instantaneous_ops_per_sec
) processed by the server using a pie chart. It can be used as a metric to compare and monitor the performance between multiple instances.TipThe
instantaneous_ops_per_sec
metric indicates the number of commands per second that the server processes. This is useful for tracing and monitoring the workload that your instances are currently handling. The number of commands per second indicates the amount of commands the server receives and processes. It helps you check the server's workload and identify performance issues. -
Key Status
It monitors the server's key status. It is necessary to understand the database running status and optimize its performance.
-
Evicted Keys: Displays the number of cached or stored keys that have expired or deleted according to the memory policy. If the Evicted Keys metric increases, there may be an issue of insufficient memory or a cache policy issue.
-
Expired Keys: Displays the number of expired keys. In Redis, you can set the expiration time for keys, and the expired keys are deleted automatically. If the Expired Keys metric increases, you may need to review the expiration policy in the database or check user behaviors.
-
Keyspace Hits: Displays the number of successful requests to find in cache or memory while retrieving and loading keys from the database. The metric helps you evaluate the cache efficiency and understand the access patterns to keys.
-
Keyspace Misses: Displays the number of unsuccessful requests to find in cache or memory while retrieving and loading keys from the database. The metric helps you evaluate the cache efficiency and take measures for performance improvement.
NoteIn the Redis database, the keys include various data types such as strings, hashes, lists, sets, and ordered sets.
-
-
Connected Clients (
connected_clients
)Number of the connected clients. You can see the number of concurrent connections to the server. A rapid increase in the number of client connections to the server can indicate an unexpected increase in traffic or other issues. Accordingly, you need to recognize and respond quickly to the issues.
-
Used Memory (
used_memory
)Amount of memory (bytes) requested for data saving. There may be a difference from the amount of memory allocated by the OS. If the
used_memory
metric is too high, it may be required to review the memory management policy and optimize the dataset to reduce the memory usage. -
Total Commands (
total_commands_processed
)A widget that displays the total number of commands processed after the server startup. It represents the cumulative number of all commands performed by the server and includes the number of requests from all clients processed while providing services. The
total_commands_processed
metric is useful for finding the level of server activities and evaluating the performance. This metric increases over time and you need to continuously monitor the activity level of the server by tracing value changes. -
Used CPU (
used_cpu
)Redis uses a single CPU per instance and displays the allocated CPU usage. The
used_cpu
metric can help you understand how the server uses CPU resources. If this metric increases to an unexpected level, it is required to recognize any performance issue and optimize the performance. -
Commands(/S) (
instantaneous_ops_per_sec
)It displays the number of commands per second (
instantaneous_ops_per_sec
) processed by the server for each instance over time through the time series chart. This widget is useful for monitoring trends over time and identifying performance trends and issues.TipThe
instantaneous_ops_per_sec
metric indicates the number of commands per second that the server processes. This is useful for tracing and monitoring the workload that your instances are currently handling. The number of commands per second indicates the amount of commands the server receives and processes. It helps you check the server's workload and identify performance issues. -
Total Net Input Bytes
Widget that displays the total amount of network input (bytes) received by the server. It measures the total amount of all network traffic received from clients. This widget helps you see how much network bandwidth the clients have consumed while communicating with the server. It may be useful for tracing the client activity levels and managing the network load.
-
Blocked Clients (
blocked_clients
)It displays the number of clients in a blocked state on the server. A blocked client is waiting for specific tasks on the server and the client is blocked until the pending task is completed. You can monitor how many clients are in a blocked state. This metric allows you to easily see the current load status of the server.
-
Keys Total (
keys_total
)Total number of keys saved in the Redis database. The keys include various data types such as strings, hashes, lists, sets, and ordered sets. By tracing the overall amount of data saved in the database, you can see the database size and trace the data growth trend. If this metric rapidly increases, you may need to allocate additional memory or review the data cleaning policy.
-
Evicted Keys (
evicted_keys
)Number of keys removed due to the database's maxmemory limit. The removal policy is based on the
maxmemory-policy
setting in the configuration file. In general, theevicted_keys
metric increases when the validity period of the key data expires or the key is deleted due to insufficient memory. By monitoring theevicted_keys
metric, you can trace how often keys are deleted and identify the memory usage and performance. -
Keyspace Hits (
keyspace_hits
)Number of successful key inquiry requests. If the value
get key
exists, it is incremented by 1. Hit Rate can be calculated as (keyspace_hits
/ (keyspace_hits
+keyspace_misses
) * 100 ). Thekeyspace_hits
metric is one of the important metrics for evaluating the cache efficiency and performance of the database server. -
Rdb Changes Since Last Save
A widget that represents the amount of data changed after the last
SAVE
command. By tracing the amount of changed data, you can see the changed amount of data. If you have a large amount of changes, adjust the backup cycle or review the data backup policy to reduce the possibility of data loss. -
Total Net Output Bytes
A widget that displays the amount of all network traffic sent by the database server to its clients. You can trace the activities of the server sending data to clients. This allows you to evaluate the server's response speed and the performance to handle client requests.
Session Table widget
You can view real-time active sessions in the Session Table widget at the bottom of the screen.
In the table list, the text colors are changed black → orange → Red, which means that the performance of the session is getting slower.
Filtering the table data
-
Select on the upper right of the table.
-
Select a column header and a condition in the table.
-
Enter a desired value in the Enter conditions field.
-
Select Save.
Setting the table columns
You can hide the table header columns or add any of them. You can also change the column order. Select .
-
After configuration, select Confirm to apply the settings in the table.
-
In the search bar, enter text to search the desired columns. Only the columns that meet the entered text are displayed.
-
Images may differ depending on the product, project, or menu.
Adding columns
From the list, select the items to add as table header columns. To select all items, select Select All.
Deleting columns
From the list, unselect the columns to delete. Alternatively, select on the right of the item to delete from the list.
Changing the column order
Drag an item to reposition from the list, and then move it to the desired position.
Initializing the configuration
To cancel all changes and reset them, select Initialized.
Column information guide
- Active session
- Process information
It is the description of column items in the Active sessions table. For more information, see the following link.
Item | Description |
---|---|
id | Unique 64-bit client ID |
addr | Client IP address and port number |
age | Total connection time |
cmd | Last executed command |
db | Current database ID |
events | r : Can read the client socket (event loop)w : Can write the client socket (event loop) |
fd | Client socket file's descriptor (file descriptor) number |
flags | The client type, N indicates normal client. (normal/master/replica/pubsub) |
idle | Time (seconds) whose connection status is idle. |
multi | Number of commands in the MULTI/EXEC context |
obl | Output buffer length |
oll | Output list length (response is queued to this list if the buffer is full) |
omem | Amount of memory the client uses for output buffers |
psub | Number of matched subscription patterns |
qbuf | Query buffer size (0 is not query dependent) |
qbuf-free | Query buffer free space (0 indicates the buffer is full, no free space) |
sub | Number of subscription channels |
user | Client's authenticated username |
WhaTap basically stores the client-related information.
The following items are the metrics collected if the XOS agent has been installed.
Item | Description |
---|---|
cputime | CPU usage time |
cpuusage | CPU Utilization |
elapse | Elapsed time of CPU usage |
vsize | Virtual memory size (Kb) |
rss | Resident Set Size (RSS) that is the number of physical pages associated with the process. |
state | Process status |
ioread | Actual time spent reading the block (milliseconds) |
iowrite | Actual time spent writing the block (milliseconds) |
pss | Process specific memory usage + Percentage of shared memory occupied by one process |
uid | user id |
cmd | Executing command |
longcmd | Full path of cmd |
Downloading the view list
You can download the current Session Table list in CSV format. Select on the upper right of the Session Table section.
The naming format of the downloaded file is as follows:
Active sessions: activeSession_HH_MM_SS.csv
Viewing in a new window
You can see the Session Table list on a new wider screen. Select on the upper right of the Session Table section.
Cluster Info widget
This widget provides the Redis cluster status and configuration data in a table format. You can see the overall health of the cluster of multiple instances at a glance.
The data for each column in the table is as follows:
-
instance: Name or identifier of each instance in the cluster.
-
cluster name: Name of the cluster to identify a cluster.
-
flags: Flag indicating the status or characteristics of each instance.
-
ip: IP address of each instance.
-
link status: Connection status of each instance.
-
slots: Slots used in the cluster. Each slot is a part of data in the cluster. This information helps you find the data distribution and load balancing status in the cluster.
To use the Cluster Info widget, set the following options in the DBX agent (whatap.conf). For more information, see the following.
cluster_name={cluster_name}