Troubleshooting
Check for possible problems upon database agent installation.
Firewall configuration checking
Unblock your firewall. Monitoring data cannot be sent to the server if the TCP outbound firewall is set for the WhaTap server.
How to check the firewall
It is normal only when the following connection-related information appears after executing the telnet command. (telnet [Server IP] [Server Port])
$ telnet 52.193.60.176 6600
Trying 52.193.60.176...
Connected to 52.193.60.176.
Escape character is '^]'.
The WhaTap collection server data can be found in Management > Agent installation.
Checking agent logs
If the agent cannot connect to the database or some data is not collected, check the agent logs first. The following sections describe causes and solutions based on the messages recorded in the logs.
- logs/: Agent log folder created in the agent installation path.
- nohup.out: Standard output record to check first when the agent fails to start. It is created in the agent installation path when you run the agent with
./startd.sh.
After changing the configuration file (whatap.conf) or the JDBC driver, restart the agent.
./stop.sh
./startd.sh
Cannot connect to the database
JDBC driver does not support the authentication method
PSQLException: The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file ...
PostgreSQL 14 or later uses scram-sha-256 as the default authentication method. If the jdbc/ folder has no driver, the agent connects with its built-in default driver, which does not support this authentication method, so the connection fails. Place a PostgreSQL JDBC 42.2.0 or later driver (.jar) in the jdbc/ folder and restart the agent. Downloading JDBC
Agent IP not in the access list (pg_hba.conf)
FATAL: no pg_hba.conf entry for host "<Agent IP>", user "whatap", database "postgres", no encryption
In pg_hba.conf on the database server, add a line that allows the IP of the server where the DBX agent is installed, and then reload the configuration. The agent also connects to databases other than the default database to collect some information, so setting the database field to all is recommended.
host all whatap {AGENT_IP}/32 scram-sha-256
SELECT pg_reload_conf();
No database connection privilege
FATAL: permission denied for database "postgres"
Detail: User does not have CONNECT privilege.
Grant the monitoring account the CONNECT privilege on the database shown in the log. If db is not set in whatap.conf, the agent connects to the postgres database.
grant connect on database postgres to whatap;
Password mismatch
FATAL: password authentication failed for user "whatap"
The agent does not stop and keeps retrying the connection. Register the account information again with uid.sh, and then restart the agent.
./uid.sh {DB_USER} {DB_PASSWORD}
Some data is not collected
If the agent is connected to the database but privileges or extension modules are missing, only the related data is not collected.
| Symptom | Action | Data not collected |
|---|---|---|
Agent log: permission denied for function pg_ls_waldir | Run grant pg_monitor to whatap;, then restart the agent | Number of WAL files |
Agent log: pg_stat_statements must be loaded via shared_preload_libraries | Add pg_stat_statements to shared_preload_libraries, then restart the database | PG SQL Statistics |
| No error in the agent log, and a setup guide is displayed on the PG SQL Statistics screen | Run create extension pg_stat_statements; in the database the agent connects to, then restart the agent | PG SQL Statistics |
Without the pg_monitor privilege, the status and SQL of other users' sessions are displayed as <insufficient privilege> without an error log.
For how to set up pg_stat_statements, see PG SQL Statistics.
DB roles and agent configuration
There are menus where data is not viewed depending on the DB roles and agent configuration. See the following to assign the required roles and set the agent options.
Example shows a monitoring account creation as whatap: create user whatap with password ‘DBPassword’;
| Menu | DB options and roles | Agent CONFIG. | Remarks |
|---|---|---|---|
| Basic menu | grant pg_monitor to whatap; | - | DBX agent v1.6.15 or later |
| Dashboard > Slow query | DB parameter:log_min_duration_statement=500(e.g. SQL records taking 500 ms or more) DB role grant PG_READ_ALL_SETTINGS to whatap; | xos.confslow_query_interval=30 (Default 30 seconds)whatap.conf slow_query_log=ON | - |
| Instance monitoring > Session kill | grant pg_signal_backend to whatap | - | - |
| Analysis > Top object | grant select on all tables in schema {schema_name} to whatap; | whatap.confpg_object=true | DBX agent v1.6.15 or later The lists in the Bloating and Age tabs may not be viewed when the thresholds (size, ratio) are not exceeded even if the roles and agent options have been configured. |
| Stat/Report > Database size | grant usage on schema {schema_name} to whatap; | whatap.conftables=true | It is the role required to view the table size under the Database size menu. |
| Object information (Object detail) modal window | grant select on all tables in schema {schema_name} to whatap; | - | Pop-up window that appears when the table is clicked in Top object, Database size, and SQL details (Plan). With no role, the column list does not appear. |
Slow queries are not collected
PostgreSQL slow queries are collected by the XOS agent installed on the DB server, which reads the PostgreSQL log files. For setup instructions, see Slow Query. If slow queries are still not collected after setup, check the following in the DBX agent log and the XOS agent log.
| Log | Cause | Action |
|---|---|---|
DBX: No whatap.conf: slow_query_log = ON line | Slow query collection of the DBX agent is disabled. The default value in PostgreSQL is off. | Set slow_query_log=on in DBX whatap.conf, then restart the DBX agent |
DBX: log_min_duration_statement: -1 slowLog OFF | The slow query threshold (log_min_duration_statement) is not set in the database. | Set log_min_duration_statement to a value greater than 0 in the database settings, then restart the DBX agent |
DBX: No log_filename: line after slowLog ON | The monitoring account does not have the privilege to query the log path. | Run grant pg_monitor to whatap; or grant pg_read_all_settings to whatap;, then restart the DBX agent |
XOS: [slow_query_pg] stat (<log file path>) failed. errno: 2 | No file exists at the path shown in the log. If PostgreSQL runs in a container, the path inside the container is passed and may differ from the actual path on the DB server (host). | Create a symbolic link so that the same path is accessible on the host. Example: ln -s /data/postgres/data /var/lib/postgresql/data |
DBX: xos.conf slow_query_log ONXOS: ignored (set in xos.conf) | If a slow_query= path is set in xos.conf, this value is used instead of the path passed by the DBX agent. | If the path differs from the actual log file, correct the path or comment out the slow_query= line, then restart the XOS agent |
Constraints
Multiple regions impossible to be selected as a single project
-
In the WhaTap monitoring service website, when creating a project, you cannot select multiple regions. When using multiple regions, create separate projects.
-
If servers exist in multiple regions as in the cloud environment, the collection servers must be located in each region to avoid constraints on the performance data collection such as network latency.
-
WhaTap builds collection servers for each region to meet customer requirements.
Duplicate agent names (ONAME) impossible to be used
WhaTap DBX monitoring uses the agent's IP address and port to identify the agent. If the database server to be monitored in the user environment uses the same IP and port, the WhaTap server cannot identify the server instance.
-
If the server instance is not dynamically expanded, you can bypass it by manually setting a name for database identification by seeing the following.
-
If the databases to be monitored use internal addresses or duplicate IPs due to network virtualization, use separate database naming patterns by seeing the following.