Event Notification Format
Common Alert Template
The common template can be applied in most cases. It allows you to receive alerts in a consistent format across various environments.
- Product Type: Application, Database, Kubernetes
- Event Type: Application Alerts, Database Alerts, Metrics Alerts
- Event Channel: SMS, Mobile, 3rd-party plugins, Plugins
Event Title
Event title format
[Level][Platform][ProjectName][ApplicationName][EventTitle]
Event title example
[Info][JAVA][Application Project][TC-0-1-8081][CRITICAL_HIGH_MEMORY]
Note
If ApplicationName is not set, it is omitted from the event title.
- Platform is displayed as one of the following.
- JAVA
- NODEJS
- PYTHON
- PHP
- DOTNET
- GO
- POSTGRESQL
- ORACLE
- MYSQL
- MSSQL
- BSM_JAVA
- CLOUDWATCH
- TIBERO
- KUBERNETES
- KUBE_NS
- URLCHECK
- URLCHECK_ADMIN
- CUBRID
- ALTIBASE
- CLUSTER
- REDIS
- MONGODB
- VR
- RUM
Event Message
This section describes the information that can be included in the event message. If optional is set to false, the item is always included in the message. If optional is set to true, the item is displayed only when the data is available.
Event message example
Project Name : Application Project
Project Code : 3
Agent Name : TC-0-1-8081
Message : RECOVERED: Memory is too high. less than 10%
Event Time : 2022-04-12 18:53:24 +0900
Event OFF Time : 2022-04-12 18:53:24 +0900
Alert Type : APPLICATION_MEMORY
Metric Name : memory
Metric Value : 20
Metric Threshold : 10
Stateful : true
Table | Event Message Components
| En | Ko | Supported Alert Types | Description |
|---|---|---|---|
| Project Name | Project name | All | - |
| Project Code | Project code | All | - |
| Agent Name | Agent name | All (optional) | oname |
| Message | Event message | All | - |
| Alert Type | Alert type | All | See the AlertType table below |
| Event Time | Event occurrence time | All | In 2022-04-13 10:40:49 +0900, +0900 indicates the GMT offset |
| Event Off Time | Event recovery time | All (optional) | In 2022-04-13 10:40:49 +0900, +0900 indicates the GMT offset |
| Metric Name | Metric name | All (optional) | Name of the metric used to evaluate the event condition |
| Metric Value | Metric value | All (optional) | Included when the metric value exceeds the threshold |
| Metric Threshold | Metric threshold | All (optional) | Threshold value used to determine the event condition |
| Stateful | Recovered-event notification | All (optional) | true if recovered-event notification is enabled; otherwise false |
| Event Rule | Event trigger condition | Metrics alerts | - |
| Target Filter | Event target filter | Metrics alerts | Evaluates event conditions only for metrics collected from specific targets |
| Repeat Count | Event repeat count | Metrics alerts | The event is triggered when the condition is met the specified number of times within the repeat duration |
| Repeat Duration | Event repeat duration | Metrics alerts | Time window used to evaluate the repeat count |
| Receiver | Receiver | Metrics alerts | - |
| Query | MXQL query | Composite metrics alerts | - |
| Rule | Event trigger condition | Composite metrics alerts | - |
| Query Period | Query period | Composite metrics alerts | - |
| Query Interval | Query interval | Composite metrics alerts | - |
| Silent Time | Silent time | Composite metrics alerts | - |
| Query | URL | Exception alerts | URL of the request that caused the exception |
| TXID | Transaction ID | Exception alerts | - |
| Class | Error class name | Exception alerts | - |
| Log Message | Log message | Server – file log alerts | - |
| Log File | Log file path | Server – file log alerts | - |
| IP | IP address | All server alerts | - |
| CPU | CPU usage | All server alerts | Snapshot at the time the event occurred |
| CPU_load1 | CPU load (1 min) | All server alerts | Snapshot at the time the event occurred |
| CPU_loadPerCore | CPU load per core | All server alerts | Snapshot at the time the event occurred |
| Memory | Memory usage | All server alerts | Snapshot at the time the event occurred |
| Swap | Swap usage | All server alerts | Snapshot at the time the event occurred |
| Disk Name | Used Percent | Free Size | IO Percent | Disk performance | All server alerts | Snapshot at the time the event occurred |
| Name | Bps | Pps | Network traffic performance | All server alerts | Snapshot at the time the event occurred |
| Message | Time | Name | Acknowledgement information | All server alerts | - |
Note
This section displays as much information as possible that can be provided for the event.
- AlertType is displayed as one of the following values.
| AlertType | Description |
|---|---|
| APPLICATION_CPU | Application CPU alert |
| APPLICATION_MEMORY | Application memory alert |
| APPLICATION_DISK | Application disk alert |
| APPLICATION_ACTIVE_TRANSACTION | Application active transaction alert |
| APPLICATION_ERROR_TRANSACTION | Application error transaction alert |
| APPLICATION_SLOW_TRANSACTION | Application slow transaction (response time) alert |
| METRICS | Metrics alert |
| COMPOSITE_METRICS | Composite metrics alert |
| ANOMALY | Anomaly detection alert |
| LOG_REALTIME | Real-time log alert |
| COMPOSITE_LOG | Composite log alert |
| SERVER_REBOOT | Server reboot alert |
| SERVER_NO_DATA | Server no-data alert |
| SERVER_PORT | Server port alert |
| SERVER_NETWORK_IOPS | Server network IOPS alert |
| SERVER_NETWORK_BPS | Server network BPS alert |
| SERVER_DISK_IO | Server disk I/O alert |
| SERVER_DISK_QUOTA | Server disk usage alert |
| SERVER_DISK_INODE | Server inode alert |
| SERVER_CPU | Server CPU alert |
| SERVER_MEMORY | Server memory alert |
| SERVER_CPU_STEAL | Server CPU steal alert |
| SERVER_MEMORY_SWAP | Server swap memory alert |
| SERVER_LOG_FILE | Server log file alert |
| SERVER_WINDOW_EVENT | Server Windows event alert |
| SERVER_OFF | Server alert OFF notification |
| SERVER_ACKNOWLEDGE | Server acknowledge notification |
| SERVER_PROCESS_COUNT | Server process count alert |
| SERVER_PROCESS_CPU | Server process CPU alert |
| SERVER_PROCESS_MEMORY | Server process memory alert |
| SERVER_PROCESS_OFF | Server process alert OFF notification |
| AGENT_ACTIVE | Agent activated alert |
| AGENT_INACTIVE | Agent deactivated alert |
| AGENT_REACTIVATED | Agent reactivated alert |
| URL | URL alert |
| TOO_MANY_EVENT | Too many events alert |
| CLOUD_WATCH | CloudWatch alert |
| EXCEPTION | Exception alert |
Server Alert Templates
Note
Server monitoring uses a different alert template as an exception.
- Product Type: Server
- Event Type: Server Alerts, Process Alerts, Log File / Windows Event Alerts
- Event Channel: SMS, Mobile, 3rd-party plugins, Plugins
Event Title
Event title format
[Level][INFRA] ProjectName ServerName EventTitle
Event title example
[Info][INFRA] Server Project TC-0-1-8081 CPU Used > 70%
Note
If ServerName is not specified, it is omitted from the event title.
Event Message
This section describes the information that can be included in an event message. If optional is set to false, the information is always included in the message. If optional is set to true, the information is displayed only when the data is available.
Event message example
Project Name : Application Project
Project Code : 3
Server Name : TC-0-1-8081
Event Message : [TEST] RECOVERED: Memory is too high. less than 10%
Event ON Time : 2022-04-12 18:53:24 +0900
Event OFF Time : 2022-04-12 18:53:24 +0900
Alert Type :
Metric Name : memory
Metric Value : 20
Metric Threshold : 10
Stateful : true
Table | Event Message Components
| En | Ko | Optional | Description |
|---|---|---|---|
| Project Name | Project Name | false | - |
| Project Code | Project Code | false | - |
| Server Name | Agent Name | true | oname |
| Event Message | Event Message | false | - |
| Event ON Time | Event Occurred Time | false | In 2022-04-13 10:40:49 +0900, +0900 indicates GMT offset. |
| Event OFF Time | Event Cleared Time | true | In 2022-04-13 10:40:49 +0900, +0900 indicates GMT offset. |
| Elapsed Time | Event Duration | true | - |
| Alert Type | Alert Type | true | - |
| Policy Name | Alert Policy Name | true | - |
| Metric Name | Metric Name | true | The name of the metric used to evaluate the event condition. |
| Metric Value | Metric Value | true | Indicates that the event condition is met when the metric value exceeds the threshold. |
| Metric Threshold | Metric Threshold | true | Indicates that the event condition is met when the metric value exceeds the threshold. |
| Event Rule | Event Rule | true | - |
| Alert Type | Alert Type | true | A field used to identify the event type. |
| Event Target Filter | Event Target Filter | true | (Metrics alerts) Evaluates event conditions only for metrics collected from specific targets. |
| Stateful | Resolved Event Notification | true | true if resolved event notifications are enabled, otherwise false. |
| Repeat Count | Event Repeat Count | true | (Metrics alerts) An event is triggered only when the condition is met the specified number of times within the repeat duration. |
| Repeat Duration | Event Repeat Duration | true | (Metrics alerts) An event is triggered only when the condition is met the specified number of times within the repeat duration. Applies to resolved events. |
| IP | IP | true | - |
| Log Content | Log Content | true | - |
| Log Source | Log Source | true | - |
| CPU | CPU | true | - |
| CPU_load1 | CPU_load1 | true | - |
| CPU_loadPerCore | CPU_loadPerCore | true | - |
| Memory | Memory | true | - |
| Swap | Swap | true | - |
| Disk Name / Used Percent / Free Size / IO Percent | Disk Name / Used Percent / Free Size / IO Percent | true | - |
| Traffic Name / Bps / Pps | Name / Bps / Pps | true | - |
| Acknowledge Message / Time / Name | Acknowledge Message / Time / Name | true | - |
Server Alert Notifications
Server alerts provide only an Event Message without an Event Title.
Server Alerts
| Event Type | Event Level | Event Message | Event Off Message |
|---|---|---|---|
| Restart | Critical | "System Rebooted" | Not supported |
| No Data | Warning | "Agent No Data" | Not supported |
| Port | Warning | "Port ${port} is down" | "Port ${port} is up" |
| Port (IP available) | Warning | "IP ${ip} Port ${port} is down" | "IP ${ip} Port ${port} is up" |
| Network IOPS | Warning or Critical | "NETWORK ${deviceId} pps > ${value} pps" | - |
| Network BPS | Warning or Critical | "NETWORK ${deviceId} bps > ${value} bps" | - |
| Disk I/O | Warning or Critical | "Disk ${mountPoint} IO Usage > ${value} %" | - |
| Disk Usage | Warning or Critical | "Disk ${mountPoint} Used > ${value} %" | - |
| inode | Warning or Critical | "Disk ${mountPoint} inode usage > ${value} %" | - |
| CPU | Warning or Critical | "CPU Used > ${value} %" | - |
| Memory | Warning or Critical | "Memory Used > ${value} %" | - |
| Steal | Warning or Critical | "CPU Steal > ${value} %" | - |
| Swap | Warning or Critical | "Swap Used > ${value} %" | - |
Process Alerts
| Event Type | Event Level | Event Message | Event Off Message |
|---|---|---|---|
| Process Count (Warning) | Warning or Critical | "Process ${processName} Count >= ${value}" | "Process ${groupName} Count < ${value}" |
| Process Count (Critical) | Warning or Critical | "Process ${processName} Count < ${value}" | "Process ${groupName} Count >= ${value}" |
| Process CPU | Warning or Critical | "Process ${processName} CPU > ${value}" | - |
| Process Memory | Warning or Critical | "Process ${processName} Memory > ${value}" | - |
Log Files and Windows Events
| Event Type | Event Level | Event Message | Event Off Message |
|---|---|---|---|
| Event Type | Event Level | Event Message | Event Off Message |
| LOG_DEFAULT | Warning or Critical | LOG ${logContent}" | - |
| LOG_FILE | Warning or Critical | LOG File: ${filePath}, Keyword : ${keyword}, [Event] ${logContent}" | - |
| LOG_WINDOW_EVENT | Warning or Critical | LOG ${logContent}" | - |
| LOG_AUDIT | Warning or Critical | AUDIT ${logContent}" | - |
| LOG_SCRIPT | Warning or Critical | SCRIPT ${logContent}" | - |