Cloud Agent Configuration

This document is a guide to configure the WhaTap Cloud Agent in the AWS environment. It is configured to be immediately applicable to the operating environment, including key settings, field descriptions, roles configuration, and execution commands.

Agent CONFIG.

Instructions for setting up AWS Aurora and RDS

Configure AWS Aurora and RDS

config.yaml
input:
  csp: "aws" # Enter the Cloud Service Provider.
  namespace: "rds"
  region: "us-east-1" # Set the region where the instance is located.
  instances: # The specified instances are collected always. Instances listed under this property are collected at all times, regardless of the autoscaling setting.
    - name: "mysql-rds"
      slow_query: true # To use the slow query page, set it to true. This feature is avaiable only in MySQL and PostgreSQL instances.
  clusters:
    autoscale:
      enabled: false # If the autoscale is enabled, the autoscaled instances are added or deleted for collection.
      interval: 60 # Autoscale checking cycle for the specified clusters (unit: second)
    names:
      - "database-cluster-name"
  metrics: # Enter the metrics to collect.
    - "CPUUtilization"
    - "FreeStorageSpace"
    - "FreeableMemory"
    - "ReadLatency"
    - "WriteLatency"
    - "ReadIOPS"
    - "WriteIOPS"
    - "NetworkReceiveThroughput"
    - "NetworkTransmitThroughput"
    - "FreeLocalStorage"
  logs:
    enabled: false # Can be enabled/disabled. Select true/false.
    groups: # Add the desired AWS log groups.
      - "/aws/rds/cluster/database-cluster-name/error"
      - "/aws/lambda/MyLambda"

output: # Enter the WhaTap information to receive the collected metrics.
  license: "abcdefg-higjgkgjk-zxcvnbnbmc"
  host: "127.0.0.1"

Field	Description
`csp`	Indicates `aws` (Amazon Web Services) as a cloud service provider.
`namespace`	Namespace of the service to be collected - Fixed value: Enter `rds` when configuring Aurora and RDS.
`region`	AWS Region code - e.g. `us-east-1` (Virginia Region)
`instances`	It always collects the specified instances regardless of whether they are auto-scaled, as long as they are in the list of instances to collect.
`instances[].name`	RDS InstanceName - e.g. `mysql-rds`
`instances[].slow_query`	Whether to collect slow queries for the instance. It is available only in MySQL and PostgreSQL instances. - `true`: To be collected - `false`: Not to be collected
`clusters`	RDS Cluster-related settings
`clusters.autoscale.enabled`	Whether to collect cluster autoscale - `true`: Automatically add/remove/collect instances - `false`: Only the instances directly entered in `instances` are included in the collection targets.
`clusters.autoscale.interval`	Polling interval for detecting autoscaling status (unit: second)
`clusters.names`	List of cluster names to check for autoscaling status
`metrics`	List of metrics to collect - e.g. `CPUUtilization`, `FreeStorageSpace`, etc - Note. Shortcut to the official Amazon CloudWatch Metrics documentation for Amazon RDS
`output.license`	WhaTap's license key to transmit the collected data
`output.host`	WhaTap server host address (IP) to transmit the collected data to

Cross-account monitoring (multi AWS account)

A single cloud agent can monitor RDS/DocumentDB across multiple AWS accounts. When using multiple accounts (separate by environment, partner accounts, etc.), you do not need to install an agent per account. One agent uses STS AssumeRole to query resources in other accounts.

Item	Description
Supported targets	AWS RDS (Aurora, standalone), DocumentDB
Authentication method	EC2 Instance Profile + STS AssumeRole

Choosing a configuration method

There are two methods for cluster configuration, and they cannot be used simultaneously.

names: Single-account only (queries only the account of the EC2 where the agent is installed, using its IAM Role)
account_configs: Cross-account support (queries multiple accounts via STS AssumeRole)

`account_configs` configuration

config.yaml
input:
  csp: "aws"
  namespace: "rds"
  region: "ap-northeast-2"
  clusters:
    autoscale:
      enabled: true
      interval: 300
    account_configs:
      # Local account (omitting assume_role_arn uses the local account)
      - clusters: ["local-aurora-cluster"]
        instances: ["standalone-mysql-1"]

      # Another account A
      - assume_role_arn: "arn:aws:iam::111111111111:role/DBXCRole"
        clusters: ["partner-cluster-1", "partner-cluster-2"]

      # Another account B
      - assume_role_arn: "arn:aws:iam::222222222222:role/QARole"
        clusters: ["qa-cluster"]
        instances: ["qa-standalone-pg"]
  metrics:
    - "CPUUtilization"
    - "FreeStorageSpace"
    - "FreeableMemory"
    - "ReadLatency"
    - "WriteLatency"

output:
  license: "your-license-key"
  host: "127.0.0.1"

Table | account_configs fields

Field	Required	Description
`assume_role_arn`	N	STS AssumeRole ARN. If omitted, the local account is used.
`clusters`	Y	Aurora/DocumentDB cluster names. Member instances are automatically discovered.
`instances`	N	Standalone RDS instance names not belonging to a cluster.

Note

Difference between clusters and instances

	`clusters`	`instances`
Target	Aurora clusters, DocumentDB clusters	Standalone RDS instances without a cluster
Behavior	Auto-discovers member instances by cluster name	Directly specifies instance names
Autoscale	Automatically reflects instance additions/deletions	N/A
Example	Aurora MySQL cluster	Standalone RDS MySQL instance

Configuration example 1: Single account + standalone instance

clusters:
  autoscale:
    enabled: true
    interval: 300
  account_configs:
    - clusters: ["prod-aurora"]
      instances: ["legacy-mysql-standalone"]

Configuration example 2: 3 accounts (local + partner + QA)

clusters:
  autoscale:
    enabled: true
    interval: 300
  account_configs:
    - clusters: ["prod-cluster", "staging-cluster"]

    - assume_role_arn: "arn:aws:iam::111111111111:role/DBXCRole"
      clusters: ["partner-prod", "partner-dev"]

    - assume_role_arn: "arn:aws:iam::222222222222:role/QADBXCRole"
      clusters: ["qa-cluster"]
      instances: ["qa-standalone-pg"]

Caution

names and account_configs cannot be used simultaneously. A validation error occurs at agent startup.
ARN format is strictly validated. arn:aws:iam::<12-digit number>:role/<name> — be careful not to omit the double colons (::).
Each account_config requires at least 1 cluster.
autoscale.enabled: true is required for cluster discovery to work.

Role setting

For the agent to operate normally, set the required roles for each service.

AWS Aurora and RDS roles

The Cloud Agent calls the AWS API to perform the following features.

Check the monitoring targets by querying the AWS Aurora and RDS instance and clusters.
Collecting CloudWatch metrics

IAM inline policy

Add the JSON policy as inline policy for IAM user or role. If you are running the agent on EC2, grant the policy to the IAM role associated with the EC2 instance.

This policy is applied to both Aurora and RDS services.
The resources are configured to apply to all Aurora and RDS resources. If needed, the resource ARNs can be restricted depending on the security policy.
The agent can use the read-only APIs only. The roles that can affect the database are not included.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RDSPermissions",
      "Effect": "Allow",
      "Action": [
        "rds:DescribeDBInstances",
        "rds:DescribeDBClusters",
        "rds:DescribeDBClusterParameters",
        "rds:DescribeOptionGroups"
      ],
      "Resource": "*"
    },
    {
      "Sid": "CloudWatchMetricsPermissions",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics"
      ],
      "Resource": "*"
    },
    {
      "Sid": "CloudWatchLogsPermissions",
      "Effect": "Allow",
      "Action": [
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:FilterLogEvents",
        "logs:GetLogEvents"
      ],
      "Resource": [
        "arn:aws:logs:*:*:log-group:/aws/rds/*"
      ]
    }
  ]
}

Cross-account IAM permissions

Additional IAM permissions are required to configure cross-account monitoring using account_configs.

Agent EC2 Instance Profile

Grant the EC2 running the agent permission to assume the role of the target account.

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": [
        "arn:aws:iam::111111111111:role/DBXCRole",
        "arn:aws:iam::222222222222:role/QARole"
      ]
    }
  ]
}

Target account Cross-Account Role

Create a role in the target account that allows access from the agent account.

Trust Relationship: Allow the agent account

{
  "Principal": {
    "AWS": "arn:aws:iam::<agent-account-ID>:role/DBXC-EC2-Instance-Role"
  },
  "Action": "sts:AssumeRole"
}

Permission Policy: RDS/CloudWatch read permissions

{
  "Effect": "Allow",
  "Action": [
    "rds:DescribeDBInstances",
    "rds:DescribeDBClusters",
    "cloudwatch:GetMetricData",
    "logs:FilterLogEvents"
  ],
  "Resource": "*"
}

Pricing information

Caution

The Cloud Monitoring Agent is free, but note that separate charges may apply depending on the use of Cloud services.

AWS Pricing

Cloud monitoring agents are provided without separate SaaS fees, and the monitoring feature can be used simply by installing the agent. However, because the metrics are collected through the API for Cloud services such as AWS CloudWatch, separate charges may apply depending on the CloudWatch usage.

This agent basically collects metrics by calling the GetMetricData API every 1 minute. Note that the charge may increase depending on the call frequency and collection targets.

For more information about the pricing policy, see the Amazon CloudWatch pricing in AWS official documentation page.

Caution

names and account_configs cannot be used simultaneously. A validation error occurs at agent startup.
ARN format is strictly validated. arn:aws:iam::<12-digit number>:role/<name> — be careful not to omit the double colons (::).
Each account_config requires at least 1 cluster.
autoscale.enabled: true is required for cluster discovery to work.

Agent CONFIG.​

Configure AWS Aurora and RDS​

Choosing a configuration method​

account_configs configuration​

Configuration example 1: Single account + standalone instance​

Configuration example 2: 3 accounts (local + partner + QA)​

Role setting​

AWS Aurora and RDS roles​

IAM inline policy​

Agent EC2 Instance Profile​

Target account Cross-Account Role​

Pricing information​

AWS Pricing​

Agent CONFIG.

Configure AWS Aurora and RDS

Choosing a configuration method

`account_configs` configuration

Configuration example 1: Single account + standalone instance

Configuration example 2: 3 accounts (local + partner + QA)

Role setting

AWS Aurora and RDS roles

IAM inline policy

Agent EC2 Instance Profile

Target account Cross-Account Role

Pricing information

AWS Pricing