Installation

WhaTap Amazon ECS Monitoring collects resource usage for each Docker container in real time via the META API and Cgroup directory. It retrieves and collects the status of Amazon ECS service, deployment, and container's instance through the WhaTap tasks linked with the Amazon ECS API.

To use the WhaTap monitoring service, after signing up Sign up, create a project. For more information about registration, see the following.

Caution

Role

Edit role is required for installation.

Project Creation

Create a project before installing the agent.

Log in WhaTap monitoring service.
To create a project, on the left of the screen, select All Projects > + Project.
Select a product in Select product for installation.
Set items for Project name, Data server region, Time zone, and more in order.
After all settings are finished, select Creating a project.

Note

A Data server region refers to a region (a bundle of data centers installed to provide cloud services). Selecting a specific region stores your data in the datacenter that belongs to that region.
Time zone is the reference time for generating alerts and reports.

To group multiple projects for management, select a group from Project groups or add a group. For more information about grouping, see the following.

To add a project with an organization selected, Groups of organization must be set.

After creating a project, you can install items in Management > Agent Installation. See the installation guide to proceed with installation.

Checking the project access key

Project access key is the unique ID for activating the WhaTap services.

In the installation guide section, select Getting the access key. After automatic reception of project access key, proceed to the next step.

Tip

After a project has been created, the Agent installation page appears automatically. If the Agent installation does not appear, select All projects on the left and then select a newly created project.

Note

Project access key

If you have already been granted a project access key, the key appears instead of the button.

Preliminary inspection

AWS CLI

For more information about installation, see the following.
Amazon ECS CLI

For more information about installation, see the following.
Python 2.x/3.x

For analysis of the json documents by the script, installation is required.

Note

After downloading the WhaTap image (whatap/ecs_mon) for installation, it is recommended to register it in the user registry.
The 6600 port must be open to send data to the WhaTap server.

Supported Platforms & Products

Items are distinguished depending on the launch type of Amazon ECS. Monitoring targets are task, service, deployment, and container instance.

EC2 Launch Type: Task, Service, Deployment, Container Instance
Fargate Launch Type: Task, Service, Deployment

The following monitoring procedures are also divided into the procedure for EC2 Launch Type monitoring and the procedure for Fargate Launch Type monitoring.

Note

For more information about the Amazon ECS launch types, see the following.

Monitoring procedure

ecs install

By referencing the installation guide section tab, release the WhaTap Amazon ECS agent through the dedicated task or sidecar.
By referencing the installation guide section tab, start the WhaTap monitoring service by enabling the monitoring service.

EC2 Launch Type

If you have created a project and then checked the Preliminary inspection, go to the installation guide section under Management > Agent Installation.
In the Create Task tab, select EC2 Launch Type.

Run the following script to create daemon type tasks. Add CLUSTER and REGION and then set ACCESSKEY and WHATAP_HOST to communicate with the collection server.

Linux
#!/usr/bin/env bash
export ACCESSKEY=
export WHATAP_HOST=
export CLUSTER=
export REGION=
export LAUNCH_TYPE=EC2
cat >whatap_ecs.json <<EOL
{ "containerDefinitions": [ {
    "name": "whatap-node-agent",
    "image": "whatap/ecs_mon",
    "cpu": 100,
    "memory": 50,
    "essential": true,
    "mountPoints": [{
        "containerPath": "/var/run/docker.sock",
        "sourceVolume": "docker_sock",
        "readOnly": true},{
        "containerPath": "/rootfs",
        "sourceVolume": "rootfs",
        "readOnly": true}],
    "environment": [{
        "name": "ACCESSKEY",
        "value": "$ACCESSKEY"
    },{
        "name": "WHATAP_HOST",
        "value": "$WHATAP_HOST"}
    ],
    "linuxParameters": {
        "capabilities": {
            "add": [
            "SYS_ADMIN",
            "SYS_RESOURCE",
            "SYS_PTRACE",
            "NET_ADMIN",
            "NET_BROADCAST",
            "NET_RAW",
            "IPC_LOCK",
            "CHOWN"]
        }}}],
"requiresCompatibilities": ["EC2"],
"volumes": [{
    "host": {"sourcePath": "/var/run/docker.sock"},
    "name": "docker_sock"},{
    "host": {"sourcePath": "/"},
    "name": "rootfs"}],
"family": "whatap-agent-task"}
EOL
aws ecs register-task-definition \
--cli-input-json file://$(pwd)/whatap_ecs.json \
--region $REGION
aws ecs create-service --cluster $CLUSTER \
--service-name whatap-node-agent \
--task-definition whatap-agent-task \
--scheduling-strategy DAEMON \
--launch-type $LAUNCH_TYPE \
--region $REGION

The following are the items required to run the script.
- ACCESSKEY (WhaTap project access key)
- WHATAP_HOST (WhaTap collection region IP)
- REGION (AWS Region)
- CLUSTER (ECS Cluster)

Fargate Launch Type

If you have created a project and then checked the Preliminary inspection, go to the installation guide section under Management > Agent Installation.

In the Create Task tab, select Fargate Launch Type.

Run the script to create the AWS ECS API Readonly IAM role.

Note
It is required for the WhaTap task to query the ECS service and collect service status in real time.
The following are the items required to run the script.
- ACCESSKEY (WhaTap project access key)
- WHATAP_HOST (WhaTap collection region IP; multiple IPs are connected with /)
- REGION (AWS Region)
- PROFILE (ECS Profile)
- CLUSTER (ECS Cluster)
- CLUSTER_CONFIG (ECS Cluster Config)
- TASK_EXECUTION_ROLE (Executing the WhaTap agent's ECS task)
- SUBNET_1 (WhaTap agent's Task SUBNET#1)
- SUBNET_2 (WhaTap agent's Task SUBNET#2)
- SGGRP (WhaTap agent's Task Security Group)

See the following example:

Linux
#!/usr/bin/env bash
export REGION=xxxxx
export WHATAP_ECS_ROLE_READONLY=WhatapEcsReadonly
export WHATAP_ECS_POLICY_READONLY=WhatapEcsReadonly
cat >ecs_trust.json <<EOL
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "ecs-tasks.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
EOL
aws iam create-role --role-name $WHATAP_ECS_ROLE_READONLY --assume-role-policy-document file://ecs_trust.json | python -c 'import json,sys;print(json.load(sys.stdin)["Role"]["Arn"])' > role_arn.txt
cat >ecs_readonly.json <<EOL
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecs:Describe*",
                "ecs:List*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}
EOL
aws iam --region $REGION create-policy --policy-name $WHATAP_ECS_POLICY_READONLY \
    --policy-document file://ecs_readonly.json | python -c 'import json,sys;print(json.load(sys.stdin)["Policy"]["Arn"])' > policy_arn.txt
export POLICY_ARN=$(cat policy_arn.txt)
aws iam --region $REGION attach-role-policy --role-name $WHATAP_ECS_ROLE_READONLY \
  --policy-arn $POLICY_ARN

Start the whatap-single agent with the created IAM ROLE. Service status is collected in real time by using the ECS API.

Linux
#!/usr/bin/env bash
export ACCESSKEY=
export WHATAP_HOST=
export PROFILE=
export REGION=
export CLUSTER=
export CLUSTER_CONFIG=
export TASK_EXECUTION_ROLE=
export SUBNET_1=
export SUBNET_2=
export SGGRP=
export WHATAP_ECS_ROLE_READONLY=WhatapEcsReadonly
export PROJNAME=whatap-monitoring
export LAUNCH_TYPE=FARGATE
export WHATAP_ROLE_ARN=$(cat role_arn.txt)
cat >docker-compose.yml <<EOL
version: '3'
services:
whatap-ecs-agent:
    image: whatap/ecs_mon
    environment:
    - ACCESSKEY=$ACCESSKEY
    - WHATAP_HOST=$WHATAP_HOST
    - FARGATE_HELPER=true
EOL
cat >ecs-params.yml <<EOL
version: 1
task_definition:
task_execution_role: $TASK_EXECUTION_ROLE
task_role_arn: "$WHATAP_ROLE_ARN"
ecs_network_mode: awsvpc
task_size:
    mem_limit: 0.5GB
    cpu_limit: 256
run_params:
network_configuration:
    awsvpc_configuration:
    subnets:
        - "$SUBNET_1"
        - "$SUBNET_2"
    security_groups:
        - "$SGGRP"
    assign_public_ip: ENABLED
EOL
ecs-cli compose --project-name $PROJNAME service up \
    --cluster-config $CLUSTER_CONFIG \
    --ecs-profile $PROFILE --region $REGION

Release the WhaTap sidecar agent to the end user task as follows. Add ACCESSKEY and WHATAP_HOST as environment variables to allow communication with the collection server.

Linux
version: '3'
services:
xxxxx:
    image: xxxx
    ports:
    - "xx:xx"
    ...
whatap-ecs-agent:
    image: whatap/ecs_mon
    environment:
    - ACCESSKEY=
    - WHATAP_HOST=

Activate monitoring

In the Start Monitoring tab, select Activate monitoring to start the monitoring service.

Enabling the monitoring service
Disabling the monitoring service

Precautions upon removal

To completely avoid incurring costs, delete the following items:

ECS Service: whatap-ecs-agent
Task Sidecar: whatap-ecs-agent
Role: WhatapEcsReadonly

Project Creation​

Checking the project access key​

Preliminary inspection​

Supported Platforms & Products​

Monitoring procedure​

EC2 Launch Type​

Fargate Launch Type​

Activate monitoring​

Precautions upon removal​

Project Creation

Checking the project access key

Preliminary inspection

Supported Platforms & Products

Monitoring procedure

EC2 Launch Type

Fargate Launch Type

Activate monitoring

Precautions upon removal