Custom EC2 CloudWatch Metrices For Memory and Disk Utilization

Set up CloudWatch Agent and collect memory and disk utilization metrices which are missing in the default EC2 instance monitoring metrices.

Introduction

If you are here then you might have noticed that EC2 instance Monitoring tab is missing two most important matrices that is very essentials for our EC2 instance which are:

1. Memory (RAM) Usage.
2. Disk Usage.

In default, EC2 instance monitoring metrices only contain CPU utilization and some other network and credit related metrices.

Default EC2 monitoring metrices

All these metrices are mostly useless to normal users, they might use CPU Utilization but it is not as helpful as memory and disk utilization metrices. To fix this we are going to get help from CloudWatch agent that is used to collect custom CloudWatch metrices.

What is CloudWatch Agent

The CloudWatch Agent is a tool from Amazon Web Services (AWS) that helps you collect and track system-level metrics and logs from your servers and applications. It allows you to monitor the performance and health of your infrastructure in real-time by sending data like CPU usage, memory usage, and disk activity to AWS CloudWatch. This helps you identify issues quickly and ensure your systems run smoothly. The CloudWatch Agent is easy to install and configure, making it a handy tool for maintaining your cloud environment.

Create Role for EC2 Instance

1. Go to IAM → Roles → Create Role → AWS Service → EC2 → Select `CloudWatchAgentServerPolicy` → Name: CloudWatchAgentRole → Create.

CloudWatchAgentServerPolicy

2. Assign Role to EC2 instance: EC2 Instance → Action → Security → Modify IAM Role → Select Role: CloudWatchAgentRole → Save.

Configure CloudWatchAgent on EC2 instance

1. SSH to your EC2 instance and run the following script to install cloudwatch agent. (Here I am using Ubuntu Server)

wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i -E ./amazon-cloudwatch-agent.deb
sudo apt-get update

2. Configure CloudWatch Agent.

• Edit amazon-cloudwatch-agent.json file:

sudo nano /opt/aws/amazon-cloudwatch-agent/bin/config.json

• Copy following .json file.

{
    "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "append_dimensions": {
        "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "disk": {
        "measurement": [
          "used_percent"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "/"
        ]
      },
      "mem": {
        "measurement": [
          "mem_used_percent"
        ],
        "metrics_collection_interval": 60
      }
    }
  }
}

• Link the config file.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

• Enable the config file.

sudo systemctl enable amazon-cloudwatch-agent

3. Restart CloudWatch Agent.

sudo systemctl restart amazon-cloudwatch-agent

4. Check CloudWatch agent logs.

tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

amazon-cloudwatch-agent.log

If everything looks good like in the above image then you are good to go.

5. Go to your instance → Monitoring → Include metrics in the CWAgent namespace. In a few minutes, you’ll find your custom CloudWatch metrices there.

Custom CloudWatch metrices

References

1. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html
2. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html