Creating Cloudwatch Dashboards per Environment with Python

Creating Cloudwatch Dashboards per Environment with Python

Show Me The Code Already

After installing Cloudwatch Agent to the machines you want to monitor, it’s time to create dashboards to view real-time metrics.

There are some ways to create Cloudwatch Dashboards such as creating them manually by selecting widgets from AWS Console, with Cloudformation etc.

I’ve decided to create them with Python because in DevOps literature, there is no such a thing as manually creating something. I also didn’t want to use Cloudformation because I like scripting and we have many applications to monitor in our company, thus, I needed something to iterate over our environments and create dashboards for each of them.

What Does This Code Do?

This particular code takes one parameter from terminal which is the environment you want to create dashboards for. You execute the script by:

python init_cloudwatch_dashboard.py {{environment}}

After executing the script, dashboards for that specified environment will get created based on the instance count. Since there is a widget count limitation per dashboard, when widget count exceeds 500, other dashboards will get created. If the count is lower than 500, only one will be created.

Let’s get right into the code!

import boto3
import sys
import json
 
cw  = boto3.client("cloudwatch")
ec2 = boto3.client("ec2")
 
if len(sys.argv) != 2:
    sys.exit("Please enter the environment parameter(Development, Staging, Tqa, Production, \"Shared Tools\")")
environment = sys.argv[1]
 
x, y                    = [0, 0]
y_coor_inc_per_ins      = 15
width, height           = [6, 6]
descriptor_width        = 24
widget_count            = 0
widget_dashboard_limit  = 450/2
widgets                 = []
instance_name           = ""
instance_id             = ""
instance_type           = ""
dashboard_name          = ""
dashboard_count         = 1
 
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>#
#>>>>>>>>>>>>>>>>>>>>> Function Definitions >>>>>>>>>>>>>>>>>>>>>>>>#
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>#
def extract_instance_info(i):
    global instance_name, instance_id, instance_type
 
    for t in i["Tags"]:
        if t["Key"] == "Name":
            instance_name = t["Value"]
 
        instance_id = i["InstanceId"]
        instance_type = i["InstanceType"]
 
def add_descriptor_widget():
    global widget_count
 
    descriptor = {"type": "text",
                  "x": x,
                  "y": y,
                  "width": descriptor_width,
                  "height": height/2,
                  "properties": {"markdown": "\n# "+ instance_name +"\n"
                                }
                 }
 
    widgets.append(descriptor)
    widget_count += 1
 
def add_memory_widget():
    global widget_count
 
    memory_widget = {"type": "metric",
                     "x": x,
                     "y": y+3,
                     "width": width,
                     "height": height,
                     "properties": {"view": "timeSeries",
                                    "stacked": False,
                                    "metrics": [[ "CWAgent", "MEM_AVAILABLE", "InstanceId", instance_id ,"InstanceType", instance_type ],
                                                [ ".", "MEM_TOTAL", ".", ".", ".", "." ],
                                                [ ".", "MEM_USED", ".", ".", ".", "." ]
                                               ],
                                    "region": "eu-west-1",
                                    "title": instance_name + " Server - Memory"
                                    }
                    }
 
    widgets.append(memory_widget)
    widget_count += 1
 
def add_storage_widget():
    global widget_count
 
    storage_widget = {"type": "metric",
                      "x": x+6,
                      "y": y+3,
                      "width": width,
                      "height": height,
                      "properties": {"view": "timeSeries",
                                     "stacked": False,
                                     "metrics": [[ "CWAgent", "DISK_USED", "InstanceId", instance_id ,"InstanceType", instance_type ]
                                                ],
                                     "region": "eu-west-1",
                                     "title": instance_name + " Server - Storage"
                                    }
                     }
 
    widgets.append(storage_widget)
    widget_count += 1
 
def add_cpu_widget():
    global widget_count
 
    cpu_widget = {"type": "metric",
                  "x": x+12,
                  "y": y+3,
                  "width": width,
                  "height": height,
                  "properties": {"view": "timeSeries",
                                 "stacked": False,
                                 "metrics": [[ "CWAgent", "CPU_USAGE_IDLE", "InstanceId", instance_id ,"InstanceType", instance_type ],
                                             [ ".", "CPU_USAGE_IOWAIT", ".", ".", ".", "." ]
                                            ],
                                 "region": "eu-west-1",
                                 "title": instance_name + " Server - CPU Usage"
                         }
                 }
 
    widgets.append(cpu_widget)
    widget_count += 1
 
def add_process_widget():
    global widget_count
 
    process_widget = {"type": "metric",
                      "x": x+18,
                      "y": y+3,
                      "width": width,
                      "height": height,
                      "properties": {"view": "timeSeries",
                                     "stacked": False,
                                     "metrics": [[ "CWAgent", "PROCESSES_RUNNING", "InstanceId", instance_id ,"InstanceType", instance_type ],
                                                 [ ".", "PROCESSES_BLOCKED", ".", ".", ".", "." ],
                                                 [ ".", "PROCESSES_ZOMBIES", ".", ".", ".", "." ]
                                                ],
                                     "region": "eu-west-1",
                                     "title": instance_name + " Server - Processes"
                                    }
                    }
 
    widgets.append(process_widget)
    widget_count += 1
 
def add_diskio_widget():
    global widget_count
 
    diskio_widget = {"type": "metric",
                     "x": x,
                     "y": y+9,
                     "width": width,
                     "height": height,
                     "properties": {"view": "timeSeries",
                                    "stacked": False,
                                    "metrics": [[ "CWAgent", "DISKIO_READ_TIME", "InstanceId", instance_id ,"InstanceType", instance_type ],
                                                [ ".", "DISKIO_WRITE_TIME", ".", ".", ".", "." ]
                                               ],
                                    "region": "eu-west-1",
                                    "title": instance_name + " Server - DiskIO"
                                    }
                    }
 
    widgets.append(diskio_widget)
    widget_count += 1
 
def add_diskinodes_widget():
    global widget_count
 
    diskinodes_widget = {"type": "metric",
                         "x": x+6,
                         "y": y+9,
                         "width": width,
                         "height": height,
                         "properties": {"view": "timeSeries",
                                        "stacked": False,
                                        "metrics": [[ "CWAgent", "DISK_INODES_FREE", "InstanceId", instance_id ,"InstanceType", instance_type ],
                                                    [ ".", "DISK_INODES_TOTAL", ".", ".", ".", "." ],
                                                    [ ".", "DISK_INODES_USED", ".", ".", ".", "." ]
                                                   ],
                                        "region": "eu-west-1",
                                        "title": instance_name + " Server - Disk Inodes"
                                        }
                        }
 
    widgets.append(diskinodes_widget)
    widget_count += 1
 
def add_diskioinprogress_widget():
    global widget_count
 
    diskioinprogress_widget = {"type": "metric",
                               "x": x+12,
                               "y": y+9,
                               "width": width,
                               "height": height,
                               "properties": {"view": "timeSeries",
                                              "stacked": False,
                                              "metrics": [[ "CWAgent", "DISKIO_IOPS_IN_PROGRESS", "InstanceId", instance_id ,"InstanceType", instance_type ]
                                                         ],
                                              "region": "eu-west-1",
                                              "title": instance_name + " Server - DiskIO In Progress"
                                            }
                              }
 
    widgets.append(diskioinprogress_widget)
    widget_count += 1
 
def add_netstat_widget():
    global widget_count
 
    netstat_widget = {"type": "metric",
                      "x": x+18,
                      "y": y+9,
                      "width": width,
                      "height": height,
                      "properties": {"view": "timeSeries",
                                     "stacked": False,
                                     "metrics": [[ "CWAgent", "NETSTAT_TCP_SYN_SENT", "InstanceId", instance_id ,"InstanceType", instance_type ],
                                                 [ ".", "NETSTAT_TCP_LISTEN", ".", ".", ".", "." ],
                                                 [ ".", "NETSTAT_TCP_ESTABLISHED", ".", ".", ".", "." ]
                                                ],
                                     "region": "eu-west-1",
                                     "title": instance_name + " Server - Netstat TCP"
                                    }
                     }
 
    widgets.append(netstat_widget)
    widget_count += 1
 
def describe_instances():
    global instances
 
    print("Gathering instances from " + environment + " environment...")
 
    instances = ec2.describe_instances(
        Filters = [{'Name':'tag:Environment', 'Values':[environment]}
                  ])
 
    print("DONE!")
 
def create_or_update_dashboard(dn):
    global widgets
 
    body   = {'widgets' : widgets}
 
    body_j = json.dumps(body)
 
    cw.put_dashboard(DashboardName = dn,
                     DashboardBody = body_j)
 
    print("Dashboard " + dn + " is created!")
 
def list_dashboards(dn):
    return cw.list_dashboards(DashboardNamePrefix=dn)
 
def delete_dashboard(dn):
 
    for x in list_dashboards(dn)["DashboardEntries"]:
        print("Deleting old dashboard " + x["DashboardName"] + "...")
        cw.delete_dashboards(DashboardNames=[x["DashboardName"]])
        print("DONE!")
 
def set_dashboard_names():
    global environment, dashboard_name
 
    if environment == "Development":
        dashboard_name = environment + "_Nidavellir"
    elif environment == "Staging":
        dashboard_name = environment + "_Vanaheim"
    elif environment == "Tqa":
        dashboard_name = environment + "_Niflheimr"
    elif environment == "Production":
        dashboard_name = environment + "_Valhalla"
    elif environment == "Shared Tools":
        dashboard_name = "SharedTools" + "_Muspelheim"
    else:
        sys.exit("Please enter a valid environment(Development, Staging, Tqa, Production, \"Shared Tools\")")
#<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<#
#<<<<<<<<<<<<<<<<<<<<<< Function Definitions <<<<<<<<<<<<<<<<<<<<<<<#
#<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<#
 
set_dashboard_names()
delete_dashboard(dashboard_name)
describe_instances()
 
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>#
#>>>>>>>>>>>>>>>>>>>>>>>> Append Widgets >>>>>>>>>>>>>>>>>>>>>>>>>>>#
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>#
print("Creating/Updating dashboard for " + environment + "...")
for r in instances['Reservations']:
    for i in r['Instances']:
 
        extract_instance_info(i)
 
        add_descriptor_widget()
 
        add_memory_widget()
        add_storage_widget()
        add_cpu_widget()
        add_process_widget()
        add_diskio_widget()
        add_diskinodes_widget()
        add_diskioinprogress_widget()
        add_netstat_widget()
 
        if widget_count > widget_dashboard_limit:
            create_or_update_dashboard(dashboard_name + "-" + str(dashboard_count))
            dashboard_count += 1;
            widgets = []
            widget_count = 0
            y = 0
        else:
            y += y_coor_inc_per_ins
#<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<#
#<<<<<<<<<<<<<<<<<<<<<<<<< Append Widgets <<<<<<<<<<<<<<<<<<<<<<<<<<#
#<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<#
if dashboard_count == 1:
    create_or_update_dashboard(dashboard_name)
else:
    create_or_update_dashboard(dashboard_name + "-" + str(dashboard_count))
print("DONE!")

This Post Has 5 Comments

  1. thanks for the code, however, I am still a learner. Could you please help, I tried your code with one of my VM with CW installed, however it doesnot populates dashboard.

    1. Hello Neil,

      Could you please share more details? What is the output of the script? By CW installed, do you mean the agent?

      1. Hi , seems like widgnet gets created but no data is coming into the dashboards. so, basically the dashboards showing no data inside. Please help

        Thanks
        Ruchira

        1. Hello ruchira,

          You need to have some data points(metrics) to be shown on your dashboards. For example, MEM_AVAILABLE is a data point which you collect from your EC2 Instances by CloudWatch Agent. You can set them via AWS Console or apply my previous blog post to collect these metrics. Check it out: https://www.devopsful.com/2018/06/11/cloudwatch-agent-installation-to-ec2-instances-with-ansible/

          Also check this documentation: https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/CloudWatch-Dashboard-Body-Structure.html

          Hopefully, this helps.

          Have a good day!

  2. Hi
    I have install cloudwatch agent on my ec2 linux instance and the python script that you have provide also execuetd but data is not populated in dashborad . I have also modiy the script as per below
    Can you plese help me

    [ “CWAgent”, “CPU_USAGE_IDLE”, “InstanceId”, instance_id ,”InstanceType”, instance_type ],
    [ “.”, “CPU_USAGE_IOWAIT”, “.”, “.”, “.”, “.” ]

Leave a Reply

Close Menu