Unified_CloudWatch_Agent_AWS_Parameter_Store_JSON_Config

Deploying Custom CloudWatch Metrics via SSM Unified CloudWatch Agent

In order to configure Windows and Linux EC2 instances to send custom CloudWatch metrics by using SSM, we need to use the new Unified CloudWatch Agent.

There is an old version of SSM Agent or EC2Config, but this was available only on Windows Instances.

The new CloudWatch Agent, integrated with AWS Systems Manager (SSM) for simplified deployment and management, unifies collecting multi-platform metrics and logs into one agent and enhances the observability of your EC2 instances and virtual machines by collecting in-guest system metrics.

Amazon CloudWatch Agent allow us to send Logs from EC2 instance to CloudWatch Logs by creating a log group. Each instance under that Log Group acts like a stream, so if you decide to search the Log Group, you just need to search by providing an instance ID, and if you have configured it properly, you will see the logs.

Update: Drawbacks of sending Event Logs to CloudWatch Console

Even though this can be very useful, CloudWatch Agent cannot filter Event Logs. This means that CloudWatch Agent actually sends all the Event Logs to CloudWatch Console.

AWS recently released Kinesis Tap Agent for Windows OS. Kinesis Agent for Windows OS is able to filter out only those Event IDs you want to be sent to CloudWatch Log Group.

Let’s say that you have a critical application running on your EC2 instance. You would like to be alerted when the application stops (service goes into stopped state, thus generating Event ID 45). You can configure Kinesis Tap Agent to monitor Event Viewer for this Event ID, and once it occurs, it will send this Event to CloudWatch Log Group, where you will have a filter pattern configured to execute Lambda Function when this Event shows up in Log Group. Lambda Function can then take the info from the Event, pack it nicely into a message, and send it to your ticketing system via SNS.

But let’s get back to CloudWatch Agent now. The CloudWatch Agent has the following capabilities:

  • Simplified agent — only a single agent is used to collect metrics and logs, simplifying installation and collection.
  • Cross-platform — we may install the agent on either 64-bit Linux or Windows. It also includes HTTP proxy server support.
  • Configurable — agent automatically tracks useful system metrics, but may be modified to collect others such as CPU threads, mounted filesystems, and network interfaces.
  • CloudWatch — supports the normal 1-minute metrics and the new 1-second high-resolution metrics option. Includes EC2 dimensions like Instance ID, Image ID, and Autoscaling Group Name, as well as custom dimensions.
  • Migration — We may migrate existing AWS SSM and EC2Config settings for the new agent.

Before we start with installation of Unified CloudWatch Agent, first we need to create a role and assign couple of policies.

  1. go to IAM console
  2. Create a role and assign the following policies:

Unified_CloudWatch_Agent_Create_Role

If you already have SSM Agent installed on your Windows instances, you can push installation of the new Agent quite easily.
On an Amazon EC2 instance, the CloudWatch agent requires that the instance is running version 2.2.93.0 or later. Before you install the CloudWatch agent, update or install the SSM Agent on the instance if you haven’t already done so, as you cannot push the installation of CloudWatch Agent if you can’t see your instance under Managed Instances.

After you have everything set, follow these steps to install Amazon CloudWatch Agent:

  1. Connect to Systems Manager console
  2. Select the ‘Run Command’ section
  3. Search for AWS-ConfigureAWSPackage document
    Unified_CloudWatch_Agent_AWS_ConfigureAWSPackage
  4. Click on the document
  5. Under Action select ‘Install
  6. Under Name, paste AmazonCloudWatchAgent as a name of the application
    Unified_CloudWatch_Agent_Command_Parameters
  7. Under version, leave Latest
  8. Specify instances on which you want to installupdate CloudWatch Agent
  9. Click Run

NOTE: The instance will not be rebooted!!!

The output should look like this:

Unified_CloudWatch_Agent_AWS_ConfigureAWSPackage_Success

Now when we have installed the CloudWatch Agent, we can configure the CloudWatch Agent with the configuration we have defined.

The configuration file is a JSON file, which we will store in the Parameter Store section.

This JSON file you can take from any of your server, where you have set manually your metrics, and those which you would like to propagate to all other instances.

To store the JSON configuration file to Parameter Store, follow these steps:

  1. Connect to Systems Manager Console
  2. Select the ‘Parameter Store’ section
  3. Click on ‘Create parameter’ button
  4. Under name, make sure you name the parameter store properly, as you will have to use this name later during the setup of CloudWatch Agent
  5. Leave the type ‘String’
  6. In the Value field, paste the content of the JSON configuration file
  7. Click on ‘Create Parameter’ buttonUnified_CloudWatch_Agent_AWS_Parameter_Store_JSON_Config

The Parameter has been created.

Unified_CloudWatch_Agent_AWS_Parameter_Store_JSON_Config_Created

Now, we are ready to configure our Unified CloudWatch Agent.

To configure the CloudWatch Agent, follow these steps:

  1. Connect to Systems Manager Console
  2. Select the ‘Run Command’ section
  3. Click on ‘Run Command’ button
  4. Search for AmazonCloudWatch-ManageAgent document
    Unified_CloudWatch_Agent_AWS_AmazonCloudWatch-ManageAgent
  5. Under Document Version, select the latest version
  6. Under Command Parameters section, leave everything as Default except for the ‘Optional Configuration Location’, where you paste the name of the Parameter, which you created and stored in Parameter Store.
    Unified_CloudWatch_Agent_Optional_Configuration_LocationNOTE: Make sure you write down exact name of the Parameter, otherwise it won’t work.
  7. Specify the instances on which you want to configure the CloudWatch agent
  8. Run the command

NOTE:  The configuration of the CloudWatch Agent doesn’t require the restart of the instance, only of an installed agent.

To check if the metrics are really there, follow these steps:

  1. Connect to CloudWatch console
  2. Navigate to Metrics section
  3. Under All Metrics, look for CWAgent custom metrics

Unified_CloudWatch_Agent_CloudWatch_Console_Metrics

The same process could be done for Linux instances as well.

Roll Back to Log Collection with SSM Agent  (weird as it looks like it is only for Windows)

If you want to return to using SSM Agent for log collection, follow these steps.

One: Retrieve Config Data from SSM Agent 

On the instance where you want to return to collecting logs with the SSM Agent, locate the contents of the SSM Agent config file.

This JSON file is typically found in the following location:

${Env:ProgramFiles}AmazonSSMPluginsawsCloudWatchAWS.EC2.Windows.CloudWatch.json

Copy this data into a text file for use in a later step.

Two: Uninstall the CloudWatch Agent 

  1. Open the AWS Systems console
  2. In the navigation pane, choose Run Command, and then choose Run command.
  3. In the Command document list, choose AWS-ConfigureAWSPackage.
  4. In the Targets section, choose an option and select the instances to update.
  5. In the Action list, choose Uninstall.
  6. In Name, type AmazonCloudWatchAgent.
  7. Choose Run.

Three: Re-enable Log Collection in SSM Agent 

  1. Open the AWS Systems Manager console
  2. In the navigation pane, choose Run Command, and then choose Run command.
  3. In the Command document list, choose AWS-ConfigureCloudWatch
  4. In the Targets section, choose an option and select the instances to update.
  5. In the Status list, choose Enabled.
  6. In the Properties box, paste the contents of the old config data you saved to the text file.
  7. Choose Run.

1 thought on “Deploying Custom CloudWatch Metrics via SSM Unified CloudWatch Agent”

  1. Pingback: Configure Kinesis Agent for Windows – Installation – Vlad's IT Blog

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Close Bitnami banner
Bitnami