Monitoring Web Endpoints with Canaries
Posted on Tue 02 May 2023 in AWS
Ensuring a positive user experience with your web endpoints is crucial for maintaining a successful online presence. A key strategy for achieving this is to continuously verify the user experience and identify any issues before they impact your users.
Historically, miners would bring canaries into the mines to detect dangerous gases and provide early warning signs of potential danger. Similarly, Amazon CloudWatch Synthetics allows you to create virtual "canaries" that simulate your customers' actions and detect issues before they cause significant problems. By following the same routes and performing the same actions as your customers, these virtual canaries provide you with an early warning sign to address any potential issues, preventing any negative impact on your website's performance.
In this post, we will go through a simple Synthetics canary script that I created, and deploy it via CloudFormation. By implementing this solution, you can ensure that your website provides a smooth user experience, leading to customer satisfaction.
The required roles and permissions
First, we need to deal with the required roles and permissions.
As the user who will be managing CloudWatch canaries, you must be signed in as an IAM user with the following roles and permissions:
- To create canaries:
- CloudWatchSyntheticsFullAccess
- To view canary details and the results of canary runs:
- CloudWatchSyntheticsFullAccess, or
- CloudWatchSyntheticsReadOnlyAccess
- To read all Synthetics data in the console:
- AmazonS3ReadOnlyAccess
- CloudWatchReadOnlyAccess
- To view the source code used by canaries:
- AWSLambda_ReadOnlyAccess
- CloudWatchSyntheticsFullAccessTo create canaries:
- CloudWatchSyntheticsFullAccess
- To view canary details and the results of canary runs:
- CloudWatchSyntheticsFullAccess, or
- CloudWatchSyntheticsReadOnlyAccess
- To read all Synthetics data in the console:
- AmazonS3ReadOnlyAccess
- CloudWatchReadOnlyAccess
- To view the source code used by canaries:
- AWSLambda_ReadOnlyAccess
- CloudWatchSyntheticsFullAccess
Now we need to create an IAM role that will be associated to each canary. During canary creation, you will have the option to have one automatically created for you. This role will have the needed permissions. However, if you decide to create the IAM role manually, ensure that the trust policy statement below is included.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Note: You will also need to include the needed permissions for the other AWS services that your canary will use. We will see an example of which in the CloudFormation template section.
The Canary Script
var synthetics = require('Synthetics'); // Synthetics dependency
const log = require('SyntheticsLogger');
const syntheticsLogHelper = require('SyntheticsLogHelper');
const loadWebpage = async function () {
let page = await synthetics.getPage();
await synthetics.executeStep('navigateToURL', async function () {
let URL = process.env.URL;
const sanitizedUrl = syntheticsLogHelper.getSanitizedUrl(process.env.URL);
// Navigate to the website/URL
const response = await page.goto(URL, { waitUntil: 'domcontentloaded', timeout: 30000});
if (response) {
const status = response.status();
const statusText = response.statusText();
logResponseString = `Response from url: ${sanitizedUrl} Status: ${status} Status Text: ${statusText}`;
//If the response status code is not a 2xx success code
if (response.status() < 200 || response.status() > 299) {
throw new Error(`Failed to load url: ${sanitizedUrl} ${response.status()} ${response.statusText()}`);
}
} else {
const logNoResponseString = `No response returned for url: ${sanitizedUrl}`;
log.error(logNoResponseString);
throw new Error(logNoResponseString);
}
});
// Wait for 15 seconds to let page load fully before executeStep ends and takes a screenshot
await page.waitFor(15000);
};
exports.handler = async () => {
return await loadWebpage();
};
The script above is a Puppeteer script that was modified to run as a Synthetics canary script. For more information about Puppeteer, see Puppeteer API v1.14.0. To go over quickly on how to convert a Puppeteer script into a Synthetics canary script:
- Create and export a handler function. This will be the entry point function for the script.
- Use the Synthetics dependency.
- Use the Synthetics.getPage function to get a Puppeteer Page object.
For more details on the conversion, see the AWS documentation.
The script basically checks if a website is available. It’ll try to navigate to the specified URL, and log some details when successful, or throws an error when it’s not.
The CloudFormation Template
Before we deploy the template, we need to do two things manually for the deployment to be successful.
- Compress our Synthetics canary script into a zip file with a folder structure: nodejs/node_modules/canary_script_name.js.
- Upload the zip file into an S3 Bucket where CloudWatch can retrieve it for the canary creation.
Below is our CloudFormation Template:
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Deploys CloudWatch Synthetics Canaries"
Parameters:
CanaryS3BucketName:
Description: S3 Bucket containing the canary scripts as well as for storing canary artifacts.
Type: String
BaseURL:
Description: Base URL of the application to monitor in the canary script.
Type: String
Resources:
SyntheticsLambdaExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: ""
Effect: "Allow"
Principal:
Service: "lambda.amazonaws.com"
Action: "sts:AssumeRole"
SyntheticsLambdaExecutionRolePolicies:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: "SyntheticsLambdaExecutionRolePolicy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "s3:GetBucketLocation"
- "s3:ListBucket"
Resource:
!Sub "arn:aws:s3:::${CanaryS3BucketName}"
Effect: "Allow"
- Action:
- "s3:PutObject"
- "s3:GetObject"
Resource:
!Sub "arn:aws:s3:::${CanaryS3BucketName}/*"
Effect: "Allow"
- Action:
- "cloudwatch:PutMetricData"
Resource: "*"
Effect: "Allow"
- Action:
- "logs:*"
Resource:
Fn::Sub: "arn:${AWS::Partition}:logs:*:*:*"
Effect: "Allow"
- Action:
- "s3:ListAllMyBuckets"
Resource: "*"
Effect: "Allow"
- Action:
- "ssm:GetParameter"
Resource: "*"
Resource: !Sub "arn:aws:ssm:us-west-2:458742683897:parameter/synthetics/marz/"
Effect: "Allow"
Roles:
- !Ref "SyntheticsLambdaExecutionRole"
CheckWebsiteAvailabilityCanary:
Type: AWS::Synthetics::Canary
DependsOn:
- SyntheticsLambdaExecutionRole
- SyntheticsLambdaExecutionRolePolicies
Properties:
Name: check-availability
ExecutionRoleArn:
Fn::GetAtt: SyntheticsLambdaExecutionRole.Arn
Code:
Handler: "check-availability.handler"
S3Bucket: !Ref "CanaryS3BucketName"
S3Key: "canary-scripts/check-availability.zip"
RunConfig:
EnvironmentVariables:
URL : !Ref "BaseURL"
RuntimeVersion: syn-nodejs-puppeteer-3.8
ArtifactS3Location:
!Sub "s3://${CanaryS3BucketName}"
FailureRetentionPeriod: 30
SuccessRetentionPeriod: 30
StartCanaryAfterCreation: true
Schedule: {Expression: 'rate(5 minutes)', DurationInSeconds: '3600'}
Creating and running a CloudFormation stack using the template above will create all the necessary AWS resources, as well as the Syncthetic Canary. In the AWS Console, you can navigate over to CloudWatch > Application Monitoring > Synthetics Canaries to see it at work.
