Skip to main content

AWS ASG

note

Currently, ASG deployments are behind the feature flag CDS_ASG_NG. Contact Harness Support to enable the feature.

This topic explains how to deploy new AWS Auto Scale Groups (ASGs) and instances to Amazon Elastic Compute Cloud (EC2) using Harness.

Deployment summary​

Here's a high-level summary of the setup steps and how Harness deploys ASGs.

Harness setup summary

Here's a summary of how you set up ASG deployments in Harness:

  1. Create the Harness ASG service.
    1. Add launch template.
    2. Add ASG configuration.
    3. Add scaling policies (optional).
    4. Add scheduled update group action (optional).
    5. Add the AMI image to use for the ASG as an artifact.
  2. Create the Harness ASG environment.
    1. Connect Harness to the AWS region where you want to deploy.
  3. Define the Harness ASG pipeline execution.
    1. Select a deployment strategy (rolling, canary, blue green) and Harness automatically creates the steps to deploy the new ASG.
  4. Deploy the Harness pipeline.
Deployment summary

Here's a summary of how Harness deploys new ASG versions:

  1. First deployment:
    1. Harness takes the launch template and ASG configuration files you provide and creates a new ASG and its instances in your AWS account and region.
  2. Subsequent deployments:
    1. Harness creates a new version of the launch template.
    2. Harness uses the new version of the launch template to update the ASG. For example, if you increased the desired capacity (desiredCapacity) for the ASG in your ASG configuration file, Harness will create a new version of the ASG with the new desired capacity.
    3. Instance refresh is triggered (a rolling replacement of all or some instances in the ASG).

Notes:

  • ASG creation differs for rolling and blue green deployments:
    • For rolling, Harness updates the existing ASG with the new configuration.
    • For blue green, Harness creates a new ASG with a new revision suffix, like asg-demo__2. Once there are two ASGs (asg-demo__1 and asg-demo__2) Harness alternately updates these existing ASGs with the new configuration on each successive deployment.

AWS policy requirements​

When you set up a Harness AWS connector to connect Harness with your AWS account, the AWS IAM role must have the following policies.

AmazonEC2FullAccess
  • Policy Name: AmazonEC2FullAccess.
  • Policy ARN: arn:aws:iam::aws:policy/AmazonEC2FullAccess.
  • Description: Provides full access to Amazon EC2 via the AWS Management Console.

Policy JSON:

{  
  "Version": "2012-10-17",  
  "Statement": [  
    {  
      "Action": "ec2:*",  
      "Effect": "Allow",  
      "Resource": "*"  
    },  
    {  
      "Effect": "Allow",  
      "Action": "elasticloadbalancing:*",  
      "Resource": "*"  
    },  
    {  
      "Effect": "Allow",  
      "Action": "cloudwatch:*",  
      "Resource": "*"  
    },  
    {  
      "Effect": "Allow",  
      "Action": "autoscaling:*",  
      "Resource": "*"  
    }  
  ]  
}
DescribeRegions
note

The DescribeRegions action is required for all AWS IAM roles used in Harness AWS connectors regardless of what AWS service you are using.

Harness needs a policy with the DescribeRegions action so that it can list the available regions for you when you define your target architecture.

Create a Customer Managed Policy, add the DescribeRegions action to list those regions, and add that to any role used by the Cloud Provider.

{  
    "Version": "2012-10-17",  
    "Statement": [  
        {  
            "Sid": "VisualEditor0",  
            "Effect": "Allow",  
            "Action": "ec2:DescribeRegions",  
            "Resource": "*"  
        }  
    ]  
}

Harness ASG services​

The Harness ASG service contains the following:

  • Launch template.
  • ASG configuration file.
  • Scaling policies (optional).
  • Scheduled update group action (optional).
  • The AMI image to use for the ASG.

Harness supports standard ASG JSON formatted files. For more information, go to Get started with Amazon EC2 Auto Scaling from AWS.

You can use remote files stored in a Git repo or the Harness File Store.

note

You can create a service when you are building the pipeline or separately in Services. In this topic, we walk through building the service separately. For more information, go to CD pipeline modeling overview.

Here's an example where the configuration files are stored in the Harness File Store.

service:
  name: svc-asg
  identifier: svcasg
  serviceDefinition:
    type: Asg
    spec:
      manifests:
        - manifest:
            identifier: launchTemplate
            type: AsgLaunchTemplate
            spec:
              store:
                type: Harness
                spec:
                  files:
                    - /asg/launchtemplate
        - manifest:
            identifier: launchConfig
            type: AsgConfiguration
            spec:
              store:
                type: Harness
                spec:
                  files:
                    - /asg/launchconfig
        - manifest:
            identifier: scalePolicy
            type: AsgScalingPolicy
            spec:
              store:
                type: Harness
                spec:
                  files:
                    - /asg/scalingPolicy.json
        - manifest:
            identifier: scheduledUpdateGroupAction
            type: AsgScheduledUpdateGroupAction
            spec:
              store:
                type: Harness
                spec:
                  files:
                    - /asg/scheduledUpdateGroupAction.json
      artifacts:
        primary:
          primaryArtifactRef: <+input>
          sources:
            - identifier: AMI-ARTIFACT
              spec:
                connectorRef: AWS_ASG_CONNECTOR
                region: us-east-1
                filters:
                  - name: ami-name
                    value: Returns local IP address at port 80
                version: Returns local IP address at port 80
              type: AmazonMachineImage
  gitOpsEnabled: false

ASG configuration files​

AWS does not have a dedicated public resource for ASG configuration file formatting because ASG creation is typically done using the AWS CLI, SDKs, or Management Console, which have their own syntax and methods for specifying the parameters.

However, the AWS CLI create-auto-scaling-group command reference documentation provides a detailed description of the parameters that can be used when creating an ASG.

ASG configuration file example
{
  "AutoScalingGroupName": "my-asg",
  "MinSize": 1,
  "MaxSize": 3,
  "DesiredCapacity": 2,
  "AvailabilityZones": ["us-west-2a", "us-west-2b"],
  "LaunchTemplate": {
    "LaunchTemplateId": "lt-0123456789abcdef0",
    "Version": "1"
  },
  "VPCZoneIdentifier": "subnet-0123456789abcdef0,subnet-0123456789abcdef1",
  "HealthCheckType": "EC2",
  "HealthCheckGracePeriod": 300,
  "Tags": [
    {
      "Key": "Environment",
      "Value": "Development",
      "PropagateAtLaunch": true
    }
  ]
}

AMI artifacts​

Review the following information about the AMI artifacts you select in your ASG service.

Only private AMIs are supported​

When you add an AMI artifact you must use a private AMI. Public AMIs from the AMI Catalog are not supported at this time.

In the AWS console, in EC2, private AMIs are listed in Images > AMIs.

Artifact AMI overrides config files AMIs​

You can specify an AMI in the launch template used in the Harness service, but it will be overridden by the AMI specified in the Harness service Artifacts section.

The AMI specified in the Harness service Artifacts section is the same as selecting an AMI in the Create launch template wizard in the AWS console or ImageId in in the create-launch-template CLI command.

Using service variables in config file​

Service variables are a powerful way to template your services or make them more dynamic.

In the Variables section of the service, you can add service variables and then reference them in any of the ASG configuration files you added to the service.

For example, you could create a variable named desiredCapacity and set its value as a fixed value, runtime input, or expression.

service variable

Next, in your ASG configuration file, you could reference the variable like this (see <+serviceVariables.desiredCapacity>):

{
  "autoScalingGroupName": "lovish2-asg",
  "minSize": 1,
  "maxSize": 3,
  "desiredCapacity": <+serviceVariables.desiredCapacity>,
  "tags": [{"key": "key7", "value":  "value7"}, {"key": "key2", "value":  "value2"}],
  "availabilityZones": ["us-east-1a", "us-east-1b"]
}

Now, when you add that service to a pipeline, you will be prompted to enter a value for this variable in the pipeline Services tab. The value you provide is then used as the desiredCapacity in your ASG configuration.

Harness ASG environments​

The Harness ASG environment is where you specify production and non-production environment settings.

You also add a infrastructure definition to define the region where you want to deploy the ASG.

note

You can create an environment when you are building the pipeline or separately in Environments. In this topic, we walk through building the environment separately. For more information, go to CD pipeline modeling overview.

Here's a YAML example of an ASG environment.

environment:
  name: ASG
  identifier: ASG
  description: ""
  tags: {}
  type: Production
  orgIdentifier: default
  projectIdentifier: myProject
  variables: []

Infrastructure definition​

To define the target ASG region, you add an infrastructure definition to a Harness environment. The infrastructure definition uses a Harness AWS connector and a region setting to define the deployment target.

You can use the same AWS connector you used when adding the AMI artifact in the Harness service. Ensure the AWS IAM user in the AWS connector credentials meets the AWS policy requirements.

Here's a YAML example of an ASG infrastructure definition.

infrastructureDefinition:
  name: asg-infra
  identifier: asginfra
  description: ""
  tags: {}
  orgIdentifier: default
  projectIdentifier: DoNotDelete_IvanBalan
  environmentRef: ASG
  deploymentType: Asg
  type: Asg
  spec:
    connectorRef: AWS_ASG_CONNECTOR
    region: us-east-1
  allowSimultaneousDeployments: false

Harness ASG pipelines​

Once you have a the service and environment created, you can create the pipeline.

note

You can create a service and environment when you are building the pipeline or separately in Services and Environments. In this topic, we walk through building these separately. For more information, go to CD pipeline modeling overview.

The pipeline models the release process using execution steps, triggers, and other settings. For more information, go to CD pipeline modeling overview.

Here's a pipeline with a service, environment, and ASG Rolling Deploy and ASG Rolling Rollback steps:

pipeline:
  projectIdentifier: CD_Docs
  orgIdentifier: default
  tags: {}
  stages:
    - stage:
        name: DeployRolling
        identifier: Deploy
        description: ""
        type: Deployment
        spec:
          deploymentType: Asg
          service:
            serviceRef: svcasg
            serviceInputs:
              serviceDefinition:
                type: Asg
                spec:
                  artifacts:
                    primary:
                      primaryArtifactRef: <+input>
                      sources: <+input>
                  variables:
                    - name: desiredCapacity
                      type: String
                      value: <+input>
          environment:
            environmentRef: ASG
            deployToAll: false
            infrastructureDefinitions:
              - identifier: asginfra
          execution:
            steps:
              - step:
                  name: ASG Rolling Deploy
                  identifier: AsgRollingDeploy
                  type: AsgRollingDeploy
                  timeout: 10m
                  spec:
                    useAlreadyRunningInstances: false
                    skipMatching: true
            rollbackSteps:
              - step:
                  type: AsgRollingRollback
                  name: ASG Rolling Rollback
                  identifier: ASG_Rolling_Rollback
                  spec: {}
                  timeout: 10m
        tags: {}
        failureStrategies:
          - onFailure:
              errors:
                - AllErrors
              action:
                type: StageRollback
  identifier: ASGROLLING
  name: ASG-ROLLING

ASG pipeline execution strategies​

Harness ASG deployments support the following deployment strategies:

  • Rolling
  • Canary
  • Blue Green

Rolling​

The first ASG rolling deployment will deploy a new ASG with the number of desired instances you have defined in your ASG configuration file in the Harness service used in the pipeline. Any other configuration files in the service are also applied.

Subsequent deployments will deploy a new version of the same ASG with any changes you have made in the Harness service configuration files (launch template, ASG configuration, etc.). This is the same as using the Edit buttons in the AWS console and creating a new version of the ASG.

Here's a flowchart that explains how Harness performs rolling deployments:

Rolling deployments flowchart

ASG rolling flowchart

The Rolling Deploy step has the following options:

  • Same as already running Instances
    • Enable this setting to use the scaling settings on the last ASG version deployed.
  • Minimum Healthy Percentage (optional)
    • The percentage of the desired capacity of the ASG that must pass the group's health checks before the refresh can continue. For more information about these health checks, go to Health checks for Auto Scaling instances from AWS.
  • Instance Warmup (optional)
  • Skip Matching
    • Choose whether AWS Auto Scaling skips replacing instances that match the desired configuration. If no desired configuration is specified, then it skips replacing instances that have the same launch template and instance types that the ASG was using before the instance refresh started. For more information, go to Use an instance refresh with skip matching from AWS.
YAML example
          execution:
            steps:
              - step:
                  name: ASG Rolling Deploy
                  identifier: AsgRollingDeploy
                  type: AsgRollingDeploy
                  timeout: 10m
                  spec:
                    useAlreadyRunningInstances: false
                    skipMatching: true

Deployment logs​

Here's an example of the log from a successful deployment.

ASG Rolling Deploy log example
Deploy

Starting Rolling Deployment
Creating new version for launchTemplate demo-asg2
Created new version for launchTemplate demo-asg2
Getting Asg demo-asg2
Creating Asg demo-asg2
Waiting for Asg demo-asg2 to reach steady state
Polling every 20 seconds
Getting Asg demo-asg2
Getting Asg demo-asg2
0/2 instances are healthy
Getting Asg demo-asg2
2/2 instances are healthy
Created Asg demo-asg2 successfully
Getting Asg demo-asg2
Modifying scaling policies of Asg demo-asg2
Getting ScalingPolicies for Asg demo-asg2
No policies found which are currently attached with autoscaling group: [demo-asg2] to detach
No scaling policy provided which is needed be attached to autoscaling group: demo-asg2
Modified scaling policies of Asg demo-asg2 successfully
Modifying scheduled actions of autoscaling group demo-asg2
Getting ScheduledUpdateGroupActions for Asg demo-asg2
No scheduled actions found which are currently attached with autoscaling group: [demo-asg2] to detach
Attached scheduled action with name: scheduledActionName1
Modified scheduled actions of autoscaling group demo-asg2 successfully
Starting Instance Refresh for Asg demo-asg2
Waiting for Instance Refresh for Asg demo-asg2 to reach steady state
Polling every 20 seconds
Percentage completed: 0
Waiting for remaining instances to be available. For example: i-03e47024515f6d45a has insufficient data to evaluate its health with Amazon EC2.
Percentage completed: 0
Waiting for remaining instances to be available. For example: i-03e47024515f6d45a has insufficient data to evaluate its health with Amazon EC2.
Percentage completed: 0
Waiting for remaining instances to be available. For example: i-03e47024515f6d45a has insufficient data to evaluate its health with Amazon EC2.
Percentage completed: 0
Waiting for remaining instances to be available. For example: i-03e47024515f6d45a has insufficient data to evaluate its health with Amazon EC2.
Percentage completed: 100
Instance Refresh for Asg demo-asg2 ended successfully
Rolling Deployment Finished Successfully

Canary​

The ASG canary deployment uses two step groups:

  1. Canary Deployment:
    1. ASG Canary Deploy step: deploys a new ASG version with the name of your ASG and the extension __Canary. The canary ASG version can use the desiredCapacity in your ASG configuration file or you can set it in the Instances setting in the ASG Canary Deploy step.
    2. ASG Canary Delete step: deletes the canary ASG.
  2. Primary Deployment:
    1. ASG Rolling Deploy step:
      1. The first deployment will deploy a new ASG with the number of desired instances you have defined in your ASG configuration file in the Harness service used in the pipeline. Any other configuration files in the service are also applied.
      2. Subsequent deployments will deploy a new version of the same ASG with any changes you have made in the Harness service configuration files (launch template, ASG configuration, etc.). This is the same as using the Edit buttons in the AWS console and creating a new version of the ASG.

Here's what the two step groups look like:

ASG canary step groups

The ASG Canary Deploy step deploys a new ASG with the name of your ASG and the extension __Canary.

In the ASG Canary Deploy step, in Instances, you can specify how many instances to use in this temporary ASG.

The Instances replaces the desiredCapacity in your ASG configuration file.

ASG Canary Deploy step log example
Getting Asg demo-asg2__Canary
Creating launchTemplate demo-asg2__Canary
Created launchTemplate demo-asg2__Canary successfully
Getting Asg demo-asg2__Canary
Creating Asg demo-asg2__Canary
Waiting for Asg demo-asg2__Canary to reach steady state
Polling every 20 seconds
Getting Asg demo-asg2__Canary
Getting Asg demo-asg2__Canary
0/1 instances are healthy
Getting Asg demo-asg2__Canary
1/1 instances are healthy
Created Asg demo-asg2__Canary successfully
Getting Asg demo-asg2__Canary
Deployment Finished Successfully

Blue Green​

An ASG Blue Green deployment uses two steps:

  1. ASG Blue Green Deploy step:
    1. You specify the production and stage listener ports and rules to use.
    2. The new ASG is deployed to the stage target group.
  2. ASG Blue Green Swap Services step:
    1. Harness modifies the ELB prod listener to forward requests to target groups associated with each new ASG version.
    2. Harness swaps all traffic from stage to production listeners and ports.
    3. Harness modifies the ELB stage listener to forward requests to target groups associated with old ASG version.

The first ASG deployed is given a suffix using the format [app_name]__1, like asgdemo__1. The second ASG deployed is given the suffix [app_name]__2.

Every subsequent deployment will simply create new versions of these ASGs instead of creating new ASGs. So the third deployment will create a new version of ASG [app_name]__1, route prod traffic to it, and route stage traffic to ASG [app_name]__2.

Blue/Green with Traffic Shift Summary

In this strategy, you specify production and stage listener ports and rules to use in the ASG Blue Green Deploy step. Next, the ASG Swap Services step swaps all traffic from stage to production.

A Blue/Green deployment reliably deploys your ASGs by maintaining new and old versions of ASGs. The ASGs run behind an Application Load Balancer (ALB) using two listeners, stage and prod. These listeners forward respectively to two target groups, stage and prod, where the new and old ASGs are run.

In the first stage of deployment, the new ASG is attached to the stage target group:

first stage

Blue/Green deployments are achieved by swapping routes between the target groups—always attaching the new ASG first to the stage target group, and then to the prod target group:

second stage

Here's a flowchart that explains how Harness performs Blue Green deployments:

Blue Green deployments flowchart

blue green flowchart map

AWS requirements

In addition to the requirements of the Harness ASG service and environment, an ASG Blue Green deployment requires you to set up the following resources up within AWS:

  • A security group that allows inbound access to your Application Load Balancer's listener ports.
  • A pair of target groups—typically staging (Stage) and production (Prod)—both with the instance target type.
  • An Application Load Balancer (ALB), with listeners for both your target groups' ports.

Let's take a look at the first two deployments.

First Blue Green deployment

The first Blue Green deployment follows these steps:

  1. ASG Blue Green Deploy step:
    1. Checks that listener ARN and listener rule ARN for prod are valid.
    2. Fetches target group ARNs for prod.
    3. Checks that listener ARN and listener rule ARN for stage are valid.
    4. Fetches target group ARNs for stage.
    5. Creates launch template and ASG using the format [app_name]__1, like asgdemo__1.
    6. Sets the tag BG_VERSION=BLUE on new ASG.
  2. ASG Blue Green Swap step:
    1. Not used as there is only one ASG at this point.
Second Blue Green deployment

At the start, the second Blue Green deployment will only have one prod ASG (the one deployed in the first Blue Green deployment).

Harness will create a new ASG with the suffix __2 and swap prod traffic to it.

The second Blue Green deployment follows these steps:

  1. ASG Blue Green Deploy step:
    1. Checks that listener ARN and listener rule ARN for prod are valid.
    2. Fetches target group ARNs for prod.
    3. Checks that listener ARN and listener rule ARN for stage are valid.
    4. Fetches target group ARNs for stage.
    5. Creates launch template and new ASG using the format [app_name]__2, like asgdemo__2.
    6. Sets the tag BG_VERSION=GREEN on new ASG, for example, asgdemo__2.
  2. ASG Blue Green Swap step swaps target groups and updates tags:
    1. Modifies the ELB prod listener to forward requests to target groups associated with new autoscaling group.
    2. Modifies the ELB stage listener to forward requests to target groups associated with the old autoscaling group.
    3. Updates tags of the autoscaling groups.

Now there are two ASGs being used: a prod and a stage ASG. Subsequent deployments will create a new versions of these ASGs and swap prod traffic to the new ASG version from the previous ASG.

Blue Green deployment steps:

The ASG Blue Green Deploy step has the following settings:

  • Load Balancer: select the load balancer to use.
  • Prod Listener: select the listener to use for prod traffic.
  • Prod Listener Rule ARN: select the ARN for the prod listener rule.
  • Stage Listener: select the listener to use for stage traffic.
  • Stage Listener Rule ARN: select the ARN for the stage listener rule.

Harness fetches these AWS settings using the Harness AWS connector you have set up in the Infrastructure Definition in the Environment section of the stage.

Advanced settings for all steps​

In the Advanced settings of all step, you can use the following options:

Notes​

AWS has the following limitations to keep in mind:

  • You are limited to creating 5000 launch templates per region and 10000 versions per launch template. For more information, go to Launch an instance from a launch template from AWS.
  • ASGs per region: 500.
  • Launch configurations per region: 200.