ALB AZ down

Introduction

ALB (Application Load Balancer) AZ (Availability Zones) down takes down the AZ on a target application load balancer for a specific duration thereby impacting the delivery. This fault restricts access to certain availability zones for a specific duration.

Use cases

ALB AZ down:

• Detaches the AZ from the application load balancer thereby disrupting the application's performance.
• Tests the application sanity, availability, and recovery workflows of the application pod attached to the load balancer.

note

• Kubernetes version 1.17 or later is required to execute this fault.
• Appropriate AWS access to attach or detach subnet from the target ALB.
• There needs to be a minimum of two AZs attached to the target ALB after injecting chaos; otherwise, the fault will fail to detach the given AZ.
• The ALB should be attached to the given availability zones.
  • The Kubernetes secret should have AWS access configuration (key) in the CHAOS_NAMESPACE. Below is a sample secret file.

  apiVersion: v1
  kind: Secret
  metadata:
    name: cloud-secret
  type: Opaque
  stringData:
    cloud_config.yml: |-
      # Add the cloud AWS credentials respectively
      [default]
      aws_access_key_id = XXXXXXXXXXXXXXXXXXX
      aws_secret_access_key = XXXXXXXXXXXXXXX
• We recommend that you use the same secret name, that is, cloud-secret. Otherwise, you will need to update the AWS_SHARED_CREDENTIALS_FILE environment variable in the fault template with the new secret name but you won't be able to use the default health check probes.
• Go to AWS named profile for chaos to use a different profile for AWS faults and superset permission or policy to execute all AWS faults.

Below is an example AWS policy to execute the fault.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInstances",
                "ec2:DescribeSubnets",
                "elasticloadbalancing:DetachLoadBalancerFromSubnets",
                "elasticloadbalancing:AttachLoadBalancerToSubnets",
                "elasticloadbalancing:DescribeLoadBalancers"
            ],
            "Resource": "*"
        }
    ]
}

Fault tunables

Mandatory tunables

Tunable	Description	Notes
LOAD_BALANCER_ARN	Target load balancer ARN whose AZ should be detached.	For example, arn:aws:elasticloadbalancing:us-east-2:100054111296:loadbalancer/app/test-alb/09121290906ffab7.
ZONES	Target zones that should be detached from the ALB.	For example, us-east-1a.
REGION	Region name for the target volumes.	For example, us-east-1.

Optional tunables

Tunable	Description	Notes
TOTAL_CHAOS_DURATION	Duration to insert chaos (in seconds).	Default: 30 s. For more information, go to duration of the chaos.
CHAOS_INTERVAL	Duration between the attachment and detachment of the volumes (in seconds).	Default: 30 s. For more information, go to chaos interval.
SEQUENCE	Sequence of chaos execution for multiple volumes.	Default: parallel. Supports serial and parallel. For more information, go to sequence of chaos execution.
RAMP_TIME	Duration to wait before and after injecting chaos (in seconds).	For example, 30 s. For more information, go to ramp time.

Target zones

Comma-separated list of target zones. Tune it by using the ZONES environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains alb az down for given zones
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: alb-az-down
    spec:
      components:
        env:
        # load balancer arn for chaos
        - name: LOAD_BALANCER_ARN
          value: 'arn:aws:elasticloadbalancing:us-east-2:100054111296:loadbalancer/app/test-alb/09121290906ffab7'
        # target zones for the chaos
        - name: ZONES
          value: 'us-east-1a,us-east-1b'
        # region for chaos
        - name: REGION
          value: 'us-east-1'