Skip to main content

CLB AZ down

Introduction

CLB (Classic Load Balancer) AZ (Availability Zones) down takes down the AZ on a target CLB for a specific duration. This fault restricts access to certain availability zones for a specific duration.

CLB AZ Down

Use cases

CLB AZ down:

  • Breaks the connectivity of a CLB with the given zones and impacts their delivery.
  • Detaches the AZ from the classic load balancer thereby disrupting the dependent application's performance.
  • Tests the application sanity, availability, and recovery workflows of the application pod attached to the load balancer.
note
  • Kubernetes version 1.17 or later is required to execute this fault.
  • Appropriate AWS access to attach or detach an AZ from CLB.
  • A minimum of one AZ attached to the CLB after injecting chaos; otherwise, the fault fails to detach the given AZ.
  • The CLB should be attached to the given availability zones.
  • The Kubernetes secret should have AWS access configuration (key) in the CHAOS_NAMESPACE. Below is a sample secret file.
    apiVersion: v1
    kind: Secret
    metadata:
    name: cloud-secret
    type: Opaque
    stringData:
    cloud_config.yml: |-
    # Add the cloud AWS credentials respectively
    [default]
    aws_access_key_id = XXXXXXXXXXXXXXXXXXX
    aws_secret_access_key = XXXXXXXXXXXXXXX
  • We recommend that you use the same secret name, that is, cloud-secret. Otherwise, you will need to update the AWS_SHARED_CREDENTIALS_FILE environment variable in the fault template with the new secret name and you won't be able to use the default health check probes.
  • Go to AWS named profile for chaos to use a different profile for AWS faults and superset permission or policy to execute all AWS faults.

Below is an example AWS policy to execute the fault.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstances",
"ec2:DescribeSubnets",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:DescribeLoadBalancers"
],
"Resource": "*"
}
]
}

Fault tunables

Mandatory tunables

Tunable Description Notes
LOAD_BALANCER_NAME Name of the load balancer whose AZ has to be detached. For example, clb-name.
ZONES Target zones that have to be detached from the CLB. For example, us-east-1a.
REGION Region name for the target volumes. For example, us-east-1.

Optional tunables

Tunable Description Notes
TOTAL_CHAOS_DURATION Duration to insert chaos (in seconds). Default: 30 s. For more information, go to duration of the chaos.
CHAOS_INTERVAL Duration between the attachment and detachment of the volumes (in seconds). Default: 30 s. For more information, go to chaos interval.
SEQUENCE Sequence of chaos execution for multiple volumes. Default: parallel. Supports serial and parallel. For more information, go to sequence of chaos execution.
RAMP_TIME Period to wait before and after injecting chaos (in seconds). For example, 30 s. For more information, go to ramp time.

Target zones

Comma-separated list of the target zones. Tune it by using the ZONES environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains clb az down for given zones
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-nginx
spec:
engineState: "active"
chaosServiceAccount: litmus-admin
experiments:
- name: clb-az-down
spec:
components:
env:
# load balancer name for chaos
- name: LOAD_BALANCER_NAME
value: 'tes-clb'
# target zones for the chaos
- name: ZONES
value: 'us-east-1a,us-east-1b'
# region for chaos
- name: REGION
value: 'us-east-1'