Kubernetes deployments
This topic shows you how to deploy a publicly-available NGINX Docker image and manifest into your Kubernetes cluster using a rolling update strategy in Harness.
Objectives
You'll learn how to:
- Install and launch a Harness Kubernetes Delegate in your target cluster.
- Set up a Kubernetes Pipeline.
- Run the new Kubernetes Pipeline and deploy a Docker image to your target cluster.
Before you begin
Make sure you have the following set up before you begin this quickstart:
- GitHub account: this quickstart uses a publicly available NGINX manifest and Docker image. DockerHub allows anonymous connections, but GitHub requires that you log into your account to access their repos.
Set up your Kubernetes cluster
You'll need a target Kubernetes cluster for the Harness Delegate and deployment. Ensure your cluster meets the following requirements:
- Number of nodes: 2.
- vCPUs, Memory, Disk Size: 4vCPUs, 16GB memory, 100GB disk. In GKE, the e2-standard-4 machine type is enough for this quickstart.
- Networking: outbound HTTPS for the Harness connection to app.harness.io, github.com, and hub.docker.com. Allow TCP port 22 for SSH.
- A Kubernetes service account with permission to create entities in the target namespace is required. The set of permissions should include
list
,get
,create
, anddelete
permissions. In general, the cluster-admin permission or namespace admin permission is enough.
For more information, go to User-Facing Roles from Kubernetes.
Create the deploy stage
Pipelines are collections of stages. For this tutorial, we'll create a new pipeline and add a single stage.
Create a Project for your new CD Pipeline: if you don't already have a Harness project, create a project for your new CD pipeline. Ensure that you add the Continuous Delivery module to the project. Go to create organizations and projects for more information.
In your Harness project, select Deployments, and then select Create a Pipeline.
Enter the name CD Quickstart and select Start.
Your pipeline appears.
Select Add Stage and select Deploy.
Enter the name Deploy Service, make sure Service is selected, and then select Set Up Stage.
The new stage settings appear.
In About the Service, select New Service.
noteLet's take a moment and review Harness services and service definitions (which are explained below). Harness services represent your microservices/apps logically. You can add the same service to as many stages as you need. Service definitions represent your artifacts, manifests, and variables physically. They are the actual files and variable values.
By separating services and service definitions, you can propagate the same Service across stages while changing the artifacts, manifests, and variables with each stage.
Give the service the name nginx and select Save.
Once you have created a service, it is persistent and can be used throughout the stages of this or any other pipeline in the project.
Add the manifest
Next, we can add a Kubernetes manifest for NGINX. We'll use the publicly-available manifest available from Kubernetes.
In Service Definition, in Deployment Type, select Kubernetes.
In Manifests, select Add Manifest.
noteWhat about Artifacts? In this quickstart we are using a publicly-available NGINX Docker image from DockerHub, and the location of the image is hardcoded in the public manifest from Kubernetes. The Artifacts section is only used when the public artifact is not hardcoded in the manifest or the repo is private. In those cases, you add the image in Artifacts with a Connector for the repo and then reference the image in your values.yaml (
image: <+artifact.image>
). For more information, go to Add Container Images as Artifacts for Kubernetes Deployments.Select K8s Manifest, and select Continue.
In Select K8sManifest Store, select GitHub, and then select New GitHub Connector.
The Git Connector settings appear. Enter the following settings.
Name: enter a name for the connector.
URL Type: select Repository.
Connection Type: select HTTP.
Git Repository URL: enter
https://github.com/kubernetes/website
.Username and Token: Enter the username and a Github Personal Access Token (PAT) for your GitHub account. You'll have to create a Harness secret for the password.
- In Personal Access Token, click Create or Select a Secret.
- Select New Secret Text.
- In Secret Name, enter a name for the secret like github-pat.
- In Secret Value, paste in a GitHub Personal access token. When you're logged into GitHub, these are typically listed at https://github.com/settings/tokens. For steps on setting up a GitHub PAT, go to creating a personal access token from GitHub. Ensure you PAT has the repo scope selected:
Select Continue.
In Connect to the provider, select Connect through a Harness Delegate, and then select Continue. We don't use the Connect through Harness Platform option here simply because you'll need a Delegate later for the connection to your target Kubernetes cluster. Typically, the Connect through Harness Platform option is a very quick way to make connections without having to use Delegates.
Expand the section below to learn more about installing delegates.
Use the delegate installation wizard
- In your Harness project, select Project Setup.
- Select Delegates.
- Select Install a Delegate.
- Follow the delegate installation wizard.
Use this delegate installation wizard video to guide you through the process.
Install a delegate using the terminal
Install Harness Delegate on Kubernetes or Docker
What is Harness Delegate?
Harness Delegate is a lightweight worker process that is installed on your infrastructure and communicates only via outbound HTTP/HTTPS to the Harness Platform. This enables the Harness Platform to leverage the delegate to execute the CI/CD and other tasks on your behalf, without any of your secrets leaving your network.
You can install the Harness Delegate on either Docker or Kubernetes.
Install Harness Delegate
Create a new delegate token
Log in to the Harness Platform and go to Account Settings -> Account Resources -> Delegates. Select the Tokens tab. Select +New Token, and enter a token name, for example firstdeltoken
. Select Apply. Harness Platform generates a new token for you. Select Copy to copy and store the token in a temporary file. You will provide this token as an input parameter in the next installation step. The delegate will use this token to authenticate with the Harness Platform.
Get your Harness account ID
Along with the delegate token, you will also need to provide your Harness accountId
as an input parameter during delegate installation. This accountId
is present in every Harness URL. For example, in the following URL:
https://app.harness.io/ng/#/account/6_vVHzo9Qeu9fXvj-AcQCb/settings/overview
6_vVHzo9Qeu9fXvj-AcQCb
is the accountId
.
Now you are ready to install the delegate on either Docker or Kubernetes.
- Kubernetes
- Docker
Prerequisite
Ensure that you have access to a Kubernetes cluster. For the purposes of this tutorial, we will use minikube
.
Install minikube
- On Windows:
choco install minikube
- On macOS:
brew install minikube
Now start minikube with the following config.
minikube start --memory 4g --cpus 4
Validate that you have kubectl access to your cluster.
kubectl get pods -A
Now that you have access to a Kubernetes cluster, you can install the delegate using any of the options below.
- Helm Chart
- Terraform Helm Provider
- Kubernetes Manifest
Install the Helm chart
As a prerequisite, you must have Helm v3 installed on the machine from which you connect to your Kubernetes cluster.
You can now install the delegate using the delegate Helm chart. First, add the harness-delegate
Helm chart repo to your local Helm registry.
helm repo add harness-delegate https://app.harness.io/storage/harness-download/delegate-helm-chart/
helm repo update
helm search repo harness-delegate
We will use the harness-delegate/harness-delegate-ng
chart in this tutorial.
NAME CHART VERSION APP VERSION DESCRIPTION
harness-delegate/harness-delegate-ng 1.0.8 1.16.0 A Helm chart for deploying harness-delegate
Now we are ready to install the delegate. The following example installs/upgrades firstk8sdel
delegate (which is a Kubernetes workload) in the harness-delegate-ng
namespace using the harness-delegate/harness-delegate-ng
Helm chart.
To install the delegate, do the following:
In Harness, select Deployments, then select your project.
Select Delegates under Project Setup.
Select Install a Delegate to open the New Delegate dialog.
Select Helm Chart under Install your Delegate.
Copy the
helm upgrade
command.Run the command.
The command uses the default values.yaml located in the delegate-helm-chart GitHub repo. If you want change one or more values in a persistent manner instead of the command line, you can download and update the values.yaml
file as per your need. You can use the updated values.yaml
file as shown below.
helm upgrade -i firstk8sdel --namespace harness-delegate-ng --create-namespace \
harness-delegate/harness-delegate-ng \
-f values.yaml \
--set delegateName=firstk8sdel \
--set accountId=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
--set delegateToken=PUT_YOUR_DELEGATE_TOKEN_HERE \
--set managerEndpoint=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE \
--set delegateDockerImage=harness/delegate:23.02.78306 \
--set replicas=1 --set upgrader.enabled=false
Create main.tf file
Harness uses a Terraform module for the Kubernetes delegate. This module uses the standard Terraform Helm provider to install the Helm chart onto a Kubernetes cluster whose config by default is stored in the same machine at the ~/.kube/config
path. Copy the following into a main.tf
file stored on a machine from which you want to install your delegate.
module "delegate" {
source = "harness/harness-delegate/kubernetes"
version = "0.1.5"
account_id = "PUT_YOUR_HARNESS_ACCOUNTID_HERE"
delegate_token = "PUT_YOUR_DELEGATE_TOKEN_HERE"
delegate_name = "firstk8sdel"
namespace = "harness-delegate-ng"
manager_endpoint = "PUT_YOUR_MANAGER_HOST_AND_PORT_HERE"
delegate_image = "harness/delegate:23.02.78306"
replicas = 1
upgrader_enabled = false
# Additional optional values to pass to the helm chart
values = yamlencode({
javaOpts: "-Xms64M"
})
}
provider "helm" {
kubernetes {
config_path = "~/.kube/config"
}
}
Now replace the variables in the file with your Harness accound ID and delegate token values. Replace PUT_YOUR_MANAGER_HOST_AND_PORT_HERE
with the Harness Manager Endpoint noted below. For Harness SaaS accounts, you can find your Harness Cluster Location on the Account Overview page under the Account Settings section of the left navigation. For Harness CDCE, the endpoint varies based on the Docker vs. Helm installation options.
Harness Cluster Location | Harness Manager Endpoint on Harness Cluster |
---|---|
SaaS prod-1 | https://app.harness.io |
SaaS prod-2 | https://app.harness.io/gratis |
SaaS prod-3 | https://app3.harness.io |
CDCE Docker | http://<HARNESS_HOST> if Docker Delegate is remote to CDCE or http://host.docker.internal if Docker Delegate is on same host as CDCE |
CDCE Helm | http://<HARNESS_HOST>:7143 where HARNESS_HOST is the public IP of the Kubernetes node where CDCE Helm is running |
Run Terraform init, plan, and apply
Initialize Terraform. This downloads the Terraform Helm provider to your machine.
terraform init
Run the following step to view the changes Terraform is going to make on your behalf.
terraform plan
Finally, run this step to make Terraform install the Kubernetes delegate using the Helm provider.
terraform apply
When prompted by Terraform if you want to continue with the apply step, type yes
, and then you will see output similar to the following.
helm_release.delegate: Creating...
helm_release.delegate: Still creating... [10s elapsed]
helm_release.delegate: Still creating... [20s elapsed]
helm_release.delegate: Still creating... [30s elapsed]
helm_release.delegate: Still creating... [40s elapsed]
helm_release.delegate: Still creating... [50s elapsed]
helm_release.delegate: Still creating... [1m0s elapsed]
helm_release.delegate: Creation complete after 1m0s [id=firstk8sdel]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Download a Kubernetes manifest template
curl -LO https://raw.githubusercontent.com/harness/delegate-kubernetes-manifest/main/harness-delegate.yaml
Replace variables in the template
Open the harness-delegate.yaml
file in a text editor and replace PUT_YOUR_DELEGATE_NAME_HERE
, PUT_YOUR_HARNESS_ACCOUNTID_HERE
, and PUT_YOUR_DELEGATE_TOKEN_HERE
with your delegate name (for example, firstk8sdel
), Harness accountId
, and delegate token values, respectively.
Replace the PUT_YOUR_MANAGER_HOST_AND_PORT_HERE
variable with the Harness Manager Endpoint noted below. For Harness SaaS accounts, you can find your Harness Cluster Location on the Account Overview page under the Account Settings section of the left navigation. For Harness CDCE, the endpoint varies based on the Docker vs. Helm installation options.
Harness Cluster Location | Harness Manager Endpoint on Harness Cluster |
---|---|
SaaS prod-1 | https://app.harness.io |
SaaS prod-2 | https://app.harness.io/gratis |
SaaS prod-3 | https://app3.harness.io |
CDCE Docker | http://<HARNESS_HOST> if Docker Delegate is remote to CDCE or http://host.docker.internal if Docker Delegate is on same host as CDCE |
CDCE Helm | http://<HARNESS_HOST>:7143 where HARNESS_HOST is the public IP of the Kubernetes node where CDCE Helm is running |
Apply the Kubernetes manifest
kubectl apply -f harness-delegate.yaml
Prerequisite
Ensure that you have the Docker runtime installed on your host. If not, use one of the following options to install Docker:
Install on Docker
Now you can install the delegate using the following command.
docker run --cpus=1 --memory=2g \
-e DELEGATE_NAME=docker-delegate \
-e NEXT_GEN="true" \
-e DELEGATE_TYPE="DOCKER" \
-e ACCOUNT_ID=PUT_YOUR_HARNESS_ACCOUNTID_HERE \
-e DELEGATE_TOKEN=PUT_YOUR_DELEGATE_TOKEN_HERE \
-e LOG_STREAMING_SERVICE_URL=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE/log-service/ \
-e MANAGER_HOST_AND_PORT=PUT_YOUR_MANAGER_HOST_AND_PORT_HERE \
harness/delegate:23.03.78904
Replace the PUT_YOUR_MANAGER_HOST_AND_PORT_HERE
variable with the Harness Manager Endpoint noted below. For Harness SaaS accounts, to find your Harness cluster location, select Account Settings, and then select Overview. In Account Overview, look in Account Settings. It is listed next to Harness Cluster Hosting Account.
For more information, go to View account info and subscribe to downtime alerts.
For Harness CDCE, the endpoint varies based on the Docker vs. Helm installation options.
Harness Cluster Location | Harness Manager Endpoint on Harness Cluster |
---|---|
SaaS prod-1 | https://app.harness.io |
SaaS prod-2 | https://app.harness.io/gratis |
SaaS prod-3 | https://app3.harness.io |
CDCE Docker | http://<HARNESS_HOST> if Docker Delegate is remote to CDCE or http://host.docker.internal if Docker Delegate is on same host as CDCE |
CDCE Helm | http://<HARNESS_HOST>:7143 where HARNESS_HOST is the public IP of the Kubernetes node where CDCE Helm is running |
If you are using a local runner CI build infrastructure, modify the delegate install command as explained in Use local runner build infrastructure
Verify delegate connectivity
Select Continue. After the health checks pass, your delegate is available for you to use. Select Done and verify your new delegate is listed.
Helm chart & Terraform Helm provider
Kubernetes manifest
Docker
You can now route communication to external systems in Harness connectors and pipelines by selecting this delegate via a delegate selector.
Delegate selectors do not override service infrastructure connectors. Delegate selectors only determine the delegate that executes the operations of your pipeline.
Troubleshooting
The delegate installer provides troubleshooting information for each installation process. If the delegate cannot be verified, select Troubleshoot for steps you can use to resolve the problem. This section includes the same information.
Harness asks for feedback after the troubleshooting steps. You are asked, Did the delegate come up?
If the steps did not resolve the problem, select No, and use the form to describe the issue. You'll also find links to Harness Support and to Delegate docs.
- Helm Chart
- Terraform Helm Provider
- Kubernetes Manifest
- Docker
Use the following steps to troubleshoot your installation of the delegate using Helm.
Verify that Helm is correctly installed:
Check for Helm:
helm
And then check for the installed version of Helm:
helm version
If you receive the message
Error: rendered manifests contain a resource that already exists...
, delete the existing namespace, and retry the Helm upgrade command to deploy the delegate.For further instructions on troubleshooting your Helm installation, go to Helm troubleshooting guide.
Check the status of the delegate on your cluster:
kubectl describe pods -n <namespace>
If the pod did not start, check the delegate logs:
kubectl logs -f <harnessDelegateName> -n <namespace>
If the state of the delegate pod is
CrashLoopBackOff
, check your allocation of compute resources (CPU and memory) to the cluster. A state ofCrashLoopBackOff
indicates insufficent Kubernetes cluster resources.If the delegate pod is not healthy, use the
kubectl describe
command to get more information:kubectl describe <pod_name> -n <namespace>
Use the following steps to troubleshoot your installation of the delegate using Terraform.
Verify that Terraform is correctly installed:
terraform -version
For further instructions on troubleshooting your installation of Terraform, go to the Terraform troubleshooting guide.
Check the status of the delegate on your cluster:
kubectl describe pods -n <namespace>
If the pod did not start, check the delegate logs:
kubectl logs -f <harnessDelegateName> -n <namespace>
If the state of the delegate pod is
CrashLoopBackOff
, check your allocation of compute resources (CPU and memory) to the cluster. A state ofCrashLoopBackOff
indicates insufficent Kubernetes cluster resources.If the delegate pod is not healthy, use the
kubectl describe
command to get more information:kubectl describe <pod_name> -n <namespace>
Use the following steps to troubleshoot your installation of the delegate using Kubernetes.
Check the status of the delegate on your cluster:
kubectl describe pods -n <namespace>
If the pod did not start, check the delegate logs:
kubectl logs -f <harnessDelegateName> -n <namespace>
If the state of the delegate pod is
CrashLoopBackOff
, check your allocation of compute resources (CPU and memory) to the cluster. A state ofCrashLoopBackOff
indicates insufficent Kubernetes cluster resources.If the delegate pod is not healthy, use the
kubectl describe
command to get more information:kubectl describe <pod_name> -n <namespace>
Use the following steps to troubleshoot your installation of the delegate using Docker:
Check the status of the delegate on your cluster:
docker container ls -a
If the pod is not running, check the delegate logs:
docker container logs <delegatename> -f
Restart the delegate container. To stop the container:
docker container stop <delegatename>
To start the container:
docker container start <delegatename>
Make sure the container has sufficient CPU and memory resources. If not, remove the older containers:
docker container rm [container id]
To learn more, watch the Delegate overview video.
Back in Set Up Delegates, you can select the new Delegate. In the list of Delegates, you can see your new Delegate and its tags.
Select the Connect using Delegates with the following Tags option.
Enter the tag of the new Delegate and select Save and Continue. When you are done, the Connector is tested.
Select Continue.
In Manifest Details, enter the following settings, test the connection, and click Submit. We are going to provide connection and path information for a manifest located at
https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/application/nginx-app.yaml
.- Manifest Identifier: enter nginx.
- Git Fetch Type**: select Latest from Branch**.
- Branch: enter main.
- File/Folder path:
content/en/examples/application/nginx-app.yaml
. This is the path from the repo root.
The manifest is now listed.
Select Next at the bottom of the Service tab.
Now that the artifact and manifest are defined, you can define the target cluster for your deployment.
Define your target cluster
The target cluster is your own Kubernetes cluster, hosted in your cloud environment. This is where we will deploy the Docker image using the manifest you selected.
Harness connects to all of the common cloud platforms and provides a platform-agnostic Kubernetes cluster connection that can connect to Kubernetes anywhere.
In Infrastructure Details, in Specify your environment, select New Environment. Just like with a service, you can create a new environment or selecting an existing one. We'll create a new one.
In New Environment, enter a name, select Pre-Production, and select Save. The new environment appears.
In Infrastructure Definition, select Kubernetes.
noteLet's take a moment and review Harness environments and infrastructure definitions. Harness environments represent your deployment targets logically (QA, Prod, etc). You can add the same Environment to as many stages as you need. Infrastructure definitions represent your target infrastructure physically. They are the actual clusters, hosts, etc.
By separating environments and infrastructure definitions, you can use the same environment in multiple stages while changing the target infrastructure settings with each stage.
An Infrastructure Definition is where you specify the target for your deployment. In this case, your Kubernetes cluster and namespace.
In Cluster Details, in Connector, select Select a connector.
Select New Connector.
The Kubernetes cluster connector appears.
The Kubernetes cluster connector is covered in detail here, but let's quickly walk through it.
Let's look at the steps:
In Kubernetes Cluster Connector, in Name, enter Kubernetes Quickstart, and select Continue.
In Details, select Use the credentials of a specific Harness Delegate. We will select the Delegate next.
Select Continue.
Select the Kubernetes Delegate you added earlier using its Tags, and then select Save and Continue. Harness verifies the Connector.
Select Finish.
Select the new Connector and then select Apply Selected.
Back in Cluster Details, in Namespace, enter the target namespace for the deployment. For example, default. You can use any namespace in your target cluster.
In Advanced, in Release name, enter
quickstart
. For a release name, you can use a built-in or custom expression or any text you like. Harness requires a release name for tracking. It is applied to the pods as a label. The release name must be unique across the cluster.When you are done, the Cluster Details will look something like this:
The target infrastructure is complete. Now we can add our stage steps.
Select Next to move onto Execution.
Add a rollout deployment Step
Now you can select the deployment strategy for this stage of the pipeline.
In Execution Strategies, select Rolling, and then select Use Strategy.
The Rollout Deployment step is added.
This is a standard Kubernetes rolling update. By default, Harness uses a
25% max unavailable, 25% max surge
strategy.
That's it. Now the pipeline stage is complete and you can deploy.
Deploy and review
Select Save > Save Pipeline and then Run. Now you can select the specific artifact to deploy.
In Primary Artifact, select stable. This is the same as using
docker pull nginx:stable
.Select Run Pipeline. Harness will verify the Pipeline and then run it. You can see the status of the deployment, and pause or abort it.
Toggle Console View to watch the deployment with more detailed logging. Select the Rollout Deployment step and expand Wait for Steady State.
You can see
deployment "my-nginx" successfully rolled out
.
Congratulations! The deployment was successful.
In your project's Deployments, you can see the deployment listed:
If you run into any errors, it is typically because the cluster does meet the requirements from Before You Begin section or the cluster's network setting does not allow the Delegate to connect to Docker Hub.In this tutorial, you learned how to:
- Install and launch a Harness Kubernetes Delegate in your target cluster.
- Connect Harness to your Kubernetes cluster and an artifact server.
- Add your manifests to Harness.
- Create an infrastructure definition that targets your cluster and namespace.
- Add a Kubernetes rolling update.
- Deploy your Kubernetes pipeline to your target cluster.
Next, try Harness Continuous Integration to build a codebase, upload it to a repo, and run unit and integrations tests: CI pipeline quickstart.
Clean Up
For steps on deleting the Delgate, go to delete a delegate.
Next Steps
See CD tutorials for other deployment features.