Skip to main content

Harness Kubernetes services

This topic describes how to add and configure a Harness Kubernetes service.

A Kubernetes service represents the microservices and other workloads you want to deploy to the cluster.

Setting up a Kubernetes service involves the following steps:

  1. Add your manifests.
  2. Add the artifacts you want to deploy.
  3. Add any service variables you want to use in your manifests or pipeline.

Manifests

Harness supports the following manifest types and orchestration methods.

Kubernetes

Use Kubernetes manifests

You can use:

  • Standard Kubernetes manifests hosted in any repo or in Harness.
  • Values YAML files that use Go templating to template manifests.
  • Values YAML files can use a mix of hardcoded values and Harness expressions.
Watch a short video
Here's a quick video showing you how to add manifests and Values YAML files in Harness. It covers Kubernetes as well as other types like Helm Charts.

Here's a YAML example for a service with manifests hosted in Github and the nginx image hosted in Docker Hub.

Example
service:
name: Kubernetes
identifier: Kubernetes
serviceDefinition:
type: Kubernetes
spec:
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: Docker_Hub_with_Pwd
imagePath: library/nginx
tag: <+input>
identifier: nginx
type: DockerRegistry
manifests:
- manifest:
identifier: nginx
type: K8sManifest
spec:
store:
type: Github
spec:
connectorRef: harnessdocs2
gitFetchType: Branch
paths:
- default-k8s-manifests/Manifests/Files/templates
branch: main
valuesPaths:
- default-k8s-manifests/Manifests/Files/ng-values.yaml
skipResourceVersioning: false
gitOpsEnabled: false

Ignore a manifest file during deployment

You might have manifest files for resources that you do not want to deploy as part of the main deployment.

Instead, you can tell Harness to ignore these files and then apply them separately using the Harness Apply step. Or you can simply ignore them and deploy them later.

See Ignore a manifest file during deployment and Apply.

Notes

  • If this is your first time using Harness for a Kubernetes deployment, see Kubernetes CD tutorial.
  • For a task-based walkthroughs of different Kubernetes features in Harness, see Kubernetes How-tos.
  • You can hardcode your artifact in your manifests, our add your artifact source to your Service Definition and then reference it in your manifests.

Helm Charts

Use Helm Charts

You can use Helm charts stored in an HTTP Helm Repository, OCI Registry, a Git repo provider, a cloud storage service (Google Cloud Storage, AWS S3, Azure Repo), a custom repo, or the Harness File Store.

Here's a YAML example for a service with manifests hosted in Github and the nginx image hosted in Docker Hub.

Example
service:
name: Helm Chart
identifier: Helm_Chart
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: nginx
type: HelmChart
spec:
store:
type: Http
spec:
connectorRef: Bitnami
chartName: nginx
helmVersion: V3
skipResourceVersioning: false
commandFlags:
- commandType: Template
flag: mychart -x templates/deployment.yaml
type: Kubernetes

Important notes

  • If this is your first time using Harness for a Helm Chart deployment, see Helm Chart deployment tutorial.
  • For a detailed walkthrough of deploying Helm Charts in Harness, including limitations and binary support, see Deploy Helm Charts.
  • Harness does not support AWS cross-account access for ChartMuseum and AWS S3. For example, if the Harness delegate used to deploy charts is in AWS account A, and the S3 bucket is in AWS account B, the Harness connector that uses this delegate in A cannot assume the role for the B account.
  • Harness cannot fetch Helm chart versions with Helm OCI because Helm OCI no longer supports helm chart list. See OCI Feature Deprecation and Behavior Changes with Helm v3.7.0.
  • Currently, you cannot list the OCI image tags in Harness. This is a Helm limitation. For more information, go to Helm Search Repo Chart issue.

Kustomize

Use Kustomize

Harness supports Kustomize deployments. You can use overlays, multibase, plugins, sealed secrets, patches, etc, just as you would in any native kustomization.

Here's a YAML example for a service using a publicly available helloword kustomization cloned from Kustomize.

Example
service:
name: Kustomize
identifier: Kustomize
serviceDefinition:
type: Kubernetes
spec:
manifests:
- manifest:
identifier: kustomize
type: Kustomize
spec:
store:
type: Github
spec:
connectorRef: Kustomize
gitFetchType: Branch
folderPath: kustomize/helloworld
branch: main
pluginPath: ""
skipResourceVersioning: false
gitOpsEnabled: false

If this is your first time using Harness for a Kustomize deployment, see the Kustomize Quickstart.

For a detailed walkthrough of deploying Kustomize in Harness, including limitations, see Use Kustomize for Kubernetes Deployments.

Important notes

  • Harness supports Kustomize and Kustomize Patches for Rolling, Blue Green and Delete steps.
  • Harness does not use Kustomize for rollback. Harness renders the templates using Kustomize and then passes them onto kubectl. A rollback works exactly as it does for native Kubernetes.
  • You cannot use Harness variables in the base manifest or kustomization.yaml. You can only use Harness variables in kustomize patches you add in Kustomize Patches Manifest Details.
  • Kustomize binary versions:
    • Harness includes Kustomize binary versions 3.5.4 and 4.0.0. By default, Harness uses 3.5.4.
    • To use 4.0.0, you must enable the feature flag NEW_KUSTOMIZE_BINARY in your account. Contact Harness Support to enable the feature.
  • Harness will not follow symlinks in the Kustomize and Kustomize Patches files it pulls.

OpenShift templates

Use OpenShift templates

Harness supports OpenShift for Kubernetes deployments.

For an overview of OpenShift support, see Using OpenShift with Harness Kubernetes.

Here's a YAML example for a service using an OpenShift template that is stored in the Harness File Store.

Example
service:
name: OpenShift Template
identifier: OpenShift
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: nginx
type: OpenshiftTemplate
spec:
store:
type: Harness
spec:
files:
- /OpenShift/templates/example-template.yml
skipResourceVersioning: false
type: Kubernetes

Deployment strategy support

In addition to standard workload type support in Harness (see What can I deploy in Kubernetes?), Harness supports DeploymentConfigRoute, and ImageStream across Canary, Blue Green, and Rolling deployment strategies.

Please use apiVersion: apps.openshift.io/v1 and not apiVersion: v1.

Harness supports list objects

You can leverage Kubernetes list objects as needed without modifying your YAML for Harness.

When you deploy, Harness will render the lists and show all the templated and rendered values in the log.

Harness supports:

  • List
  • NamespaceList
  • ServiceList
  • For Kubernetes deployments, these objects are supported for all deployment strategies (Canary, Rolling, Blue/Green).
  • For Native Helm, these objects are supported for Rolling deployments.

If you run kubectl api-resources you should see a list of resources, and kubectl explain will work with any of these.

Important notes

Artifacts

You have two options when referencing the artifacts you want to deploy:

  • Add an artifact source to the Harness service and reference it using the Harness expression <+artifacts.primary.image> in the values YAML file.
  • Hardcode the artifact into the manifests or values YAML file.
Use the artifact expression

Add the image location to Harness as an artifact in the Artifacts section of the service.

This allows you to reference the image in your values YAML files using the Harness expression <+artifacts.primary.image>.

...  
image: <+artifacts.primary.image>
...

You cannot use Harness variables expressions in your Kubernetes object manifest files. You can only use Harness variables expressions in values YAML files, or Kustomize Patch file.

When you select the artifact repo for the artifact, like a Docker Hub repo, you specify the artifact and tag/version to use.

You can select a specific tag/version, use a runtime input so that you are prompted for the tag/version when you run the pipeline, or you can use an Harness variable expression to pass in the tag/version at execution.

Here's an example where a runtime input is used and you select which image version/tag to deploy.

With a Harness artifact, you can template your manifests, detaching them from a hardcoded location. This makes your manifests reusable and dynamic.

Hardcode the artifact

If a Docker image location is hardcoded in your Kubernetes manifest (for example, image: nginx:1.14.2), then you can simply add the manifest to Harness in Manifests and Kubernetes will pull the image during deployment.

When you hardcode the artifact in your manifests, any artifacts added to your Harness service are ignored.

Docker

Use artifacts in any Docker registry

To use a Docker artifact, you create or use a Harness connector to connect to your Docker repo and then use that connector in your Harness service and reference the artifact to use.

Docker connector YAML
connector:
name: Docker Hub with Pwd
identifier: Docker_Hub_with_Pwd
description: ""
orgIdentifier: default
projectIdentifier: CD_Docs
type: DockerRegistry
spec:
dockerRegistryUrl: https://index.docker.io/v2/
providerType: DockerHub
auth:
type: UsernamePassword
spec:
username: johndoe
passwordRef: Docker_Hub_Pwd
executeOnDelegate: false
Service using Docker artifact YAML
service:
name: Example K8s2
identifier: Example_K8s2
serviceDefinition:
type: Kubernetes
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /Templates/deployment.yaml
valuesPaths:
- /values.yaml
skipResourceVersioning: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: Docker_Hub_with_Pwd
imagePath: library/nginx
tag: stable-perl
identifier: myimage
type: DockerRegistry
gitOpsEnabled: false

Important notes

  • For pulling Docker images from Docker repos, Harness is restricted by the limits of the Docker repo. For example, Docker Hub limits.
  • The maximum number of artifact image tags fetched by Harness that is 10000.

Google Container Registry (GCR)

Use GCR artifacts

You connect to GCR using a Harness GCP Connector. For details on all the GCR requirements for the GCP Connector, see Google Cloud Platform (GCP) Connector Settings Reference.

To use a GCR artifact, you create or use a Harness GCP Connector to connect to GCR repo and then use that connector in your Harness service and reference the artifact to use.

GCP connector YAML

This example uses a Harness delegate installed in GCP for credentials.

connector:
name: GCR
identifier: GCR
description: ""
orgIdentifier: default
projectIdentifier: CD_Docs
type: Gcp
spec:
credential:
type: InheritFromDelegate
delegateSelectors:
- gcpdocplay
executeOnDelegate: true
Service using GCR artifact YAML
service:
name: Google Artifact
identifier: Google_Artifact
serviceDefinition:
type: Kubernetes
spec:
manifests:
- manifest:
identifier: manifests
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- account:/Templates
valuesPaths:
- account:/values.yaml
skipResourceVersioning: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: GCR
imagePath: docs-play/todolist-sample
tag: <+input>
registryHostname: gcr.io
identifier: myapp
type: Gcr
gitOpsEnabled: false

Permissions

For Google Container Registry (GCR), the following roles are required:

  • Storage Object Viewer (roles/storage.objectViewer)
  • Storage Object Admin (roles/storage.objectAdmin)

For more information, go to the GCP documentation about Cloud IAM roles for Cloud Storage.

Ensure the Harness delegate you have installed can reach storage.cloud.google.com and your GCR registry host name, for example gcr.io.

Google Artifact Registry

Use Google Artifact Registry artifacts

You connect to Google Artifact Registry using a Harness GCP Connector.

For details on all the Google Artifact Registry requirements for the GCP Connector, see Google Cloud Platform (GCP) Connector Settings Reference.

This example uses a Harness delegate installed in GCP for credentials.

Google Artifact Registry connector YAML
connector:
name: Google Artifact Registry
identifier: Google_Artifact_Registry
description: ""
orgIdentifier: default
projectIdentifier: CD_Docs
type: Gcp
spec:
credential:
type: InheritFromDelegate
delegateSelectors:
- gcpdocplay
executeOnDelegate: true
Service using Google Artifact Registry artifact YAML
service:
name: Google Artifact Registry
identifier: Google_Artifact_Registry
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /Templates
valuesPaths:
- /values.yaml
skipResourceVersioning: false
enableDeclarativeRollback: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- identifier: myapp
spec:
connectorRef: Google_Artifact_Registry
repositoryType: docker
project: docs-play
region: us-central1
repositoryName: quickstart-docker-repo
package: quickstart-docker-repo
version: <+input>
type: GoogleArtifactRegistry
type: Kubernetes

Permissions

For Google Artifact Registry, the following roles are required:

  • Artifact Registry Reader
  • Artifact Registry Writer

For more information, go to the GCP documentation Configure roles and permissions.

Ensure the Harness delegate you have installed can reach your Google Artifact Registry region, for example us-central1.

Amazon Elastic Container Registry (ECR)

Use ECR artifacts

You connect to ECR using a Harness AWS connector. For details on all the ECR requirements for the AWS connector, see AWS Connector Settings Reference.

This example uses a Harness delegate installed in AWS for credentials.

ECR connector YAML
connector:
name: ECR
identifier: ECR
orgIdentifier: default
projectIdentifier: CD_Docs
type: Aws
spec:
credential:
type: ManualConfig
spec:
accessKey: xxxxx
secretKeyRef: secretaccesskey
region: us-east-1
delegateSelectors:
- doc-immut
executeOnDelegate: true
Service using ECR artifact YAML
service:
name: ECR
identifier: ECR
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /values.yaml
valuesPaths:
- /Templates
skipResourceVersioning: false
enableDeclarativeRollback: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: ECR
imagePath: todolist-sample
tag: "1.0"
region: us-east-1
identifier: myapp
type: Ecr
type: Kubernetes

Permissions

Ensure that the AWS IAM user account you use in the AWS Connector has the following policy.

Pull from ECR policy
  • Policy Name: AmazonEC2ContainerRegistryReadOnly
  • Policy ARN: arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
  • Description: Provides read-only access to Amazon EC2 Container Registry repositories.
  • Policy JSON:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:DescribeImages",
"ecr:BatchGetImage"
],
"Resource": "*"
}
]
}

Azure Container Registry (ACR)

Use ACR artifacts

You connect to ACR using a Harness Azure Connector. For details on all the Azure requirements for the Azure Connector, see Add a Microsoft Azure cloud connector.

This example uses a Harness delegate installed in Azure for credentials.

Azure connector for ACR YAML
connector:
name: ACR-docs
identifier: ACRdocs
description: ""
orgIdentifier: default
projectIdentifier: CD_Docs
type: Azure
spec:
credential:
type: ManualConfig
spec:
applicationId: xxxxx-xxxx-xxxx-xxxx-xxxxx
tenantId: xxxxx-xxxx-xxxx-xxxx-xxxxx
auth:
type: Secret
spec:
secretRef: acrvalue
azureEnvironmentType: AZURE
executeOnDelegate: false
Service using ACR artifact YAML
service:
name: Azure with ACR
identifier: Azure
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /Templates
valuesPaths:
- /values.yaml
skipResourceVersioning: false
enableDeclarativeRollback: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: ACRdocs
tag: <+input>
subscriptionId: <+input>
registry: <+input>
repository: <+input>
identifier: myapp
type: Acr
type: Kubernetes

Permissions

The Harness Azure connectors that you'll use to connect Harness to ACR must have the Reader role, at minimum. You can also use a custom role that includes the permissions of the Reader role.

Reader role information

ACR Reader role

The Reader role must be assigned at the Subscription or Resource Group level that is used by the Application (Client) Id that you'll use in the Azure connector's settings. The application must have permission to list all container registries.

tip

Make sure you:

  • Don't put the Reader role in a different IAM section of Azure.
  • Don't provide only the AcrPull role, instead of Reader. It might appear that the AcrPull role gives access to a specific registry, but Harness needs to list all registries.
Custom role information

The following permissions (actions) are necessary for any Service Principal and/or Managed Identity user, regardless of whether you are using Kubernetes RBAC or Azure RBAC:

  • Microsoft.ContainerRegistry/registries/read
  • Microsoft.ContainerRegistry/registries/builds/read
  • Microsoft.ContainerRegistry/registries/metadata/read
  • Microsoft.ContainerRegistry/registries/pull/read
  • Microsoft.ContainerService/managedClusters/read
  • Microsoft.ContainerService/managedClusters/listClusterUserCredential/action
  • Microsoft.Resource/subscriptions/resourceGroup/read

For Helm deployments, the version of Helm must be >= 3.2.0. The Harness HELM_VERSION_3_8_0 feature flag must be activated.

You can't use Pod Assigned Managed Identity and System Assigned Managed Identity for the same cluster.

The following JSON sample creates a custom role with the required permissions. To use this sample, replace xxxx with the role name, subscription Id, and resource group Id.

{
"id": "/subscriptions/xxxx/providers/Microsoft.Authorization/roleDefinitions/xxxx",
"properties": {
"roleName": "xxxx",
"description": "",
"assignableScopes": [
"/subscriptions/xxxx/resourceGroups/xxxx"
],
"permissions": [
{
"actions": [],
"notActions": [],
"dataActions": [
"Microsoft.ContainerService/managedClusters/configmaps/read",
"Microsoft.ContainerService/managedClusters/configmaps/write",
"Microsoft.ContainerService/managedClusters/configmaps/delete",
"Microsoft.ContainerService/managedClusters/secrets/read",
"Microsoft.ContainerService/managedClusters/secrets/write",
"Microsoft.ContainerService/managedClusters/secrets/delete",
"Microsoft.ContainerService/managedClusters/apps/deployments/read",
"Microsoft.ContainerService/managedClusters/apps/deployments/write",
"Microsoft.ContainerService/managedClusters/apps/deployments/delete",
"Microsoft.ContainerService/managedClusters/events/read",
"Microsoft.ContainerService/managedClusters/events/write",
"Microsoft.ContainerService/managedClusters/events/delete",
"Microsoft.ContainerService/managedClusters/namespaces/read",
"Microsoft.ContainerService/managedClusters/nodes/read",
"Microsoft.ContainerService/managedClusters/pods/read",
"Microsoft.ContainerService/managedClusters/pods/write",
"Microsoft.ContainerService/managedClusters/pods/delete",
"Microsoft.ContainerService/managedClusters/services/read",
"Microsoft.ContainerService/managedClusters/services/write",
"Microsoft.ContainerService/managedClusters/services/delete",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/read",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/write",
"Microsoft.ContainerService/managedClusters/apps/statefulsets/delete",
"Microsoft.ContainerService/managedClusters/apps/replicasets/read",
"Microsoft.ContainerService/managedClusters/apps/replicasets/write",
"Microsoft.ContainerService/managedClusters/apps/replicasets/delete"
],
"notDataActions": []
}
]
}
}

Important notes

  • Harness supports 500 images from an ACR repo. If you don't see some of your images, then you might have exceeded this limit. This is the result of an Azure API limitation.
  • If you connect to an ACR repo via the platform-agnostic Docker Connector, the limit is 100.

Nexus

Use Nexus artifacts

You connect to Nexus using a Harness Nexus Connector. For details on all the requirements for the Nexus Connector, see Nexus Connector Settings Reference.

Nexus connector YAML
connector:
name: Harness Nexus
identifier: Harness_Nexus
description: ""
orgIdentifier: default
projectIdentifier: CD_Docs
type: HttpHelmRepo
spec:
helmRepoUrl: https://nexus3.dev.harness.io/repository/test-helm/
auth:
type: UsernamePassword
spec:
username: harnessadmin
passwordRef: nexus3pwd
delegateSelectors:
- gcpdocplay
Service using Nexus artifact YAML
service:
name: Nexus Example
identifier: Nexus_Example
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /Templates
valuesPaths:
- /values.yaml
skipResourceVersioning: false
enableDeclarativeRollback: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: account.Harness_Nexus
repository: todolist
repositoryFormat: docker
tag: "4.0"
spec:
artifactPath: nginx
repositoryPort: "6661"
identifier: myapp
type: Nexus3Registry
type: Kubernetes

Permissions

Ensure the connected user account has the following permissions in the Nexus Server.

  • Repo: All repositories (Read)
  • Nexus UI: Repository Browser

Nexus repo perms for Harness

See Nexus Managing Security.

For Nexus 3, when used as a Docker repo, the user needs:

  • A role with the nx-repository-view-*_*_* privilege.

Artifactory

Use Artifactory artifacts

You connect to Artifactory (JFrog) using a Harness Artifactory Connector. For details on all the requirements for the Artifactory Connector, see Artifactory Connector Settings Reference.

Artifactory connector YAML
connector:
name: artifactory-tutorial-connector
identifier: artifactorytutorialconnector
orgIdentifier: default
projectIdentifier: CD_Docs
type: Artifactory
spec:
artifactoryServerUrl: https://harness.jfrog.io/artifactory/
auth:
type: Anonymous
executeOnDelegate: false
Service using Artifactory artifact YAML
service:
name: Artifactory Example
identifier: Artifactory_Example
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /Templates
valuesPaths:
- /values.yaml
skipResourceVersioning: false
enableDeclarativeRollback: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- spec:
connectorRef: artifactorytutorialconnector
artifactPath: alpine
tag: 3.14.2
repository: bintray-docker-remote
repositoryUrl: harness-docker.jfrog.io
repositoryFormat: docker
identifier: myapp
type: ArtifactoryRegistry
type: Kubernetes

Permissions

Make sure the following permissions are granted to the user:

  • Privileged User is required to access API, whether Anonymous or a specific username (username and passwords are not mandatory).
  • Read permission to all Repositories.

If used as a Docker Repo, user needs:

  • List images and tags
  • Pull images

See Managing Permissions: JFrog Artifactory User Guide.

Github packages

Use Github packages as artifacts

You can use Github Packages as artifacts for deployments.

Currently, Harness supports only the packageType as docker(container). Support for npm, maven, rubygems, and nuget is coming soon.

You connect to Github using a Harness Github Connector, username, and Personal Access Token (PAT).

tip

New to Github Packages? This quick video will get you up to speed in minutes.

GitHub Packages connector YAML
connector:
name: GitHub Packages
identifier: GitHub_Packages
orgIdentifier: default
projectIdentifier: CD_Docs
type: Github
spec:
url: https://github.com/johndoe/myapp.git
validationRepo: https://github.com/johndoe/test.git
authentication:
type: Http
spec:
type: UsernameToken
spec:
username: johndoe
tokenRef: githubpackages
apiAccess:
type: Token
spec:
tokenRef: githubpackages
delegateSelectors:
- gcpdocplay
executeOnDelegate: true
type: Repo
Service using Github Packages artifact YAML
service:
name: Github Packages
identifier: Github_Packages
tags: {}
serviceDefinition:
spec:
manifests:
- manifest:
identifier: myapp
type: K8sManifest
spec:
store:
type: Harness
spec:
files:
- /Templates
valuesPaths:
- /values.yaml
skipResourceVersioning: false
enableDeclarativeRollback: false
artifacts:
primary:
primaryArtifactRef: <+input>
sources:
- identifier: myapp
spec:
connectorRef: GitHub_Packages
org: ""
packageName: tweetapp
packageType: container
version: latest
type: GithubPackageRegistry
type: Kubernetes

Permissions

The Github Personal Access Token (PAT) must have the write:packages and read:packages permissions.

Ensure that you enable API access in the Harness Github connector. In the Github connector YAML, the setting is apiAccess:

GitHub Packages connector YAML
connector:
name: GitHub Packages
identifier: GitHub_Packages
orgIdentifier: default
projectIdentifier: CD_Docs
type: Github
spec:
url: https://github.com/johndoe/myapp.git
validationRepo: https://github.com/johndoe/test.git
authentication:
type: Http
spec:
type: UsernameToken
spec:
username: johndoe
tokenRef: githubpackages
apiAccess:
type: Token
spec:
tokenRef: githubpackages
delegateSelectors:
- gcpdocplay
executeOnDelegate: true
type: Repo

You can use the same Harness secret that you used for user authentication.

Custom artifact source

Use artifacts from a custom artifact source

For enterprises that use a custom repository, Harness provides the Custom Artifact Source.

To use this artifact source, you provide a script to query your artifact server via its API (for example, REST) and then Harness stores the output on the Harness Delegate in the Harness-initialized variable $HARNESS_ARTIFACT_RESULT_PATH.

The output must be a JSON array, with a mandatory key for a Build Number/Version. You then map a key from your JSON output to the Build Number/Version variable.

For steps on adding a Custom Artifact source, go to Add a custom artifact source for CD.

Go templating

Harness supports Go templating for Kubernetes manifests and values YAML files.

You can add one or more values YAML files containing values for different scenarios, and then use Go templating in the manifest files to reference the values in the values YAML files.

Built-in Go templating support enables you to use Kubernetes without the need for Helm.

For more information, see Example Kubernetes Manifests using Go Templating.

Let's look at a few Kubernetes templating examples.

Basic values YAML and manifests for a public image

Here's the values YAML file:

name: <+stage.name>  
replicas: 2

image: <+artifacts.primary.image>
# dockercfg: <+artifacts.primary.imagePullSecret>

createNamespace: true
namespace: <+infra.namespace>

serviceType: LoadBalancer

servicePort: 80
serviceTargetPort: 80

env:
config:
key1: value10
secrets:
key2: value2

Here's the manifest containing multiple objects referring to the values in the values YAML file:

{{- if .Values.env.config}}  
apiVersion: v1
kind: ConfigMap
metadata:
name: {{.Values.name}}
data:
{{.Values.env.config | toYaml | indent 2}}
---
{{- end}}

{{- if .Values.env.secrets}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}
stringData:
{{.Values.env.secrets | toYaml | indent 2}}
---
{{- end}}

{{- if .Values.dockercfg}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}-dockercfg
annotations:
harness.io/skip-versioning: true
data:
.dockercfg: {{.Values.dockercfg}}
type: kubernetes.io/dockercfg
---
{{- end}}

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{.Values.name}}-deployment
spec:
replicas: {{int .Values.replicas}}
selector:
matchLabels:
app: {{.Values.name}}
template:
metadata:
labels:
app: {{.Values.name}}
spec:
{{- if .Values.dockercfg}}
imagePullSecrets:
- name: {{.Values.name}}-dockercfg
{{- end}}
containers:
- name: {{.Values.name}}
image: {{.Values.image}}
{{- if or .Values.env.config .Values.env.secrets}}
envFrom:
{{- if .Values.env.config}}
- configMapRef:
name: {{.Values.name}}
{{- end}}
{{- if .Values.env.secrets}}
- secretRef:
name: {{.Values.name}}
{{- end}}
{{- end}}

Pull an image from a private registry

Typically, if the Docker image you are deploying is in a private registry, Harness has access to that registry using the credentials set up in the Harness connector you use with your service Artifacts.

If some cases, your Kubernetes cluster might not have the permissions needed to access a private Docker registry.

For these cases, the values YAML file in Service Definition Manifests section must use the dockercfg parameter.

If the Docker image is added in the Service Definition Artifacts section, then you reference it like this: dockercfg: <+artifacts.primary.imagePullSecret>.

This key will import the credentials from the Docker credentials file in the artifact.

Open the values.yaml file you are using for deployment.

Verify that dockercfg key exists, and uses the <+artifacts.primary.imagePullSecret> expression to obtain the credentials:

name: <+stage.variables.name>  
replicas: 2

image: <+artifacts.primary.image>
dockercfg: <+artifacts.primary.imagePullSecret>

createNamespace: true
namespace: <+infra.namespace>
...

Reference dockercfg in Kubernetes objects

Next, verify that the Deployment and Secret objects reference dockercfg: {{.Values.dockercfg}}.

Reference dockercfg in Kubernetes objects
...  
{{- if .Values.dockercfg}}
apiVersion: v1
kind: Secret
metadata:
name: {{.Values.name}}-dockercfg
annotations:
harness.io/skip-versioning: true
data:
.dockercfg: {{.Values.dockercfg}}
type: kubernetes.io/dockercfg
---
{{- end}}

apiVersion: apps/v1
kind: Deployment
metadata:
name: {{.Values.name}}-deployment
spec:
replicas: {{int .Values.replicas}}
selector:
matchLabels:
app: {{.Values.name}}
template:
metadata:
labels:
app: {{.Values.name}}
spec:
{{- if .Values.dockercfg}}
imagePullSecrets:
- name: {{.Values.name}}-dockercfg
{{- end}}
containers:
- name: {{.Values.name}}
image: {{.Values.image}}
...

With these requirements met, the cluster imports the credentials from the Docker credentials file in the artifact.

Sidecar workloads

You can use Harness to deploy both primary and sidecar Kubernetes workloads.

Kubernetes sidecar workloads are a powerful way to modularize and encapsulate application functionality while keeping the overall architecture simple and easy to manage.

Sidecars are commonly used to implement cross-cutting concerns like logging, monitoring, and security. By separating these concerns into separate containers, it's possible to add or modify them without affecting the primary container or the application running inside it.

For example, a logging sidecar can be used to capture and store application logs, metrics, and other data, without requiring the primary container to implement any logging code.

Sidecars can also be used to implement advanced features like load balancing, service discovery, and circuit breaking. By using a sidecar container for these features, it's possible to keep the primary container simple and focused on its core functionality, while still providing advanced capabilities to the application.

For more information, go to Add a Kubernetes sidecar container.

Variables

In the Variables section of the service you can add variables are use them in values YAML and params files, or in other settings in your stage that support expressions.

Harness service variables section

You can use the variable in your values YAML files. For example, here's a service variable used in the name key in the values YAML:

Service variable in values YAML file
name: <+serviceVariables.appname>  
replicas: 2

image: <+artifacts.primary.image>
# dockercfg: <+artifacts.primary.imagePullSecret>

createNamespace: true
namespace: <+infra.namespace>

serviceType: LoadBalancer

servicePort: 80
serviceTargetPort: 80

env:
config:
key1: value10
secrets:
key2: value2

You can set variable values as runtime inputs and then provide their values when the pipeline is run.

For more information on runtime inputs and expressions, go to Fixed values runtime inputs and expressions.

Harness pipeline, stage, service, and built-in variables

You can use pipeline, stage, service, environment, and Harness built-in variables in your values YAML files and service settings.

See Built-in Harness variables reference or watch this short video.

Propagate and override artifacts, manifests, and service variables

You can propagate services between stages and override service settings by using multiple values YAML files and/or Environment Overrides.

For more information, go to:

Next steps

Once you've configured your service, you can move onto the stage's Environment settings and define the target Kubernetes cluster and namespace for your deployment.

See Define your Kubernetes target infrastructure.