FAQ
This article addresses some frequently asked questions about Harness Continuous Delivery (CD).
How to use the "for" condition while using jexl condition for the trigger?
Suppose that trigger payload has multiple records and you want to search for a particular string so you can make use of jexl for loop to iterate the list and match a string as below:
for (item : <+trigger.payload.commits>) { if (item.message == "mymessage") {return true;} }; return false;
How to use the token for OCI repo in AWS ECR as the token by default expires every 12 hours?
We can set up the AWS Secret Manager connector, then save the ECR auth token into it. Set up automatic token rotation (say at 10hr intervals) within AWS secret manager. Then have the Harness connector link to that AWS SecretManager secret, so it pulls a fresh token every time.
In First Gen we use WINDOWS_RUNTIME_PATH while setting up a runtime directory, what is the corresponding way in Next Gen?
In NG we are not using any setup variables anymore, since it is Harness's internal step where we basically create a temp dir for the execution. We are creating a working directory in the Command Init unit on this %USERPROFILE% location.
In templateInput window why do we only show variables that have runtime input and not the ones which have a static value for input?
We only show runtime because we intend to show the user what is required of them for input. The form gets too long if we expose all the fixed values and we only require in the form the ones which need input and not what has already defined values.
How do we clean the state file for Terraform if there is no remote backend configured?
For the terraform step if the remote backend is not configured, the state file is being managed by Harness and it maps to the provisioner identifier itself. Hence the only way to get rid of the state file is to change the provisioner identifier in this scenario.
How to use reference modules in sub-directories as part of a Terraform step?
In Harness, users may want to reference sub-modules as part of a Terraform step. Here is a sample code as part of the tf file that can be utilized:
module "transit-gateway" { source = "git::https://gitlab.com/rubrik-octo/lab/source-modules.git//site-deploy/transit-gateway" }
Here you see a single git repository named 'source-modules, that has multiple modules inside various folders. By using the '//' at the end of the source location, you can instruct Terraform to checkout a specific folder.
Do we need to install jq library in delegate machine or harness itself providing jq by default?
Harness by default does not provide by default the jq on delegate host. You need to add the below command in your INIT_SCRIPT for this.
microdnf install jq
Can plan from terraform step be encrypted using a read only secret manager ?
For encrypting terraform plan with the selected secret manager we need the ability to write the encrypted plan to the secret manager and hence read only secret manager will not work for this scenario.
What operations are performed as part of the cleanup step in ssh command task.
For SSH, we by default add an initialize step and a clean step apart from command execution step. As part of the cleanup step we we delete the working directory that lies within /tmp on the remote connected host.
Whether pipeline GitHub triggers support for project variable reference?
Pipeline GitHub triggers won’t support project variable reference. As trigger yaml is independent of the pipeline yaml and the trigger will not be aware of the expression output.
Is it possible to use conditional execution in looping strategies?
If you use a looping strategy then you will not be able to apply conditional execution on the child steps.
How to upload a file into a specific folder present in the harness filestore from the pipeline stage (PowerShell script)?
This can be achieved by using an API which you can invoke using PowerShell https://apidocs.harness.io/tag/File-Store/#operation/create
How to Use Expressions or Variables in Repeat Looping Strategy?
To pass a dynamic array as an input to the looping strategy of the next step, you can replace <+execution.steps.ShellScript_1.output.outputVariables.ARRAY1> with <+<+execution.steps.ShellScript_1.output.outputVariables.ARRAY1>.split(",")>. This change allows you to split the array into individual items using a comma as the delimiter.
Why the "Always Execute this Step” condition does not always run in the CD pipeline?
Always execute step runs regardless of success or failure but in order to trigger this condition on failure the previous step should be considered as failure, if the error is rolled back then it is not considered a failure. Hence, the next step's Conditional Execution is not executed. Therefore, a failure strategy such as “Mark as failure” or "ignore failure" is required.
Can we retain more than 2 older release secrets and config maps?
No, Harness uses a fixed limit of 2 in its release history cleanup logic. This value cannot be changed.
What happens if my manifest files are changed during pipeline execution, will harness pick the latest file?
The files are fetched only during the execution step i.e. during rollout. if the files are changed and committed before the "Fetch file" step is executed in the rollout phase, Harness will pick the latest file.
Can I encrypt the Token/Secret passed in the INIT_SCRIPT?
Directly this cannot be encrypted but this use can be achieved by creating the k8s secret for the credentials and referring them in the init script.
example -
aws_access_key=kubectl get secrets/pl-credentials --template={{.data.aws_access_key}} | base64 -d
aws_secret_key= kubectl get secrets/pl-credentials --template={{.data.aws_secret_key}} | base64 -d
Another approach would be saving the value in Harness's secret manager/any other secret manager and referencing it in the script. Check this for more info - https://developer.harness.io/docs/platform/Secrets/add-use-text-secrets
K8s delete command is not working with the native helm?
The K8s delete command/step does not work with native helm deployment because Harness has different logic to maintain versioning and rollback for native helm and k8s.In the case of the native helm, If the deployment fails, we’ll uninstall it ourselves. However, if the user wants to pass some command flags with Uninstall, that can be passed by selecting Uninstall and passing the relevant command flags.
Check this for more details - https://developer.harness.io/docs/continuous-delivery/deploy-srv-diff-platforms/helm/deploy-helm-charts/#uninstall-command-flag
How do I run helm uninstall after a successful deployment?
To run Helm uninstall manually after a successful deployment. you can leverage the shell script step and run the helm uninstall release-name command from the delegate onto the cluster.
To run the shell script onto the required cluster, we need to specify the k8s cluster credentials to delegate.
For this use case within the shell script, you can simply reference credentials as ${HARNESS_KUBE_CONFIG_PATH}
export KUBECONFIG=${HARNESS_KUBE_CONFIG_PATH} kubectl get pods -n pod-test
With this even when running the shell script on the delegate host, it can refer to the credentials of the K8s cloud provider which is used inside the infrastructure definition associated with the workflow.
In the Rollout Deployment step, how Harness retrieves the events in the Wait for Steady State phase?
During the "Wait for Steady State" phase, Harness retrieves events using the kubectl rollout status command, which retrieves information directly from the Kubernetes API server. Harness continuously polls the Kubernetes API server while a rollout is in progress, ensuring that it remains updated until the rollout is either completed or encounters an error.
When migrating from FirstGen to NextGen, will the release number of ConfigMaps and Secrets be reset to 1?
In the case of migrating from Harness FirstGen to Harness NextGen, the numbering of ConfigMaps and Secrets in Kubernetes will not be automatically reset to start from 1 again. The numbering is based on the release history and is incremented based on the latest release number.
When you migrate your application to Harness NextGen and continue to use the same release name as before, the versioning will not be reset. Harness will fetch the ConfigMap in the cluster that stores all the Harness releases with their respective numbers. It will retrieve the latest release number from the ConfigMap and increment it by 1 for the next deployment. If versioning is enabled, Harness will append -<release-number> to each ConfigMap/Secret declared in the manifest.
Therefore, if you migrate to Harness NextGen and use the same cluster and release name, the release number will not break. The numbering will continue based on the existing release history.
It's important to note that Harness provides a declarative rollback feature, which eliminates the need for resource versioning. This means that even if you don't maintain the numbering scheme, you can still perform rollbacks effectively using the declarative rollback feature provided by Harness.
For more information, you can refer to the documentation on declarative rollback in the link provided: Harness Declarative Rollback.
How can I turn off FG (First Generation) responses or remove the switch to CG option?
To disable FG responses, please follow these steps:
- Go to your account settings.
- Locate the "Allow First Gen Access" option.
- Turn off the "Allow First Gen Access" setting.
- Once disabled, the "Launch First Gen" button will no longer be visible, and you will no longer receive FG responses.
We have multiple accounts, like sandbox and prod, and we want to move the developments from sandbox to prod easily. Is there a solution for this?
Absolutely! We recommend customers to use test orgs or projects for sandbox development. Our hierarchical separation allows them to isolate test cases from production workloads effectively.
For pipeline development concerns, we have a solution too. Customers can utilize our built-in branching support from GitX. You can create a separate branch for building and testing pipeline changes. Once the changes are tested and verified, you can merge the changes into their default branch.
Sandbox accounts are most valuable for testing external automation running against Harness, which helps in building or modifying objects. This way, you can test changes without affecting production environments.
Can you provide an example of deploying the delegate in a task-definition for ECS (Amazon Elastic Container Service)?
Certainly! You can find a step-by-step guide on how to deploy the delegate in a task-definition for ECS on our official documentation page: link to documentation.
Additionally, we have a GitHub repository with a Terraform module that demonstrates the process of deploying the delegate in ECS Fargate: link to GitHub repository. This resource can further assist you in implementing the delegate deployment.
What are the main differences between using JSON and YAML as a Values file for GO Templating?
- YAML 1.2 does not allow tabs for indentation.
- YAML imposes limitations on key lengths.
- YAML uses some different unicode escape sequences.
However, apart from these differences, basic JSON documents are considered valid YAML and can be used with GO Templating.
Under what condition does an immutable delegate automatically upgrade?
AutoUpgrade initiates when a new version of the delegate is published, not when the delegate is expired.
Under what condition does an immutable delegate automatically upgrade?
AutoUpgrade initiates when a new version of the delegate is published, not when the delegate is expired.
Is there an environment variable to set when starting the container to force the Docker delegate to use client tool libs from harness-qa-public QA repo?
To achieve this, you need to create a test image that points to the harness-qa-public QA repository. This involves updating the Docker file with the appropriate path to the QA buckets.
Is there a method to simulate CloudFormation changes without actually applying them?
Yes, you can achieve this by utilizing the Change Set Feature. First, create a change set to preview the changes that will be made. Once you are satisfied with the preview, you can execute the change set using the command: aws cloudformation execute-change-set. This allows you to assess the impact of the changes before applying them.
Is it possible to include FirstGen measures and dimensions in custom dashboards using NextGen dashboards?
Yes, NG Dashboards support CG (Custom Group) Data, and you can create custom dashboards with FirstGen measures and dimensions using the "create dashboard" option.
What steps are involved in obtaining output from a chained pipeline for use in a different stage?
To get output from a chained pipeline and utilize it in another stage, you need to specify the expression of the output variable for the chained pipeline at the parent pipeline level in the output section.
If I delete an infradef after deployments are done to it, what are the implications other than potential dashboard data loss for those deployments ?
At the moment there is no dependency on the instance sync and infrastructure definition. Infra definition is used only to generate infra details the instance sync itself is done for service and environment, only in case if any these are deleted the instance sync will stop and delete instances.
Note:
If you are using the default release name format in Harness FirstGen as release-${infra.kubernetes.infraId}, it's important to note that when migrating to Harness NextGen, you will need to replace ${infra.kubernetes.infraId} with the new expression.
In Harness NextGen, a similar expression <+INFRA_KEY> is available for defining release names. However, it's crucial to understand that these expressions will resolve to completely different values compared to the expressions used in Harness FirstGen.
Is it possible to have drop down options for multiple input?
You can make the variable as Input and define multiple allowed values by selecting checkbox Allowed values
How to Make a Pipeline Failure or Step Failure if some condition is not passed In Bash script?
You can set below in script set -e - Exit immediately when a command fails, or you can set exit code to non zero if certain conditions match and that should fail the step.
Is there an easy way to see all the recent deployments of that workflow that have run?
You can use deployment filter and select the workflow and time range and you will able to see all the deployment for that workflow within that time range
Is there any option to execute HTTP steps on the target environment?
As HTTP step is meant to connect over http protocol, delegate can initiate http sessions and get the response as per request setup so using target environment will not help.
WINRM Download artifact is not working in NG, after setting correct environment variables(HARNESS_ENV_PROXY and HTTP_PROXY).
Please check the delegate version used as this feature was released with delegate version 791xx and make sure in console logs you are able to see Using HTTP_PROXY environment variable.
Error with release name too long
In the deployment logs in Harness you may get an error similar to this:
6m11s Warning FailedCreate statefulset/release-xxx-xxx create Pod release-xxx-xxx-0 in StatefulSet release-xxx-xxx failed error: Pod "release-xxx-xxx-0" is invalid: metadata.labels: Invalid value: "release-xxx-xxx-xxx": must be no more than 63 characters
This is an error coming from the kubernetes cluster stating that the release name is too long. This can be adjusted in Environments > click Name of the Environment in Question > Infrastructure Definitions > click Name of the Infrastructure Definition in Question > scroll down > expand Advanced > modify the Release name to be something shorter
Pipeline GitHub trigger support for project variable reference?
This is not possible as trigger yaml is independent of the pipeline yaml and the trigger will not be aware of the expression output.
Procedure to take backup of the Services
We do not have any backup ability for services out of the box but you can take the backup of service yamls and use them later for creating service if there is any issue with the service.
Harness FirstGen Graphql API to create Harness pipelines in a specific application
We do not have a way to create a new pipeline using Graphql in FirstGen. However, we do support API to create Harness pipelines in NextGen.
How can only set of user able to approve the deployment?
You can create a user group of specific users and specify the same user group in the Approval stage so only those users can able to approve the execution.
For reference: Select Approvers
How Kubernetes Pruning option work during the deployment?
If you have enabled the Kubernetes Pruning in your deployment. In that case, it will remove any resources that were present in an old manifest but are no longer present in the manifest used for the current deployment.
For reference: Prune Kubernetes resources
How release: {{ .Release.Name }} will help in steady state check in helm deployment?
We perform a pod fetch based on this label, which allows us to show deployed pods in the step output and also track the same for instance sync. If we don't add these, both won't work as expected.
For reference: Steady state check
Where we need to add label release: {{ .Release.Name }}?
For any manifest object which creates the pod, you have to add this label in its spec. Adding it in Service, Deployment, StatefulSet and DaemonSet should be enough.
What does the release name mean in the Infrastructure?
The release name is used to create a harness release history object, which contains some metadata about the workloads. This helps us perform the steady state check.
I have a pipeline in CG that has a variable of ${artifact.buildNo} in a command and same variable is not working in NextGen.
You can use artifact.tag in NG , which is equivalent to artifact.buildNo from CG, you can find more details around mapping as below: https://developer.harness.io/docs/platform/variables-and-expressions/harness-variables/#migrating-firstgen-expressions-to-nextgen
Is it possible to apply Notification Rule on Environment level for workflow failure/success
Workflow Notification strategy we can only interpret below field so all the notification rule will be applied on workflow level Condition,Scope, User Group
Does Harness support cloning "Instance type requirements" and "Instance purchase options" from base ASG in CG
No, We do not support copying of these properties in CG. All of them come under the MixedInstancesPolicy property of an Auto Scaling group which we do not copy from base ASG. Allocation strategies & Instance purchase options come under the InstancesDistribution property of MixedInstancesPolicy.
Would it be possible to be able to modify the looping stage runs inside of pipeline execution
You can use Matrix strategies, there you can use labels: To use the matrix labels naming strategy, do the following:
- In Harness, select Account Settings.
- Select Account Resources, then select Pipeline.
- Set Enable Matrix Labels By Name to true.
- Select Save.
I have a placmenetStrategy defined but I don't see it reflected in the task.
As placmenetStrategy can be defined in task definition as well as in service definition. Harness picks placmenetStrategy from service definition, so please make sure its added under service definition.
When querying the Harness Approval API, the Approval Details are returning with message No Approval found for execution
The api will only return Approval details if there are any approval step pending for approval, If there are no such executions currently than its expected to return No Approval found for execution