Configure audit streaming

info

Currently, this feature is in Beta and behind the feature flag PL_AUDIT_LOG_STREAMING_ENABLED. Contact Harness Support to enable the feature.

You can configure a streaming destination in Harness to send audit log data to another location for processing. Integrating audit data with other Security Incident and Event Management (SIEM) tools lets you do the following:

Trigger alerts for specific events.
Create views of audit data.
Perform anomaly detection.
Store more than 2 years audit data, which is the maximum amount of data that Harness keeps for your accounts.
Keep your resources and data protected, and ensure compliance with regulations.

Harness streams the audit events to your chosen SIEM tool as structured JSON.

Important

Streaming destinations receive all audit event data, which could include sensitive information. Make sure you trust the streaming destination.
Harness supports streaming to Amazon S3.
You can add multiple streaming destinations in Harness.

Add a streaming destination

To add a streaming destination in Harness:

In your Harness account, select Account Settings.
Select Audit Trail and then select Audit Log Streaming.
Select New Streaming Destination to open the configuration settings.
Enter a name for the streaming destination.
Harness populates the destination's Id automatically. You can retain or change the Id.
Enter a description and tag(s) for the streaming destination.
Select Continue.

Configure the streaming connector

Select Amazon S3.
In Select Connector, select an existing AWS Cloud Provider connector or create a new one.
Go to Add an AWS connector for steps to create a new AWS Cloud Provider connector.
Select Apply Selected.
In Amazon S3 Bucket, enter the bucket name.
Harness writes all the streaming records to this destination.
Select Save and Continue.
After the connection test is successful, select Finish.
info
Harness does not consider AWS buckets while testing the connection. Also, it tests the connector used without testing the bucket.
The streaming destination gets configured and appears in the list of destinations under Audit Log Streaming. By default the destination is inactive.

Activate or deactivate streaming

To activate streaming to this destination, toggle the status to Active besides the streaming destination.
Harness starts writing audit logs after the destination is activated.
Harness streams audit logs to the destination every 30 minutes.
You can pause audit streaming, preventing any new audit events from being streamed to the configured endpoint by setting the status to Inactive.
When you reactivate the streaming destination, Harness starts streaming the audit logs from the point where it was paused.

Update audit stream

You can change the audit stream configuration by clicking three dots beside the stream destination. This opens a pop-up menu with the following options:

Edit: Select a different streaming destination or make changes to the existing destination.
Delete: Delete the audit stream destination.

Amazon S3 audit file details

Here is an example of an audit stream file in one of the Amazon S3 buckets.

This file has a list of audit events in JSON format.

Following are the key points about the naming convention of the audit stream file:

There are three timestamps in the file name: <t1>_<t2>_<t3>.
<t1> and <t2> indicate the time range of audit events in the file. This time range is provided for information only and is not always accurate. The timestamp can also be out of range if there is a delay in capturing the event.
<t3> indicates the time when the file was written.

info

Harness recommends not building any business logic based on the file name.

Payload schema

Streamed audit events have a predictable schema in the body of the response.

Field	Description	Is required
auditEventId	Unique ID for the audit event.	Required
auditEventAuthor	Principal attached with audit event.	Required
auditModule	Module for which the audit event is generated.	Required
auditResource	Resource audited.	Required
auditResourceScope	Scope of the audited resource.	Required
auditAction	Action on the audited resource.	Required
auditEventTime	Date and time of the event.	Required
auditHttpRequestInfo	Details of the HTTP request.	Optional
auditEventMetadata	Additional details required for streaming the audit log.	Optional

JSON payload

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "auditEventId": {
      "type": "string"
      "description":"Unique ID for each audit event"
    },
    "auditEventAuthor": {
      "type": "object",
      "properties": {
        "principal": {
          "type": "object",
          "properties": {
            "type": {
              "type": "string"
            },
            "identifier": {
              "type": "string"
            },
            "email": {
              "type": "string"
            }
          },
          "required": [
            "type",
            "identifier",
          ]
        }
      },
      "required": [
        "principal"
      ]
      "description":"Information about Author of the audit event"
    },
    "auditModule": {
      "type": "string"
      "description":"Information about Module of audit event origin"
    },
    "auditResource": {
      "type": "object",
      "properties": {
        "type": {
          "type": "string"
        },
        "identifier": {
          "type": "string"
        }
      },
      "required": [
        "type",
        "identifier"
      ]
      "description":"Information about resource for which Audit event was generated"
    },
    "auditResourceScope": {
      "type": "object",
      "properties": {
        "accountIdentifier": {
          "type": "string"
        },
        "orgIdentifier": {
          "type": "string"
        },
        "projectIdentifier": {
          "type": "string"
        }
      },
      "required": [
        "accountIdentifier",
      ]
      "description":"Information about scope of the resource in Harness"
    },
    "auditAction": {
      "type": "string"
      "description":"Action CREATE,UPDATE,DELETE,TRIGGERED,ABORTED,FAILED , Not exhaustive list of events"
    },
    "auditHttpRequestInfo": {
      "type": "object",
      "properties": {
        "requestMethod": {
          "type": "string"
        }
        "clientIP": {
          "type": "string"
        }
      },
      "required": [
        "requestMethod",
        "clientIP"
      ]
      "description":"Information about HTTP Request"
    },
    "auditEventTime": {
      "type": "string"
      "description":"Time of auditEvent in miliiseconds"
    },
    "auditEventMetadata": {
      "type": "object"
      "description":"No fixed schema , Additional meta about audit event"
    }
  },
  "required": [
    "auditEventId",
    "auditEventAuthor",
    "auditModule",
    "auditResource",
    "auditResourceScope",
    "auditAction",
    "auditEventTime",
  ]
}

Important​

Add a streaming destination​

Configure the streaming connector​

Activate or deactivate streaming​

Update audit stream​

Amazon S3 audit file details​

Payload schema​

JSON payload​