Org-Level EDH (AWS - CloudTrail Mode)

In addition to existing Event-Driven Harvesting (EDH) capabilities, InsightCloudSec includes support for Org-level (CloudTrail Mode) EDH. In the regular deployment, EDH augments standard harvesting methods, pulling data from AWS CloudWatch Events and AWS CloudTrail into a central Eventbus for InsightCloudSec's consumption.

More information on standard EDH, including a detailed diagram, is included on the Event-Driven Harvesting (AWS) page. Org-Level EDH behaves in the same way as the existing EDH but with two key differences:

• Org-Level EDH retrieves information in approximately 10-15 minute intervals.
  • This interval is imposed by AWS at the cloud-level and is determined by the collection and reporting times associated with the configuration.
• Org-Level EDH does not require additional manual configuration when new cloud accounts are added.
  • Any new cloud accounts added to your overall footprint are automatically discovered and included in your EDH setup. There are no complex configuration requirements to connect new cloud accounts and ensure the associated resources are harvested.

📘

Getting Started with Org-Level EDH

Event-Driven Harvesting is an advanced and complex feature. Before getting started, we recommend reaching out to us through the Customer Support Portal to ensure that you have the best possible experience getting started with EDH.

Deployment Diagram

Prerequisites

• A functioning InsightCloudSec Platform
• AWS Organization(s) added to InsightCloudSec
• InsightCloudSec Admin permissions
• Basic familiarity with Terraform and the general capabilities.
  • If you are not familiar with Terraform we are happy to assist with this configuration, reach out to us through the Customer Support Portal.
• Required Terraform template(s) (provided below)
  • For customers that do not anticipate using Terraform to deploy, reach out to us for assistance in creating the files required to establish this configuration
• Admin access to your AWS Master/Payer Account.
  • We strongly recommend installing Org-Level EDH inside of the master/payer AWS account and not in the same account that you are using to run your InsightCloudSec platform (this avoids conflicting issues around permissions during configuration)

📘

DivvyCloud vs. InsightCloudSec

Some examples, database values, and other items may refer to DivvyCloud vs. InsightCloudSec. The behavior and performance is the same, just ensure that you are consistent in naming your values.

Required Permissions

🚧

The role applying the Terraform template requires the following permissions:

https://s3.amazonaws.com/get.divvycloud.com/prodserv/divvycloud-deployment-native/aws/iam/DivvyCloud-EDH-CloudTrail-Org-Deployment.json

Required Templates

🚧

DivvyCloud EDH Org CloudTrail via Terraform

https://s3.amazonaws.com/get.divvycloud.com/prodserv/divvycloud-tf/divvycloud-org-cloudtrail-edh-tf.zip

Configuring Org-Level EDH

Terraform Setup (v0.12.x required)

For this part of the deployment we assume that you have downloaded the required templates and have the appropriate permissions to execute the required Terraform commands.

🚧

Variables.tf

The template below is provided as an example and will need to be updated to reflect the configuration for your specific needs.

// AWS master/payer account
variable "aws_account_id" {
  type    = string
  default = "XXXXXXXXXXXX"
}

// AWS target region
variable "region" {
  type    = string
  default = "us-east-1"
}

// Existing harvesting role in master/payer account
variable "existing_harvesting_role" {
  type    = string
  default = "DivvyCloud-Standard-Role"
}

// If set to false, CloudTrail and associated S3 bucket are not created
variable "create_cloudtrail" {
  type    = bool
  default = false
}

// If `create_cloudtrail` is false, specify existing CloudTrail S3 bucket and associated SNS topic name
variable "existing_cloudtrail_bucket" {
  type    = string
  default = "arn:aws:s3:::EXISTING-CLOUDTRAIL-BUCKET-NAME"
}

// Update `arn:aws:sns:REGION:ACCOUNT-ID:DivvyCloud-EDH-Org-CloudTrail` in provided IAM policy to reference existing SNS topic
variable "existing_cloudtrail_topic" {
  type    = string
  default = "EXISTING-SNS-TOPIC-NAME"
}

InsightCloudSec Setup

Refer to the steps below to configure Org-Level EDH in InsightCloudSec. In general, we do not recommend changing an existing setup or creating a new setup to use Org-Level EDH that would compete with an existing setup.

1. Navigate to "Cloud --> Clouds" and select "EDH Consumers". Click "EDH Configuration", then click "AWS SQS Consumer".

2. Complete this form by selecting the Consumer Account from the drop-down list. You will also need to do the following:

• Provide the ARN for the SQS Consumer First-In-First-Out (FIFO) queue (not the capture queue).
• Select the "CloudTrail via Lambda" from the Producer Type drop-down menu.
• Click "Configure". Note: The Consumer will remain in a pending state while setup is in progress.