AWS Event-Driven Harvesting

InsightCloudSec has the capability to augment standard polling-based harvesting along with Event-Driven Harvesting (EDH). EDH pulls data from AWS CloudWatch Events and AWS CloudTrail into a central Eventbus for InsightCloudSec's consumption.

This dynamic approach to data collection not only improves InsightCloudSec's cadence for providing resource visibility and opportunities for remediation, but also enriches the data with lifecycle changes that enable auditing capabilities. With EDH-provided data, identifying how a resource entered a noncompliant state becomes much easier at scale.

Prerequisites

📘

New Onboarding

As of InsightCloudSec v. 23.4.11, the new universal onboarding experience for AWS accounts uses CloudFormation Templates (CFTs) to automatically provision relevant accounts with the necessary policies and roles. This means it is easiest to perform EDH configuration while onboarding an account/organization. Review AWS Cloud - Onboarding for more information.

Before you get started with EDH, you need to ensure you have the following:

• A functioning InsightCloudSec platform with the appropriate Admin (Org or Domain) permissions
• A basic understanding of the relevant AWS services
• Appropriate AWS IAM permissions
• A strategy around the behaviors you want to use to configure EDH

If you have questions or encounter issues, reach out to us through the the Customer Support Portal.

Understanding EDH Concepts

EDH relies on a relationship between a Producer and Consumer.

• A Producer is an AWS account that is configured to forward cloud events to another account.
• A Consumer is a central account that ingests events from one or more Producers. Consumers also create their own events and are essentially Producers as well.

EDH uses an Amazon Simple Queue Service (SQS) topic to retrieve the consumed events on a region-by-region basis. Customers are encouraged to set up at least two Consumers: one to consume data from development environments and another to consume data from the production environment.

EDH Supported Resources

For a complete and up-to-date list of support AWS resources refer to the EDH - Supported Resources (AWS) page.

Configuration Options

📘

EDH Support Details

InsightCloudSec includes:

• Support for EDH in multiple environments with both environments monitoring the same accounts as Producers, perhaps one as ReadOnly and the other as PowerUser. If you plan on enabling EDH in multiple environments, using the same accounts as Producers, there is some customization required in your deployment. The "fix" is easy, but best done with help. Contact your CSM or support.
• Support for AWS-GovCloud is configured and behaves in the same manner as EDH in non-AWS GovCloud environments. Supported resources are also the same, except where a resource is not offered by AWS within GovCloud. If you have specific support questions or concerns reach out to us.

Check out EDH - Supported Resources (AWS) for full details.

Contact us through the Customer Support Portal

One additional advantage with EDH resides in the configuration options available. Our current platform includes support for two possible configuration routes, as detailed below.

Fully Automatic Setup for Producer

Using this approach InsightCloudSec will automatically configure both the AWS IAM and CloudWatch resources. You will be required to provide the appropriate permissions, and InsightCloudSec creates the IAM Eventbus role/policy. InsightCloudSec will also create and update CloudWatch rules during any platform upgrade.

• Refer to EDH - Fully Automatic Setup Option for details.

Manual Setup for IAM Option

Using this approach InsightCloudSec will not automatically provision IAM resources; however, it will set up your CloudWatch resources. You will be required to manually create the Eventbus role/policy. For customers who do not want to use InsightCloudSec's exact paths or names for the role/policy, we can work with you on alternatives. Reach out to us through the Customer Support Portalbefore you get started using this approach.

• Refer to EDH - Manual IAM Setup Option for details.

Org-Level (CloudTrail EDH)

In addition to standard Event-Driven Harvesting (EDH) capabilities, InsightCloudSec also includes support for Org-level (CloudTrail Mode) EDH. Org-Level EDH behaves in the same way as the existing EDH but with two key differences, speed and maintenance. Org-Level EDH offers an implementation of EDH that provides data in 10-15 minute intervals with far less configuration and maintenance overhead.

• Refer to Org-Level EDH (CloudTrail Mode) for details.