EDH - Fully Automatic Setup Option (AWS)

Detailed Instructions for a Fully-Automatic Approach to AWS Event-Driven Harvesting

This page includes step-by-step instruction to complete the InsightCloudSec setup for Event-Driven Harvesting (EDH) using a "fully automatic" approach.

Using this approach means that InsightCloudSec will automatically configure both the AWS IAM and CloudWatch resources.

  • You are required to provide the appropriate permissions, and InsightCloudSec creates the IAM Eventbus role/policy.
  • InsightCloudSec will also create and update CloudWatch rules during any platform upgrade.

For questions on this specific setup, assistance with this process for AWS GovCloud, or other EDH-related questions, reach out to us through the Customer Support Portal. For details on other EDH setup options, check out the main Event-Driven Harvesting (AWS) page.

Setup for EDH in AWS

Configure Consumer Account

Consumer accounts are tasked with ingesting events from other accounts and also producing their own events (i.e., consumers are both producers and consumers).

📘

Opt-In Regions

If you are interested in disabling opt-in regions from harvesting, you can include that configuration when you establish the settings within your consumer(s). If you need assistance, contact us through the Customer Support Portal.

Refer to additional documentation on Opt-In Regions here.

1. Login to your AWS Console and open the IAM dashboard.

2. Open the "Policies" section and create a new policy using the "JSON" tab to assign to the existing user/role that DivvyCloud uses to access the consumer account.

  • This policy allows the InsightCloudSec user to read and write the appropriate event data.
  • In this case, our user is 'DivvyCloud-PowerUser' and our policy name is 'DivvyCloud-EDH-Consumer'.

In addition, you can review the AWS IAM Policies for details.

Here is a sample of what the policy looks like:

📘

InsightCloudSec vs. DivvyCloud

Note that some examples, database values, and other items may refer to DivvyCloud vs. InsightCloudSec. The behavior and performance is the same - just ensure that you are consistent in naming your values.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "events:DeleteRule",
                "events:DescribeRule",
                "events:PutRule",
                "events:PutTargets",
                "events:RemoveTargets"
            ],
            "Resource": "arn:aws:events:*:CONSUMER-ACCT-ID:rule/divvycloud*"
        },
        {

            "Effect": "Allow",
            "Action": [
                "sqs:CreateQueue",
                "sqs:ListQueues",
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage"
            ],
            "Resource": [
                "arn:aws:sqs:*:CONSUMER-ACCT-ID:divvycloud-event-aggregator-*.fifo"
            ]
        },
        {
            "Action": [
                "organizations:DescribeOrganization"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

And here is the Create Policy interface in the AWS Console.

22042204

Create Policy in AWS

3. Click on the "Review Policy" option to add the Name, Description, and to review the Summary.

  • In this example we've named our policy 'DivvyCloud-EDH-Consumer' to identify it as the 'consumer' policy.
12741274

Verify Consumer Policy

4. Locate your Power User. Open the permissions tab to attach the newly created consumer policy to your Power User (e.g., 'DivvyCloud-PowerUser' below).

17211721

Attach the EDH Consumer Policy

❗️

Configuring CloudTrail

If you already have at least one CloudTrail enabled with at least Write events being logged, you do not need to create an additional trail for this and can skip to Step 7.*

5. With our consumer policies in place, you will need to configure CloudTrail. In the AWS console, open "Services" --> "CloudTrail."

  • In this example the trail is named 'DivvyCloud-EDH'.
27962796

CloudTrail Dashboard

6. Click on "Trails" --> "Create Trail" and complete the details based on your organization's requirements (e.g., all regions vs. specific, etc.).

11091109

Create a New Cloud Trail

7. Navigate to "CloudWatch --> Events --> Event Buses." Click on the "Add permission" button and enable Organization access to default event bus.

12091209

Event Bus Permissions

8. If organization permissions do not already exist, add them using your organization ID.

13801380

Add Permission to Event Bus

Configure Producer Account

1. Login to your AWS Console and open the IAM dashboard.

2. Open the "Policies" section and create a new policy using the "JSON" tab. This new policy will be assigned to the existing user/role that InsightCloudSec uses to access the producer account.

  • This policy allows the user to write the appropriate event data.
  • In this case, our user is 'DivvyCloud-PowerUser' and our policy name is 'DivvyCloud-EDH-Producer'.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "events:DeleteRule",
                "events:DescribeRule",
                "events:PutRule",
                "events:PutTargets",
                "events:RemoveTargets"
            ],
            "Resource": "arn:aws:events:*:*:rule/divvycloud-events*"
        },
        {
            "Action": [
                "organizations:DescribeOrganization"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

3. Create a second policy that will allow the automated creation of roles and permissions and allow the producers to feed our consumer.

  • This policy will be named 'DivvyCloud-EDH-AutoRole’.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:AttachRolePolicy",
                "iam:PassRole"
            ],
            "Resource": "arn:aws:iam::*:role/service-role/event-driven-harvest/divvycloud-eventbus-role"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreatePolicy",
                "iam:CreatePolicyVersion",
                "iam:DeletePolicyVersion"
            ],
            "Resource": "arn:aws:iam::*:policy/service-role/event-driven-harvest/divvycloud-eventbus-policy"
        }

    ]
}

4. InsightCloudSec will automatically create the divvycloud-eventbus-role and associated policy.

5. Now that you have created your producer policies, you will need to attach them to your DivvyCloud-PowerUser.

  • Click the "Add permissions" button and select your previously created policies.
24882488

Attach Policies

❗️

Enabling CloudTrail

If you already have at least one CloudTrail enabled with at least Write events being logged, you do not need to create an additional trail for this and can skip to Step 7.

6. With your producer policies in place, you can now configure CloudTrail. Navigate to "Services" --> "CloudTrail."

  • In this example, the trail is named ‘DivvyCloud-EDH’.
27962796

Configure CloudTrail

7. Click on "Trails" --> "Create Trail" and complete the details based on your organization's requirements (e.g., all regions vs. specific, etc.).

11091109

Create CloudTrail

Setup EDH in InsightCloudSec

After completing the AWS configuration, you are now ready to complete the configuration of EDH inside of InsightCloudSec.

1. Navigate to "Cloud --> Clouds" and select "Event-Driven Harvesting".

27902790

Event-Driven Harvesting Landing Page

2. If no EDH has been set up previously, select "Begin" and then confirm EDH is enabled. Otherwise, go to Step 3.

3. From the Event-Driven Harvesting section, select "Consumer" and then select the "Add Consumer" button.

24402440

EDH - Add Consumer

4. Complete this form by selecting the Consumer Account from the drop-down list. You will also need to do the following:

  • Use the field to disable EDH in specific regions. By default, InsightCloudSec will not attempt to set up EDH in AWS opt-in regions. You can override this default by removing the regions from the form above. Ensure that you have configured these regions appropriately by visiting this link.
  • Include an encryption key if you are interested in encrypting your simple queue service (SQS) queues.
12341234

Add Event Consumer

  • The Consumer will remain in a pending state while setup is in progress.
16781678

Consumer "pending"

❗️

Enabled Note

Do not add producers until the consumer displays as enabled.

24002400

Consumer "Enabled"

5. From the Event-Driven Harvesting tab, select the Producers subtab and click "Add Producer."

24502450

EDH - Add Producer

6. Complete the "Enable Producer" form with the details from the AWS setup completed earlier.

561561

EDH - Add Producer Form

Note: If you have questions about the policy associated with enabling any options on this form, contact us through the Customer Support Portal.

  • You can choose to "Automatically update enabled resource types to use all resource types." This will provide event support for all future resource types as they become supported by InsightCloudSec(this is recommended).
  • You can also set EDH to "Automatically Provision IAM resources" and "Automatically Provision Cloudwatch resources" (These are enabled be default and we recommend leaving this option enabled as part of the "Fully Automatic" setup.)

7. After adding your producer, it may remain in the pending state for several minutes while setup is in progress. (Note this will not refresh automatically; you will need to manually reload the browser.)

16861686

EDH - Add Producer Pending

8. Once the configuration is complete, the producer account’s status will be labeled enabled.

📘

Enabling Producer

If there are any issues enabling your producer, the status will read "error" and you’ll be able to click the read icon to see related errors.

9. If you added a heavily used account as your producer, you should see events within a minute or so.

  • If not, you can create and delete a test user to generate events. In this case, we’ll create a user named ‘EDH-TEST-DELETE-ME’ to generate events and verify your configuration.
17071707

EDH - Test User


Did this page help you?