Self-Hosted AWS Cloud Setup (Organizations)

Integrating an AWS Organization with the Self-Hosted Version of InsightCloudSec

Once your InsightCloudSec instance is up and running the first thing you'll want to do is begin integrating your AWS organization(s) to take advantage of the security insights that apply to your entire cloud footprint. If you have any issues or questions with this setup, reach out to the support team through the Customer Support Portal.

🚧

Deployment Method Assumptions

These instructions assume that you have deployed InsightCloudSec to AWS using the recommended production deployment method: AWS - EC2 or ECS Fargate - Terraform. If your deployment method was different, reach out to the support team through the Customer Support Portal.

If you need to add a single account to a self-hosted version of InsightCloudSec, review Self-Hosted AWS Cloud Setup (Single Cloud).

Setup Overview

For InsightCloudSec to securely access the information contained within your AWS Organization and its member accounts, you'll need to create and setup some roles, policies, and trust relationships. If you're not familiar with this process we recommend that you review AWS' IAM documentation for more information on these concepts. In addition, this setup relies on CloudFormation Templates to achieve proper harvesting for InsightCloudSec.

This setup will require you to complete the following steps within your AWS Organization:

  • Step 1: Master Account Setup - Organization Harvesting -- Create a policy in the Organization's master account for harvesting organization data (roots, accounts, organizational units, and tags for member accounts). This policy will be attached to a role (with optional External ID) that designates your unique InsightCloudSec role a trusted entity.

  • Step 2: Master Account Setup - Standard Harvesting -- Create a policy in the Organization's master account for harvesting the various AWS resources in use. This policy will be attached to a role (with optional External ID) that designates your unique InsightCloudSec role a trusted entity. Note: The role will be re-created in each member account as well (same name, allowed session duration, and external ID).

  • Step 3: Member Account Setup -- Create a policy for harvesting the various AWS resources in use for each member account. This policy will be attached to a role (with optional External ID) that designates your unique InsightCloudSec role a trusted entity. Note: The role will have the same name, allowed session duration, and external ID as the standard harvesting role in the master account.

  • Step 4: InsightCloudSec Configuration -- Setup AWS Organization harvesting within InsightCloudSec and begin receiving resource data.

The diagram below outlines the details of the setup components.

AWS Organizations Setup Overview for Self-Hosted versions of InsightCloudSecAWS Organizations Setup Overview for Self-Hosted versions of InsightCloudSec

AWS Organizations Setup Overview for Self-Hosted versions of InsightCloudSec

Prerequisites

📘

Value Names (DivvyCloud vs. InsightCloudSec)

Some components use our former product name (DivvyCloud vs. InsightCloudSec). Updates to the naming of these components will be communicated when changes are made, but note that the name difference does not affect setup or functionality within the product.

Before you configure anything in your AWS environment, you'll need the following:

  • Admin access to your AWS organization and the member accounts you want to harvest
  • The unique Amazon Resource Name (ARN) for your InsightCloudSec instance. Your unique InsightCloudSec ARN should look something like this: arn:aws:iam::123456789123:role/DivvyCloud-Install-Role, with the 12-digit account ID value being replaced with the ID of the account where InsightCloudSec is hosted
    • Note: If you followed the AWS - EC2 or ECS Fargate - Terraform deployment instructions, this role will be created automatically alongside the InsightCloudSec instance inside the desired account.
  • Domain Admin permissions within InsightCloudSec
  • InsightCloudSec IAM CloudFormation Templates (CFTs) (see below) and the permissions to use & implement CFTs

Note: We recommend that you review the documentation on AWS Additional Configuration, particularly the additional steps to support Opt-in regions.

CloudFormation Templates

🚧

Using CFTs

If you are unwilling or unable to use the CFTs required for setup, contact the support team through the Customer Support Portal.

Our team maintains the following templates to help automate policy and role setup across your master and member accounts:

Service Control Policies - Implementation & Known Issues

Within AWS, a Service Control Policy (SCPs) is a type of policy used to manage permissions at the organization level. Per AWS, "SCPs offer central control over the maximum available permissions for all accounts in your organization. SCPs help you to ensure your accounts stay within your organization’s access control guidelines.

  • Read more from AWS here on using SCPs.

Many organizations choose to implement SCPs for security purposes, while also providing the permissions required for InsightCloudSec to function with full visibility into their AWS accounts.

It is important to note that when implemented, many SCPs will block the required permissions InsightCloudSec needs to operate - even when permissions have been explicitly granted via roles and policies.

  • If this scenario applies to your environment, you will need to revise your SCP to ensure you have permitted the required InsightCloudSec permissions.
  • This is normal SCP behavior as they are organization-wide policies. SCPs are configured within AWS to supersede other types of permissions.
  • You can review our AWS IAM Policies for details, otherwise refer to the AWS documentation on SCPs.

Another item to note - In more limited cases an SCP in conflict with an existing role/policy can also result in visibility issues (noted below).

🚧

Warnings with False Positives - Known AWS Service Control Policy Issue

When viewing details on the Clouds Listing page, InsightCloudSec may provide false positive "Warnings" around missing permissions. In some scenarios the permissions are granted within a Service Control Policy (SCP) but falsely report as denied.

This scenario is the result of a known issue within AWS where if an Organization has an SCP with conditions based on global keys (e.g. aws:PrincipalArn) the IAM Policy Simulator results are not accurate because it does not have context with the global keys.

If you have verified that your resources are being harvested as expected you can safely disregard these warnings. If you're not sure or otherwise have remaining questions or concerns, contact us through the Customer Support Portal.

Step 1: Master Account Setup - Organization Harvesting

Setting up your AWS Master account for organization data harvesting prior to integration with InsightCloudSec is easy. It requires implementing a CFT that will create a single policy and attach it to a role that will harvest organizational data. We recommend reviewing the Organization Master Account Role Policy that you'll be creating before getting started.

1. Login as an Admin to your Master AWS account and access the CloudFormation service.

  • This service can be found on the Services main page under Management & Governance. You can also enter "CloudFormation" into the search bar.
  • Once at the CloudFormation dashboard, click "Stacks" in the left-hand menu.

2. In the top right corner of the Stacks table, click "Create stack --> With new resources (standard)".

  • At the Import overview page, click "Next".

3. Specify the Master Account Organization Role CFT URL.

  • Click "Template is ready".
  • Click "Amazon S3 URL".
  • Input the Harvest Organization Role CFT URL: https://s3.amazonaws.com/get.divvycloud.com/prodserv/divvycloud-deployment-native/aws/cft/iam/DivvyCloud-AWS-IAM-Harvest-Role-Org-CFT.yaml
  • Click "Next".
Uploading the Organization Harvest Role CFTUploading the Organization Harvest Role CFT

Uploading the Organization Harvest Role CFT

4. Specify stack details.

  • Provide a name for the stack.
  • Optionally, update the default role and/or policy name.
  • Input the ARN for your InsightCloudSec instance (contains your unique AWS account ID and role name).
    • Note: If you followed the AWS - EC2 or ECS Fargate - Terraform deployment instructions, this role will be created automatically alongside the InsightCloudSec instance inside the desired account.
  • Click "Next".
Specifying the Organization Harvest Role CFT Stack DetailsSpecifying the Organization Harvest Role CFT Stack Details

Specifying the Organization Harvest Role CFT Stack Details

5. Configure stack options.

  • Optionally, provide tags to help identify the stack, provide an existing IAM role to provide explicit permissions to the stack, update the stack failure, and/or update the advanced options.
  • Click "Next".

6. Review and create the stack.

  • Review the stack's configuration to ensure everything is accurate.
  • Acknowledge the warning about IAM capabilities toward the bottom of the page.
  • Click "Create stack".
  • Verify the stack is created successfully.

Step 2: Master Account Setup - Standard Harvesting

Your AWS Master account will also need a standard harvesting role/policy to ensure proper integration with InsightCloudSec. The relevant CFT for this setup will configure the Master account for an additional policy and role.

1. Login as an Admin to your Master AWS account and access the CloudFormation service.

  • This service can be found on the Services main page under Management & Governance. You can also enter "CloudFormation" into the search bar.
  • Once at the CloudFormation dashboard, click "Stacks" in the left-hand menu.

2. In the top right corner of the Stacks table, click "Create stack --> With new resources (standard)".

  • At the Import overview page, click "Next".

3. Configure the template.

  • Click "Template is ready".
  • Click "Amazon S3 URL".
  • Input the Harvest Member Role CFT URL: https://s3.amazonaws.com/get.divvycloud.com/prodserv/divvycloud-deployment-native/aws/cft/iam/DivvyCloud-AWS-IAM-Harvest-Role-Member-CFT.yaml
  • Click "Next".

4. Specify stack details.

  • Provide a name for the stack.
  • Edit the parameters:
    • Select the "Standard-Managed (read only, AWS managed)" Harvest role type. Review AWS-Managed Supplemental Policy for more information about this policy.
    • Optionally, update the default role and/or policy name.
    • Input the ARN (same as used in the stack above) for your InsightCloudSec instance (contains your unique AWS account ID and role name).
      • Note: If you followed the AWS - EC2 or ECS Fargate - Terraform deployment instructions, this role will be created automatically alongside the InsightCloudSec instance inside the desired account.
    • Optionally, select "Yes" to create an external ID, then provide an external ID
  • Click "Next".

5. Configure stack options.

  • Optionally, provide tags to help identify the stack, provide an existing IAM role to provide explicit permissions to the stack, update the stack failure, and/or update the advanced options.
  • Click "Next".

6. Review and create the stack.

  • Review the stack's configuration to ensure everything is accurate.
  • Acknowledge the warning about IAM capabilities toward the bottom of the page.
  • Click "Create stack".
  • Verify the stack is created successfully.

Step 3: Member Account Setup

Setting up proper harvesting of your accounts and their associated resources is straightforward: each account that contains resource data you want to harvest for InsightCloudSec will need access to the same harvesting role (Role ARN, external ID, etc.) with the same policy attached. The relevant CFT for this setup will configure all provided accounts accordingly.

🚧

Prerequisites

Before proceeding with the instructions below, you'll need the following information from the role you created in step 2:

  • Role name
  • Policy name
  • External ID (if applicable)

1. Login as an Admin to your Master AWS account and access the CloudFormation service.

  • Ensure you're logged into the Organization master account so the StackSet can be run from there to access all the member accounts you wish to harvest.
  • This service can be found on the Services main page under Management & Governance. You can also enter "CloudFormation" into the search bar.
  • Once at the CloudFormation dashboard, click "StackSets" in the left-hand menu.

2. In the top right corner of the Stacks table, click "Create StackSet".

3. Configure the template.

  • Optionally, provide an IAM admin role to perform all the operations in the StackSet within your account(s) and adjust the IAM execution role name as necessary.
  • Click "Template is ready".
  • Click "Amazon S3 URL".
  • Input the Harvest Member Role CFT URL: https://s3.amazonaws.com/get.divvycloud.com/prodserv/divvycloud-deployment-native/aws/cft/iam/DivvyCloud-AWS-IAM-Harvest-Role-Member-CFT.yaml
  • Click "Next".
Uploading the Member Harvest Role StackSetUploading the Member Harvest Role StackSet

Uploading the Member Harvest Role StackSet

4. Specify stack details.

  • Provide a name for the StackSet.
  • Optionally, update the StackSet description.
  • Edit the parameters:
    • Note: The parameters should match the role you created in step 2.
    • Select the "Standard-Managed (read only, AWS managed)" Harvest role type. Review AWS-Managed Supplemental Policy for more information about this policy.
    • Optionally, update the default role and/or policy name.
    • Input the ARN (same as used in the stack above) for your InsightCloudSec instance (contains your unique AWS account ID and role name).
      • Note: If you followed the AWS - EC2 or ECS Fargate - Terraform deployment instructions, this role will be created automatically alongside the InsightCloudSec instance inside the desired account.
    • Optionally, select "Yes" to create an external ID, then provide an external ID
  • Click "Next".
Specifying the Member Harvest Role DetailsSpecifying the Member Harvest Role Details

Specifying the Member Harvest Role Details

5. Configure stack options.

  • Optionally, provide tags to help identify the stack and/or update the execution configuration.
  • Click "Next".

6. Set deployment options.

  • Click "Deploy new stacks".
  • Choose to either deploy to accounts or organizational units, then provide a comma-delimited list of accounts or organizational units (or upload a CSV file).
  • Select us-east-1 to deploy the stack. Note: Currently only single-region role deployment is supported. Additionally IAM resources are global so the region doesn't matter here.
  • Optionally, update the deployment options.
  • Click "Next".

7. Review and create the stack.

  • Review the StackSet's configuration to ensure everything is accurate.
  • Acknowledge the warning about IAM capabilities toward the bottom of the page.
  • Click "Submit".
  • Verify the StackSet is created successfully.

Step 4: InsightCloudSec Configuration

Now that the AWS Organization and relevant member accounts have been configured for harvesting, it's time to enable harvesting within InsightCloudSec.

🚧

Prerequisites

Before you can successfully add an Organization to InsightCloudSec, you will need the following on hand:

  • The ARN/External ID for the organization harvesting role (created in step 1)
  • The ARN/External ID for the standard harvesting role (created in step 2 and step 3)

1. Log into InsightCloudSec with an admin user.

2. Navigate to the Clouds section and click the "Organizations" tab.

Organizations ViewOrganizations View

Organizations View

🚧

Cloud Organizations are Globally Unique

Note that a Cloud Organization cannot be added multiple times on the same InsightCloudSec installation. If you attempt to add the same Organization twice, the request will be rejected.

3. Click "Add Organization" in the top-right corner.

4. Select "Amazon Web Services" from Cloud Type drop-down menu.

Adding an AWS OrganizationAdding an AWS Organization

Adding an AWS Organization

5. Provide a nickname for the Organization. This creates a system Badge containing the nickname that can be searched or referenced throughout InsightCloudSec.

❗️

Impact to existing AWS accounts within InsightCloudSec

Adding a Cloud Organization will replace the credentials of associated cloud accounts already in InsightCloudSec.

Misconfiguration of the roles in member accounts will result in gaps in visibility.

6. Provide credentials for harvesting Organization data.

  • Provide the Role ARN for the Master Account role (created in step 1)
  • Provide a session name and duration.
    • The session name is what will display in any CloudTrail logs and is useful for auditing purposes.
  • Provide the External ID for the Master Account role (created in step 1), if you created one.
AWS Credential Required for Harvesting Organizational DataAWS Credential Required for Harvesting Organizational Data

AWS Credential Required for Harvesting Organizational Data

7. Provide credentials for harvesting Organization member accounts data.

  • Provide the Role ARN suffix for the standard harvesting role (created in step 2 and step 3)
  • Provide a session name and duration.
    • This is what will display in any CloudTrail logs and is useful for auditing purposes.
  • Provide the External ID for the standard harvesting role (created in step 2 and step 3), if you created one.
Organization Form ContinuedOrganization Form Continued

Organization Form Continued

8. Configure the optional scope-limiting settings.

  • Provide one or more prefixes to match accounts against. Any accounts with names that match those prefixes will be excluded.
  • Select the "Limit Import Scope" checkbox and provide Organizational Unit ID(s) to only include nested accounts and OUs associated with a given ID (or set of IDs).
  • Select the "Auto-remove suspended accounts" checkbox to automatically remove suspended AWS accounts from InsightCloudSec. As soon as this checkbox is enabled, a background process will begin running and remove the accounts automatically as they're found.
Additional Optional DataAdditional Optional Data

Additional Optional Data

9. Once you've completed all of the fields and details outlined above, click "Add".

📘

Expected Validation

When the form is submitted the form values are validated for formatting correctness. Next we test the Master Role for the following and will reject submission if any of the following fail.

  • Attempt to perform AssumeRole operation with Instance Role credentials.
  • Test role permissions with iam:SimulatePrincipalPolicy to verify we can perform all necessary Organization harvesting. We test the policy for all actions documented above in the "Master Account Role" section.

The member account roles are not validated at this point. Validation of member accounts is done during the syncing process and any failures are reflected in the Cloud account status on the Cloud Listing page.

Contact [email protected] with questions.

After successful submission a background job is enqueued that will fetch and synchronize all of your accounts. Depending on the number of accounts this will take a few minutes. In this example walkthrough, 127 accounts took a little over 1 minute.

❗️

Editing Organization Credentials

To make changes to any part of the credential configuration requires a complete resubmission of all fields due to all fields being encrypted in storage.

Filtering options can be updated independently by leaving all credential fields blank. If blank the existing credential configuration is left as is.

From the Organizations page, click the link for the number of accounts. This will redirect you to the Cloud Listing page filtered by the Cloud Organization using a badge associated with all member accounts.

  • As noted above, errors in permissions or failure to assume a role are represented by the cloud status.
Cloud Listing with Organization BadgeCloud Listing with Organization Badge

Cloud Listing with Organization Badge

Post-setup Information

Congratulations on integrating your AWS Organization with InsightCloudSec. Below you'll find some key information about your new integration as well as managing it.

Badging of Accounts

Accounts added via an AWS Organization will have a few Badges automatically associated to them:

  • cloud_org_path: shows the location of the account in the Organization tree
  • All tags associated with accounts are added as badges

Note: Despite not being listed explicitly, the system.cloud_organization:<cloud_org_id> badge is associated with all accounts in an Organization.

BadgingBadging

Badging

Changes to Credential Management

Because all accounts within the AWS Organization use the same credential configuration, they are considered as "managed" by the organization. This is reflected on the cloud settings page where the option to edit credentials and delete the account are not available.

Auto-badging

As an enhancement to support for provider-base organizations InsightCloudSec includes auto badging capabilities. The purpose of auto-badging is to create a 1:1 map of AWS account-level tags to Badges in InsightCloudSec. This allows Clouds to be scoped to a badge that maps to the account tag.

Auto badging takes place in two stages.

  • Periodically a process retrieves tags/labels from each account/project and compares them with ResourceTags associated with the corresponding cloud in the InsightCloudSec database.

    • If there are any changes detected, the ResourceTags in the database are overwritten with the values from the account/project. This means that Cloud Account tags should not be locally modified since any local changes will be overwritten the next time the process runs. Additionally any local changes that are made to Cloud Account tags are not pushed back up to the cloud provider.
  • Periodically a process retrieves all ResourceTags from the local database that are associated with the accounts managed by an organization. For each cloud the list of tags for that cloud is compared with the current list of Badges and for each Key/Value pair of tags:

    • Existing Badges with a Key prefix of system. are skipped.
    • If the corresponding Badge with the Key/Value pair for that cloud does not already exist, it is created.
    • If a tag Value changes, the Badge with the corresponding Key will be updated to that value.
    • If a Badge no longer has a tag with a corresponding Key, it will be deleted.
    • All Badges that have a corresponding tag will have their autogenerated column set to ‘true’ even if they were previously set to ‘false’.

Did this page help you?