InsightCloudSec Docs

Welcome to the InsightCloudSec Docs!

InsightCloudSec by Rapid7 (formerly DivvyCloud) is a Cloud-Native Security Platform that provides real-time analysis and automated remediation for continuous security and compliance for your multi-cloud environment.

For questions reach out to us through [email protected].

Take Me to the Docs!    Release Notes

AWS Cloud Setup (Organizations)

Integrating an AWS Organization with InsightCloudSec

Overview

For users looking to harvest data from a multiple cloud accounts grouped under an AWS Organization, InsightCloudSec requires a bit of configuration within AWS. After you've finished with this set of instructions, you'll have added an AWS Organization to InsightCloudSec and your organization's services and data will be harvested. Learn more about AWS Organizations here.

Note this page and the functionality detailed here refer to the provider-specific organizations capability available under "Clouds --> Organizations". This functionality should not be confused with the InsightCloudSec-specific Organizations capability that allows for multi-tenant functionality available under "System Administration --> Organizations".

If you need to add a single account, review AWS Cloud Setup (Single Cloud Account).

Service Control Policies - Implementation & Known Issues

Within AWS, a Service Control Policy (SCPs) is a type of policy used to manage permissions at the organization level. Per AWS, "SCPs offer central control over the maximum available permissions for all accounts in your organization. SCPs help you to ensure your accounts stay within your organization’s access control guidelines.

  • Read more from AWS here on using SCPs.

Many organizations choose to implement SCPs for security purposes, while also providing the permissions required for InsightCloudSec to function with full visibility into their AWS accounts.

It is important to note that when implemented, many SCPs will block the required permissions InsightCloudSec needs to operate - even when permissions have been explicitly granted via roles and policies.

  • If this scenario applies to your environment, you will need to revise your SCP to ensure you have permitted the required InsightCloudSec permissions.
  • This is normal SCP behavior as they are organization-wide policies. SCPs are configured within AWS to supersede other types of permissions.
  • You can review our AWS IAM Policies for details, otherwise refer to the AWS documentation on SCPs.

Another item to note - In more limited cases an SCP in conflict with an existing role/policy can also result in visibility issues (noted below).

🚧

Warnings with False Positives - Known AWS Service Control Policy Issue

When viewing details on the Clouds Listing page, InsightCloudSec may provide false positive "Warnings" around missing permissions. In some scenarios the permissions are granted within a Service Control Policy (SCP) but falsely report as denied.

This scenario is the result of a known issue within AWS where if an Organization has an SCP with conditions based on global keys (e.g. aws:PrincipalArn) the IAM Policy Simulator results are not accurate because it does not have context with the global keys.

If you have verified that your resources are being harvested as expected you can safely disregard these warnings. If you're not sure or otherwise have remaining questions or concerns, contact us at [email protected].

Prerequisites

Before getting started with adding an AWS Organization in InsightCloudSec you will need the following:

  • A functioning InsightCloudSec platform installation.
  • Domain Admin permissions within InsightCloudSec.
  • One or more AWS Organizations.
  • Access to AWS Console and/or API.
  • Familiarity with STS Assume Role and cross-account trust relationships.
  • We recommend that you review the documentation on AWS Additional Configuration, particularly the additional steps to support Opt-in regions.
  • Permissions to create IAM Roles in ALL accounts within the AWS Organization.
  • Permission to create "trust relationship" between roles and users.
  • Access and permission to use CloudFormation's StackSets for deploying roles across all accounts.

📘

General Support

If you have questions or need information before you get started reach out to us at [email protected].

View Your Organization in the AWS Console

1. From anywhere in the AWS console, select the dropdown in the upper right hand side with the current user session.

2. Select "My Organization".

  • The default view lists all Account that are members of the Organization. This view is helpful for identifying individual Account numbers as well as which Account is designated as the Master account.

3. Identify the Master account and note the Account ID.

AWS Organizations - Master AccountAWS Organizations - Master Account

AWS Organizations - Master Account

4. From the three tabs Accounts, Organize Accounts, and Policies, select "Organize Accounts".

AWS Organizations - Organize AccountsAWS Organizations - Organize Accounts

AWS Organizations - Organize Accounts

This view shows Organizational Units (OUs) as well as the Accounts within a particular OU. Accounts and OUs can be nested within another OU creating a rich file-tree like structure for organizing Accounts.

  • Each Account has a location in this Organization tree.
  • InsightCloudSec uses the location to give contextual information about the Account. Later on this can be viewed as the cloud_org_path Badge associated with each Account.

📘

Saving the Organizational Unit ID for later

If you want to only sync accounts within a particular OU, copy the OU's ID and save for later. See the high-lighted text in the above screenshot.

Next we need to setup all the credentials necessary to onboard accounts using the Organization.

What Authentication Method is Used?

To authenticate against the Amazon (AWS) API, multiple IAM roles must be provisioned. IAM roles can only be defined at the account level, which is an individual account within a company's Amazon's Organization.

All authentication is done via Amazon's Secure Token Service (STS) with an AssumeRole operation by the caller, an IAM User or ec2 Instance Role, to the role in the target account. There must be a trust relationship between the caller and the role being assumed.

  • The AssumeRole operation returns a set of temporary session credentials that are used by InsightCloudSec to perform harvesting.

  • An Instance Assume Role (AWS) is recommended when deploying to an Amazon environment. Refer to IAM Policies for more information.

🚧

Opt-in Regions

InsightCloudSec uses the Global STS endpoint for all STS operations. InsightCloudSec includes support for AWS regions with a the "opt-in" classification. This requires an additional step of updating the STS Endpoint configuration to use longer session token in the account with the Caller Identity that will be performing the AssumeRole operation.

To do this, visit the AWS console here and modify the Global Endpoint option to allow larger session tokens to the global endpoint (https://sts.amazonaws.com).

For a guided experience, please see our guide AWS Additional Configuration

Deployment Recommendations

The manual setup of roles, policies, and trust relationships across all accounts in an AWS Organization can be tedious and error prone.

The recommended pattern for a reliable deployment is to use Amazon's native CloudFormation StackSets. These provide a way to define a set of roles and policies and deploy them across all accounts in the organization.

For questions on this process refer to the linked AWS documentation, or for additional assistance reach out to us through any of the options outlined on our Getting Support page.

Steps to deploy IAM Roles and Policies in Accounts

Task Specific Roles

InsightCloudSec requires that a single role be created in the master account for harvesting Organization data and a role in each member account in the AWS Organization for general purpose harvesting.

Master Account Role

InsightCloudSec makes the distinction between the task of harvesting information about the Organization and the traditional harvesting of cloud resources within the individual accounts.

Making a separate role limits the permissions to ones only relevant to the Organization level data plane. Additionally, the organizations:ListTagsForResource endpoint, used for listing tags on accounts, can only be called from the master account.

Review the Master Account Role policy for details. The permissions specified in the aforementioned policy allow InsightCloudSec to list all Account, Account tags, Organizational Units, Service Control Policies (SCPs), and their respective attachments. With that in hand, InsightCloudSec will be able to create an internal configuration for the Accounts and start harvesting with the credential configuration described below.

📘

Save Master Account Role Info

Write down the Role ARN, SessionDuration, and optionally the ExternalID.

Member Account Roles

❗️

Harvesting Roles

All general purpose harvesting roles must use the same configuration. The form provides a single configuration that all member accounts in the Organization must adhere to for harvesting to work.

For general purpose harvesting there needs to be a role in every member account of the AWS Organization with the exact same name and configuration. The IAM Policies attached to this role should follow the standard Policy recommendations for harvesting.

Using the same caller, IAM User with API/Secret key or Instance role, we perform the STS AssumeRole operation on a role ARN that we format dynamically with each of the account IDs that we harvest from the organization.

# ARN pattern
arn:aws:iam::{account_id}:role/{RoleArnSuffix}

# Example after formatting
arn:aws:iam::123456789012:role/InsightCloudSecReadOnlyRole

The RoleArnSuffix can be any valid role name but it must be the same across all accounts.

Additionally if the user wishes to use an ExternalId on the role, this too must also be the same across all accounts.

📘

Session Duration

The form allows you the set the requested duration of the temporary STS credentials to 43200 seconds, which is the maximum value that AWS allows. However the actual max value you can request is determined on a per role basis.

Set all of the roles to have the same session duration and use that allowed value in the Duration Seconds form field.

See screenshot below of session duration in AWS console.

AWS Session Duration ExampleAWS Session Duration Example

AWS Session Duration Example

Establishing Trust Relationships

Now that you have roles deployed in all accounts, those roles must have a policy attached to them that grants an explicit trust relationship to a caller identity.

See our guide on using STS AssumeRole to setup trust relationships from the roles in each member account to the caller identity. That documentation is available here --> IAM Policies.

The identity that's assuming the role can be one of two things:

1. An EC2 Instance Role

  • This is the preferred method when running inside of AWS since it uses the Instance credentials available from the Instance Metadata Service (IMDSv2) which reduces the complexity of creating and managing long term API credentials.
  • If using an Instance Role the role must be attached to the Instance Profile of all Instances of the InsightCloudSec installation.

2. IAM User

  • If your InsightCloudSec installation is running outside of AWS, you may use a set of Access/Secret API to perform the STS Assume Role operation.
  • To successfully assume the role, there must be a trust relationship between the user and role.

📘

Using an IAM User with Access/Secret API keys

If you are using an IAM user, store your API keys in a safe location. These will be needed when adding the Organization configuration.

AWS Setup Review

If you setup manually or with a CloudFormation StackSet, at this point the following IAM resources should exist.

Caller Identity That Will Assume Roles

  • One EC2 Instance Role with permissions to perform the AssumeRole operation, attached to the instances used in the InsightCloudSec install.
    OR
  • One IAM User with a policy to perform the AssumeRole operation with a set of long term credentials.

Other Roles

  • Two roles in the master account.
    • One role with the specific policy for harvesting only organization data.
    • One role for standard harvesting (see next bullet).
  • One role in ALL member accounts for standard harvesting with the exact same RoleName, SessionDuration, and ExternalID configuration.

Add AWS Organization to InsightCloudSec

With the IAM Role and user configuration we gathered from the prerequisites in hand:

1. Log into InsightCloudSec with an admin user.

2. Navigate to the Clouds section and open the "Organizations" tab.

Organizations ViewOrganizations View

Organizations View

🚧

Cloud Organizations are Globally Unique

Note that a Cloud Organization cannot be added multiple times on the same InsightCloudSec installation. If you attempt to add the same Organization twice the request will be rejected.

3. Click "Add Organization" button on top right corner of card.

4. Select "Amazon Web Services" from Cloud Type selection.

Adding an AWS OrganizationAdding an AWS Organization

Adding an AWS Organization

5. Provide a nickname for the Organization. This creates a system Badge containing the nickname that can be searched or referenced throughout InsightCloudSec.

❗️

Note impact to existing AWS accounts already added to InsightCloudSec

Adding a Cloud Organization will replace the credentials of associated cloud accounts already in InsightCloudSec.

Misconfiguration of the roles in member accounts will result in gaps in visibility.

6. Gather the RoleArn, SessionDuration, and ExternalID configuration we saved from the role created to harvest AWS Organization information. This is not the standard harvesting role.

7. Enter the RoleArn, SessionDuration and ExternalID.

8. Choose a Session Name.

  • This is what will display in CloudTrail logs and is useful for auditing purposes.
AWS Credential Required for Harvesting Organizational DataAWS Credential Required for Harvesting Organizational Data

AWS Credential Required for Harvesting Organizational Data

9. Gather configuration for standard harvesting for member accounts.

  • Enter role name used in ALL accounts. Note: The description has an example ARN that will update to show what the dynamically-formatted ARN will look like.

10. Choose a Session Name.

  • This is what will display in CloudTrail logs and is useful for auditing purposes.
  • This will be the same for all member accounts.
Organization Form ContinuedOrganization Form Continued

Organization Form Continued

11. Optionally, if using an IAM User, select the checkbox to reveal access key fields, then enter the appropriate credentials

IAM User CredentialsIAM User Credentials

IAM User Credentials

ADVANCED OPTIONS

12. We offer three additional options for limiting the scope of account onboarding:

Optional: Provide one or more prefixes to match accounts against. Any accounts with names that match those prefixes will be excluded.

Optional: Select the "Limit Import Scope" checkbox and provide Organizational Unit ID(s) to only include nested accounts and OUs associated with a given ID (or set of IDs). To find the Organizational Unit's ID, reference the screenshot and callout in the View Your Organization in AWS section above.

Optional: Select the "Auto-remove suspended accounts" checkbox to automatically remove suspended AWS accounts from InsightCloudSec. As soon as this checkbox is enabled, a background process will begin running and remove the accounts automatically as they're found.

Additional Optional DataAdditional Optional Data

Additional Optional Data

13. Once you've completed all of the fields and details outlined above, click to "Submit form".

📘

Expected Validation

When the form is submitted the form values are validated for formatting correctness. Next we test the Master Role for the following and will reject submission if any of the following fail.

  • Attempt to perform AssumeRole operation with Instance Role or IAM User credentials.
  • Test role permissions with iam:SimulatePrincipalPolicy to verify we can perform all necessary Organization harvesting. We test the policy for all actions documented above in the "Master Account Role" section.

The member account roles are not validated at this point. Validation of member accounts is done during the syncing process and any failures are reflected in the Cloud account status on the Cloud Listing page.

Contact [email protected] with questions.

14. After successful submission a background job is enqueued that will fetch and synchronize all of your accounts. Depending on the number of accounts this will take a few minutes. In this example walkthrough, 127 accounts took a little over 1 minute.

❗️

Note about Editing

To make changes to any part of the credential configuration requires a complete resubmission of all fields due to all fields being encrypted in storage.

Filtering options can be updated independently by leaving all credential fields blank. If blank the existing credential configuration is left as is.

15. From the Organizations page, click the link for the number of accounts. This will redirect you to the Cloud Listing page filtered by the Cloud Organization using a badge associated with all member accounts.

  • As noted above, errors in permissions or failure to assume a role are represented by the cloud status.
Cloud Listing with Organization BadgeCloud Listing with Organization Badge

Cloud Listing with Organization Badge

Badging of Accounts

Accounts added via an AWS Organization will have a few Badges automatically associated to them.

  • cloud_org_path: shows the location of the account in the Organization tree.
  • All tags associated with accounts are added as badges.

Note: Despite not being listed explicitly, the system.cloud_organization:<cloud_org_id> badge associated with all accounts in an Organization.

BadgingBadging

Badging

Changes to Credential Management

Because all accounts within the AWS Organization use the same credential configuration, they are considered as "managed" by the organization. This is reflected on the cloud settings page where the option to edit credentials and delete the account are not available.

Auto Badging

As an enhancement to support for provider-base organizations InsightCloudSec includes auto badging capabilities. The purpose of auto badging is to create a 1:1 map of AWS account-level tags or GCP project-level labels to Badges in InsightCloudSec. This allows Clouds to be scoped to a badge that maps to the account tag.

Auto badging takes place in two stages.

  • Periodically a process retrieves tags/labels from each account/project and compares them with ResourceTags associated with the corresponding cloud in the InsightCloudSec database.

    • If there are any changes detected, the ResourceTags in the database are overwritten with the values from the account/project. This means that Cloud Account tags should not be locally modified since any local changes will be overwritten the next time the process runs. Additionally any local changes that are made to Cloud Account tags are not pushed back up to the cloud provider.
  • Periodically a process retrieves all ResourceTags from the local database that are associated with the accounts managed by an organization. For each cloud the list of tags for that cloud is compared with the current list of Badges and for each Key/Value pair of tags:

    • Existing Badges with a Key prefix of system. are skipped.
    • If the corresponding Badge with the Key/Value pair for that cloud does not already exist, it is created.
    • If a tag Value changes, the Badge with the corresponding Key will be updated to that value.
    • If a Badge no longer has a tag with a corresponding Key, it will be deleted.
    • All Badges that have a corresponding tag will have their autogenerated column set to ‘true’ even if they were previously set to ‘false’.

Updated 3 days ago

AWS Cloud Setup (Organizations)


Integrating an AWS Organization with InsightCloudSec

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.