AWS - EC2 or ECS Fargate - Terraform

This page provides instructions for deploying and installing the InsightCloudSec platform via Terraform, with the option of using ECS Fargate (preferred) or EC2 as the compute component. The content on this page applies to self-hosted customers. For hosted customers we recommend that you contact your CSM or reach out through the Customer Support Portal with any questions or concerns.

Product name to be replaced

You may observe that some components, screen captures, or examples use our former product name, DivvyCloud. This doesn't affect the configuration or the product's functionality, and we will notify you as we replace these component names.

Prerequisites

Before getting started with this deployment you will need to have the following:

  • The appropriate AWS permissions to create network, IAM, database, and compute resources
    • An AdministratorAccess policy is recommended
  • Existing SSL certificate in AWS Certificate Manager
  • Service-linked roles for the following services: ecs, ecs.application-autoscaling, elasticache, elasticloadbalancing, and rds
  • Terraform 1.0.x

Step 1: Download Template

InsightCloudSec offers two deployment templates: the default template (which requires internet connectivity) and the standalone template. The default template loads Terraform modules from an InsightCloudSec S3 bucket to allow for streaming updates to our deployment. If connectivity is not possible, download the standalone deployment template. Click one of the links below to download the template .zip file.

Each template comes with several .tf files that will control various aspects of the deployment, but there are only two files that should be edited: divvycloud-example.tfvars and variables.tf.

Step 2: Update Parameter Overrides

One of the files included with the downloaded template, divvycloud-example.tfvars, will allow you to specify/override common configuration parameters specified in variables.tf. Descriptions of key parameters in the file are organized by feature below. Not all of the parameters in the file will be represented here, so reach out to your CSM or to Support through the Customer Support Portal with any questions or concerns.

Deployment Parameters

The parameters in this section are specifically related to deploying InsightCloudSec within AWS.

ParameterDescription
divvycloud_versionThe AWS Elastic Container Registry URI for the image version of InsightCloudSec that will be deployed, e.g., public.ecr.aws/rapid7-insightcloudsec/ics/core:latest.Visit the public gallery to view a list of all available image versions.
account_idThe AWS account ID for the account where InsightCloudSec will be deployed, e.g., 123456789012.
custom_ecs_envList of custom environment variable maps for ECS Fargate, e.g., {name=DIVVY_LOG_LEVEL value=debug}.
custom_ec2_envList of custom environment variables for EC2 instances, e.g., DIVVY_LOG_LEVEL=debug

Access Explorer / Least-Privileged Access (LPA)

The parameters in this section relates to the Access Explorer and Least-Privileged Access (LPA). Access Explorer currently requires an additional license. Reach out to your CSM or to Support through the Customer Support Portal with any questions or concerns.

ParameterDescription
enable_iam_analyzerSet to true to enable the Access Explorer.
use_p3_autoscalingThis is set to true by default (for Azure EDH and LPA). If you do not plan on using the Azure EDH or LPA features, set this to false.

Deployment Failure Troubleshooting

If use_p3_autoscaling is set to true and enable_iam_analyzer is set to false, the deployment will fail.`

Azure EDH

The parameters in this section relate to Azure Event-Driven Harvesting.

Azure EDH Users

If you plan on using the Azure EDH feature, you'll need to update the cloud-collector version and enable the cloud collector prior to deployment. Reach out through the Customer Support Portal for details before you start the deployment process.

ParameterDescription
cloud_collector_versionThe AWS Elastic Container Registry URI for the image version of the Azure Cloud Collector that will be deployed, e.g., public.ecr.aws/rapid7-insightcloudsec/ics/edh-worker:latest. Visit the public gallery to view a list of all available image versions.
enable_cloud_collectorSet to true to enable the Azure Cloud Collector. This is required for Azure EDH.

Step 3: Update Compute Deployment Variables

The other file included with the downloaded template, variables.tf, will allow you to configure many AWS-related deployment parameters. Descriptions of key parameters in the file are organized by deployment type below. Not all of the parameters in the file will be represented here, so reach out to your CSM or to Support through the Customer Support Portal with any questions or concerns.

General

The parameters in this section are related to the databases used by InsightCloudSec to serve and store information. For more information on deployment architecture, review Product Architecture.

Database Instance Sizing

  • We do not recommend decreasing the default instance size of the ElastiCache nodes or RDS instances.
  • RDS instance size may be increased if your environment demands it.
ParameterDescription
redis_node_typeThe instance type and size used for the Redis node.
db_instance_classThe instance type and size used for the MySQL node.

ECS Fargate

ECS Fargate is the recommended deployment method. The parameters in this section are related to the size and performance of the InsightCloudSec cluster.

Database Instance Sizing

  • We do not recommend decreasing the default task counts or resource allocations below the defined defaults
  • Worker (P2) tasks will be automatically scaled based on workload/demand
  • Interface server task counts can be increased to accommodate a high request volume, but note that autoscaling (use_interfaceserver_autoscaling) is on by default.
ParameterDescription
interface_server_task_countThe total number of CPU tasks available to the interface server. The tasks will be split evenly (as possible) amongst the interface servers. This is ignored if use_instance_docker is set to true.
scheduler_task_countThe total number of CPU tasks available to the scheduler server. The tasks will be split evenly (as possible) amongst the scheduler servers. This is ignored if use_instance_docker is set to true.
worker_p2_task_countThe total number of P2 worker instances. This is ignored if use_instance_docker is set to true.
worker_p0_p1_task_countThe total number of P0/P1 persistent instances. This number is replicated to however many worker instances there are if use_instance_docker is set to true.
interface_task_cpuThe total number of CPU units for interface Docker containers.
interface_task_memThe total amount of memory for interface Docker containers.
scheduler_task_cpuThe total number of CPU units for scheduler Docker containers.
scheduler_task_memThe total amount of memory for scheduler Docker containers.
worker_p0_p1_task_cpuThe total number of CPU units for P0/P1 persistent Docker containers.
worker_p0_p1_task_memThe total amount of memory for P0/P1 persistent Docker containers.
worker_p2_task_cpuThe total number of CPU units for P2 worker Docker containers.
worker_p2_task_memThe total amount of memory for P2 worker Docker containers.

EC2

If ECS Fargate (preferred) is not approved for use, EC2 instances can be used in its place.

Enable EC2 Support

If using EC2 for compute, be sure to set use_instance_docker to true and configure any other related EC2 parameters.

ParameterDescription
amiList of Amazon Machine Images to use per region.
ec2_generate_ssh_keyIf set to false, an existing SSH key is used (ec2_custom_ssh_key).
ec2_custom_ssh_keyThe ARN of an existing SSH key (used in the event ec2_generate_ssh_key is set to false).
use_instance_dockerIf set to true, instances with local Docker containers will be created instead of using the cloud-specific container service.
worker_instance_countThe total number of worker Docker containers.
worker_instance_typeThe instance type and size used for the workers.
interface_scheduler_instance_countThe total number of interface and scheduler servers.
interface_scheduler_instance_typeThe instance type and size used for the interface and scheduler servers.

Plan and Apply

Completing this deployment method should take around 60 minutes.

  1. Rename your customized insightcloudsec-example.tfvars file to insightcloudsec.tfvars.
  2. Update Terraform to ensure you have the latest modules: terraform get -update
  3. Update providers.tf to include your environment's backend configuration.
  4. Create the Terraform plan from your .tfvars file: terraform plan -var-file=insightcloudsec.tfvars
  5. Apply the new Terraform plan: terraform apply insightcloudsec.tfplan