AWS - EC2 or ECS Fargate - Terraform
Deploying InsightCloudSec to AWS EC2 or ECS Fargate Using Terraform
This page provides instructions for deploying and installing the InsightCloudSec platform via Terraform, with the option of using ECS Fargate (preferred) or EC2 as the compute component.
- Note: The content on this page applies to self-hosted customers. For hosted customers we recommend that you contact your CSM or reach out through the Customer Support Portal with any questions or concerns.
Value Names (DivvyCloud vs. InsightCloudSec)
Some components use our former product name (DivvyCloud vs. InsightCloudSec). Updates to the naming of these components will be communicated when changes are made, but note that the name difference does not affect functionality within the product.
Prerequisites
Before getting started with this deployment you will need to have the following:
- The appropriate AWS permissions to create network, IAM, database, and compute resources
- An
AdministratorAccesspolicy is recommended
- An
- Existing SSL certificate in AWS Certificate Manager
- Service-linked roles for the following services:
ecs,ecs.application-autoscaling,elasticache,elasticloadbalancing, andrds - Terraform 1.0.x
Step 1: Download Template
InsightCloudSec offers two deployment templates: the default template (which requires internet connectivity) and the standalone template. The default template loads Terraform modules from an InsightCloudSec S3 bucket to allow for streaming updates to our deployment. If connectivity is not possible, download the standalone deployment template. Click one of the links below to download the template .zip file.
- Default Template
- Standalone Template (only for environments with no internet access)
Each template comes with several .tf files that will control various aspects of the deployment, but there are only two files that should be edited: divvycloud-example.tfvars and variables.tf.
Step 2: Update Parameter Overrides
One of the files included with the downloaded template, divvycloud-example.tfvars, will allow you to specify/override common configuration parameters specified in variables.tf. Descriptions of key parameters in the file are organized by feature below.
- Note: Not all of the parameters in the file will be represented here, so reach out to your CSM or to Support through the Customer Support Portal with any questions or concerns.
Deployment Parameters
The parameters in this section are specifically related to deploying InsightCloudSec within AWS.
| Parameter | Description |
|---|---|
divvycloud_version | The AWS Elastic Container Registry URI for the image version of InsightCloudSec that will be deployed, e.g., "public.ecr.aws/rapid7-insightcloudsec/ics/core:v23.4.11".Visit the public gallery to view a list of all available image versions. |
account_id | The AWS account ID for the account where InsightCloudSec will be deployed, e.g., 123456789012. |
custom_ecs_env | List of custom environment variable maps for ECS Fargate, e.g., {name="DIVVY_LOG_LEVEL" value="debug"}. |
custom_ec2_env | List of custom environment variables for EC2 instances, e.g., "DIVVY_LOG_LEVEL=debug" |
Access Explorer
The parameters in this section relates to the Access Explorer.
| Parameter | Description |
|---|---|
enable_iam_analyzer | Set to true to enable the Access Explorer.Note: This feature currently requires an additional license. Reach out to your CSM or to Support through the Customer Support Portal with any questions or concerns. |
Azure EDH
The parameters in this section relate to Azure Event-Driven Harvesting.
Azure EDH Users
If you plan on using the Azure EDH feature, you'll need to update the
cloud-collectorversion and enable the cloud collector prior to deployment. Reach out through the Customer Support Portal for details before you start the deployment process.
| Parameter | Description |
|---|---|
cloud_collector_version | The AWS Elastic Container Registry URI for the image version of the Azure Cloud Collector that will be deployed, e.g., "public.ecr.aws/divvycloud/divvycloud:cloud-collector-v1.0.17".Visit the public gallery to view a list of all available image versions. |
enable_cloud_collector | Set to true to enable the Azure Cloud Collector. This is required for Azure EDH. |
Step 3: Update Compute Deployment Variables
The other file included with the downloaded template, variables.tf, will allow you to configure many AWS-related deployment parameters. Descriptions of key parameters in the file are organized by deployment type below.
- Note: Not all of the parameters in the file will be represented here, so reach out to your CSM or to Support through the Customer Support Portal with any questions or concerns.
General
The parameters in this section are related to the databases used by InsightCloudSec to serve and store information. For more information on deployment architecture, review Product Architecture.
Database Instance Sizing
- We do not recommend decreasing the default instance size of the ElastiCache nodes or RDS instances.
- RDS instance size may be increased if your environment demands it.
| Parameter | Description |
|---|---|
"redis_node_type" | The instance type and size used for the Redis node. |
"db_instance_class" | The instance type and size used for the MySQL node. |
ECS Fargate
ECS Fargate is the recommended deployment method. The parameters in this section are related to the size and performance of the InsightCloudSec cluster.
Task Count/Definition
- We do not recommend decreasing the default task counts or resource allocations below the defined defaults.
- Worker (P2) tasks will be automatically scaled based on workload/demand.
- Interface server task counts can be increased to accommodate a high request volume (auto-scaling coming soon)
| Parameter | Description |
|---|---|
"interface_server_task_count" | The total number of CPU tasks available to the interface server. The tasks will be split evenly (as possible) amongst the interface servers. Note: This is ignored if "use_instance_docker" is set to true. |
"scheduler_task_count" | The total number of CPU tasks available to the scheduler server. The tasks will be split evenly (as possible) amongst the scheduler servers. Note: This is ignored if "use_instance_docker" is set to true. |
"worker_p2_task_count" | The total number of P2 worker instances. Note: This is ignored if "use_instance_docker" is set to true. |
"worker_p0_p1_task_count" | The total number of P0/P1 persistent instances. Note: This number is replicated to however many worker instances there are if "use_instance_docker" is set to true. |
"interface_task_cpu" | The total number of CPU units for interface Docker containers. |
"interface_task_mem" | The total amount of memory for interface Docker containers. |
"scheduler_task_cpu" | The total number of CPU units for scheduler Docker containers. |
"scheduler_task_mem" | The total amount of memory for scheduler Docker containers. |
"worker_p0_p1_task_cpu" | The total number of CPU units for P0/P1 persistent Docker containers. |
"worker_p0_p1_task_mem" | The total amount of memory for P0/P1 persistent Docker containers. |
"worker_p2_task_cpu" | The total number of CPU units for P2 worker Docker containers. |
"worker_p2_task_mem" | The total amount of memory for P2 worker Docker containers. |
EC2
If ECS Fargate (preferred) is not approved for use, EC2 instances can be used in its place.
Enable EC2 Support
If using EC2 for compute, be sure to set
use_instance_dockertotrueand configure any other related EC2 parameters.
| Parameter | Description |
|---|---|
"ami" | List of Amazon Machine Images to use per region. |
"ec2_generate_ssh_key" | If set to false, an existing SSH key is used (ec2_custom_ssh_key). |
"ec2_custom_ssh_key" | The ARN of an existing SSH key (used in the event ec2_generate_ssh_key is set to false). |
"use_instance_docker" | If set to true, instances with local Docker containers will be created instead of using the cloud-specific container service. |
"worker_instance_count" | The total number of worker Docker containers. |
"worker_instance_type" | The instance type and size used for the workers. |
"interface_scheduler_instance_count" | The total number of interface and scheduler servers. |
"interface_scheduler_instance_type" | The instance type and size used for the interface and scheduler servers. |
Plan and Apply
Note: Completing this deployment method should take around 60 minutes.
1. Rename your customized insightcloudsec-example.tfvars file to insightcloudsec.tfvars.
2. Update Terraform to ensure you have the latest modules:
terraform get -update
3. Update providers.tf to include your environment's backend configuration.
4. Create the Terraform plan from your .tfvars file:
terraform plan -var-file=insightcloudsec.tfvars
5. Apply the new Terraform plan:
terraform apply insightcloudsec.tfplan
Updated about 2 months ago