These setup instructions are provided to outline the steps required to complete the setup of Azure Event-driven Harvesting (EDH) in InsightCloudSec using the Azure Console.
For self-hosted customers, this feature is only supported using the Fargate ECS via Terraform deployment method. Additionally, regardless of your method of deployment, there are settings that must be configured prior to deployment to prevent issues with EDH functioning correctly. In general, for self-hosted customers interested in using Azure EDH we recommend connecting with your CSM or the Customer Support Portal prior to enabling this feature.
Before getting started with the setup of Azure EDH, ensure you have the following:
- A functioning InsightCloudSec platform with the appropriate Admin (Org or Domain) permissions
- A basic understanding of the relevant Azure services
- Appropriate Azure administrative permissions
- A strategy around the behaviors you want to use to configure EDH
If you have questions or encounter issues, reach out to us through the Customer Support Portal.
Before getting started with the deployment steps in the Azure Console, you will need to obtain details from the Consumer to use in your Service Bus for the unique namespace.
1. From you InsightCloudSec installation, navigate to "Cloud --> Clouds -- Event-Driven Harvesting".
2. On the Event-Driven Harvesting tab, select the "Consumers" subtab.
3. Select the "Azure EDH Setup" option.
4. Copy the Namespace ID (formatted as r7-ics-xxxxxx) and Queue name and save these details in a safe place. You will need these details to complete the configuration in the Azure Console in subsequent steps.
You will be creating a new service bus, a queue for your service bus, and an event grid subscription in the Azure Console to complete the setup for EDH. Refer to the appropriate section for the steps required to complete each item.
Complete the steps outlined below to create a new Service Bus.
1. Log in to the Azure Console as an administrator and search for “Service Bus”.
2. Click “Create” to add a new Service Bus.
Complete the form details as follows:
- Subscription: Select your Azure subscription from the drop-down menu
- Resource Group: Select your Resource Group
- Namespace name: Update this field with the name information you previously saved from the InsightCloudSec platform
- Location: Using the default is fine, update if you'd like to run this in a different location
- Pricing Tier: Select the "standard" pricing tier from the drop-down menu
3. Click “Review + Create” to complete this step. This will take a few minutes to finalize. You will receive a success confirmation once the Service Bus setup is complete.
Consumer & Producer Visibility
Deploying a service bus and namespace to the same subscription as the event grid will mean that this subscription is both a consumer and a producer, however InsightCloudSec will only surface it as a consumer in the UI (events will still be visible)
1. From your new Service Bus click on the name to open the Service Bus resource page.
2. Navigate to “Entities → Queues” on the main navigation (left-side) and select "Queues".
3. Click the “+Queue” button to create a new queue with the following settings:
- Name: rapid7-insightcloudsec-edh-queue (required)
- Max queue size: 5GB
- Max delivery count: 25 (to account for any connection issues)
- Message TTL: 5 hours (can be customized)
- Lock duration: 5 minutes (locks messages to a batch)
- Enable duplicate detection: Select this option and set to 15 minutes
4. Once you have completed the form with the required fields, click “Create” to finalize the creation of the queue.
1. From your new Service Bus queue list, select the queue to open the detail page.
2. Navigate to “Access Control (IAM)” on the main navigation (left-side) and select "Check Access" tab.
3. Within the card "Grant Access to this Resource", click “Add Role Assignment”.
4. Select the built- in "Azure Service Bus Data Receiver" role by clicking on the row and select "Next".
5. Complete the form as follows:
- Assign access to: User, group, or service principle
- Members: Click "+ Select Members" and search for the InsightCloudSec Service Principal used for harvesting in this subscription.
Capturing Your Subscriptions
You will need to repeat the steps outlined below for each individual subscription that you want to capture events from.
1. From the Azure console, search for “Event Grid Subscriptions” (note that this is not the same as an Azure subscription).
2. Click the “+Event Subscription” button to create a new event subscription.
3. Complete the form as follows:
- Name: rapid7-insightcloudsec-edh-eventgrid-subscription
- Event Schema: default
- Topic Types: Azure Subscriptions
- Subscription: Select your subscription
- Resource Group: Select your Resource Group
- System Topic Name: rapid7-insightcloudsec-eventgrid-topic
- Note: If a topic for this Azure Subscription already exists, the default will automatically be selected.
- Event Types: Select only “Write Success” and “Delete Success”
- Endpoint Type: Select the Service Bus you previously created
4. Under the “Additional Features” tab, locate “Retry Policies” and update the Max Delivery Attempts to “30” and Event Time to Live to “23 hours”.
5. Click the “Create” button to complete the creation of your Event Subscription.
Navigate to the Event Grid System Topics to locate your new parent topic and confirm the queue.
After completing the setup within the Azure Console a Consumer should automatically display within the InsightCloudSec UI.
- Navigate to "Cloud --> Clouds --> Event-Driven Harvesting --> Consumers" to view and confirm.
Producers will display when they receive events (except for the Consumer itself).
- Navigate to "Cloud --> Clouds --> Event-Driven Harvesting --> Producers" to view and confirm.
Note: If there is nothing showing in the UI then it is likely that the name of the Service Bus was not set correctly (using the license) or the queue itself is incorrectly named.
Updated 3 months ago