These setup instructions are provided to outline the steps required to complete the setup of Azure Event-driven Harvesting (EDH) in InsightCloudSec using the Azure Console.
For self-hosted customers, this feature is only supported using the Fargate ECS via Terraform deployment method. Additionally, regardless of your method of deployment, there are settings that must be configured prior to deployment to prevent issues with EDH functioning correctly. In general, for self-hosted customers interested in using Azure EDH we recommend connecting with your CSM or the Customer Support Portal prior to enabling this feature.
Before getting started with the setup of Azure EDH, ensure you have the following:
- A functioning InsightCloudSec platform with the appropriate Admin (Org or Domain) permissions
- A basic understanding of the relevant Azure services
- Appropriate Azure administrative permissions
If you have questions or encounter issues, reach out to us through the Customer Support Portal.
You will be creating a new service bus, a queue for your service bus, and an event grid subscription in the Azure Console to complete the setup for EDH. Refer to the appropriate section for the steps required to complete each item.
Complete the steps outlined below to create a new Service Bus.
1. Log in to the Azure Console as an administrator and search for “Service Bus”.
2. Click “Create” to add a new Service Bus.
Complete the form details as follows:
- Subscription: Select your Azure subscription from the drop-down menu
- Resource Group: Select your Resource Group
- Namespace name: Provide a name to help distinguish the service bus for InsightCloudSec usage
- Note: InsightCloudSec recommends including the short, unique installation ID provided to you on the "Create Azure EDH Configuration" window to ensure the data source is only being consumed by single installation. This window can be found by logging into your InsightCloudSec instance then navigating to Clouds -> EDH Consumers -> EDH Configuration -> Azure ServiceBus Consumer.
- Location: Using the default is fine, update if you'd like to run this in a different location
- Pricing Tier: Select the "Standard" pricing tier from the drop-down menu. The "Premium" pricing tier has also been validated if you would prefer to use a Premium Service Bus.
3. Ensure you keep the resource group and namespace name on hand for when you configure InsightCloudSec.
4. Click “Review + Create” to complete this step. This will take a few minutes to finalize. You will receive a success confirmation once the Service Bus setup is complete.
Consumer & Producer Visibility
Deploying a service bus and namespace to the same subscription as the event grid will mean that this subscription is both a consumer and a producer; however, InsightCloudSec will only surface it as a consumer in the UI (events will still be visible).
1. From your new Service Bus click on the name to open the Service Bus resource page.
2. Navigate to “Entities → Queues” on the main navigation (left-side) and select "Queues".
3. Click the “+Queue” button to create a new queue with the following settings:
- Name: Provide a unique name
- Max queue size: 5GB
- Max delivery count: 25 (to account for any connection issues)
- Message TTL: 5 hours (can be customized)
- Lock duration: 5 minutes (locks messages to a batch)
- Enable duplicate detection: Select this option and set to 15 minutes
4. Ensure you keep the queue name on hand for when you configure InsightCloudSec.
5. Once you have completed the form with the required fields, click “Create” to finalize the creation of the queue.
1. From your new Service Bus queue list, select the queue to open the detail page.
2. Navigate to “Access Control (IAM)” on the main navigation (left-side) and select "Check Access" tab.
3. Within the card "Grant Access to this Resource", click “Add Role Assignment”.
4. Select the built- in "Azure Service Bus Data Receiver" role by clicking on the row and select "Next".
5. Complete the form as follows:
- Assign access to: User, group, or service principle
- Members: Click "+ Select Members" and search for the InsightCloudSec Service Principal used for harvesting in this subscription.
Capturing Your Subscriptions
You will need to repeat the steps outlined below for each individual subscription that you want to capture events from.
1. From the Azure console, search for “Event Grid Subscriptions” (note that this is not the same as an Azure subscription).
2. Click the “+Event Subscription” button to create a new event subscription.
3. Complete the form as follows:
- Name: rapid7-insightcloudsec-edh-eventgrid-subscription
- Event Schema: default
- Topic Types: Azure Subscriptions
- Subscription: Select your subscription
- Resource Group: Select your Resource Group
- System Topic Name: rapid7-insightcloudsec-eventgrid-topic
- Note: If a topic for this Azure Subscription already exists, the default will automatically be selected.
- Event Types: Select only “Write Success” and “Delete Success”
- Endpoint Type: Select the Service Bus you previously created
4. Under the “Additional Features” tab, locate “Retry Policies” and update the Max Delivery Attempts to “30” and Event Time to Live to “23 hours”.
5. Click the “Create” button to complete the creation of your Event Subscription.
Navigate to the Event Grid System Topics to locate your new parent topic and confirm the queue.
After completing the setup within the Azure Console, you're ready to configure an Azure EDH consumer within InsightCloudSec.
Before you can add an Azure ServiceBus Consumer to InsightCloudSec, you will need the following on hand:
- The resource group and namespace name for the Service Bus (created in Create a New Service Bus)
- The queue name for the queue created within the Service Bus (created in Create a Queue for Your Service Bus)
- The name of the subscription that contains the Service Bus (created in Azure Setup - Single Cloud)
1. Login to your InsightCloudSec platform and click "Clouds" in the left-hand navigation menu.
- Click "Consumers".
- Click "EDH Configuration".
- From the drop-down menu, click "Azure Service Bus Consumer".
2. Update the configuration for the necessary information.
- Select the Azure Subscription that contains the service bus.
- Provide the service bus ID in the format of
<resource_group_name>|<namespace_id>|<queue_name>, ensuring you replace the placeholders with the appropriate values.
- Click "Configure".
Congratulations on setting up Event Driven Harvesting for your Azure Subscription(s) within InsightCloudSec. Below you'll find some important links about EDH in general.
Updated 5 days ago