EDH - Azure Setup

These setup instructions are provided to outline the steps required to complete the setup of Azure Event-driven Harvesting (EDH) in InsightCloudSec using the Azure Console.

❗️

Self-Hosted Customers

For self-hosted customers, this feature is only supported using the Fargate ECS via Terraform deployment method. Additionally, regardless of your method of deployment, there are settings that must be configured prior to deployment to prevent issues with EDH functioning correctly. In general, for self-hosted customers interested in using Azure EDH we recommend connecting with your CSM or the Customer Support Portal prior to enabling this feature.

Prerequisites

Before getting started with the setup of Azure EDH, ensure you have the following:

• A functioning InsightCloudSec platform with the appropriate Admin (Org or Domain) permissions
• A basic understanding of the relevant Azure services
• Appropriate Azure administrative permissions

If you have questions or encounter issues, reach out to us through the Customer Support Portal.

Steps to Configure EDH in the Azure Console

You will be creating a new service bus, a queue for your service bus, and an event grid subscription in the Azure Console to complete the setup for EDH. Refer to the appropriate section for the steps required to complete each item.

Create a New Service Bus

Complete the steps outlined below to create a new Service Bus.

1. Log in to the Azure Console as an administrator and search for “Service Bus”.

2. Click “Create” to add a new Service Bus.

Complete the form details as follows:

• Subscription: Select your Azure subscription from the drop-down menu
• Resource Group: Select your Resource Group
• Namespace name: Provide a name to help distinguish the service bus for InsightCloudSec usage
• Note: InsightCloudSec recommends including the short, unique installation ID provided to you on the "Create Azure EDH Configuration" window to ensure the data source is only being consumed by single installation. This window can be found by logging into your InsightCloudSec instance then navigating to Clouds -> EDH Consumers -> EDH Configuration -> Azure ServiceBus Consumer.
• Location: Using the default is fine, update if you'd like to run this in a different location
• Pricing Tier: Select the "standard" pricing tier from the drop-down menu

3. Ensure you keep the resource group and namespace name on hand for when you configure InsightCloudSec.

4. Click “Review + Create” to complete this step. This will take a few minutes to finalize. You will receive a success confirmation once the Service Bus setup is complete.

📘

Consumer & Producer Visibility

Deploying a service bus and namespace to the same subscription as the event grid will mean that this subscription is both a consumer and a producer; however, InsightCloudSec will only surface it as a consumer in the UI (events will still be visible).

Create a Queue for Your Service Bus

1. From your new Service Bus click on the name to open the Service Bus resource page.

2. Navigate to “Entities → Queues” on the main navigation (left-side) and select "Queues".

3. Click the “+Queue” button to create a new queue with the following settings:

• Name: Provide a unique name
• Max queue size: 5GB
• Max delivery count: 25 (to account for any connection issues)
• Message TTL: 5 hours (can be customized)
• Lock duration: 5 minutes (locks messages to a batch)
• Enable duplicate detection: Select this option and set to 15 minutes

4. Ensure you keep the queue name on hand for when you configure InsightCloudSec.

5. Once you have completed the form with the required fields, click “Create” to finalize the creation of the queue.

Add Role Assignment for Queue

1. From your new Service Bus queue list, select the queue to open the detail page.

2. Navigate to “Access Control (IAM)” on the main navigation (left-side) and select "Check Access" tab.

3. Within the card "Grant Access to this Resource", click “Add Role Assignment”.

4. Select the built- in "Azure Service Bus Data Receiver" role by clicking on the row and select "Next".

5. Complete the form as follows:

• Assign access to: User, group, or service principle
• Members: Click "+ Select Members" and search for the InsightCloudSec Service Principal used for harvesting in this subscription.

Create an Event Grid Subscription

📘

Capturing Your Subscriptions

You will need to repeat the steps outlined below for each individual subscription that you want to capture events from.

1. From the Azure console, search for “Event Grid Subscriptions” (note that this is not the same as an Azure subscription).

2. Click the “+Event Subscription” button to create a new event subscription.

3. Complete the form as follows:
- Name: rapid7-insightcloudsec-edh-eventgrid-subscription

• Event Schema: default
• Topic Types: Azure Subscriptions
• Subscription: Select your subscription
• Resource Group: Select your Resource Group
• System Topic Name: rapid7-insightcloudsec-eventgrid-topic
  • Note: If a topic for this Azure Subscription already exists, the default will automatically be selected.
• Event Types: Select only “Write Success” and “Delete Success”
• Endpoint Type: Select the Service Bus you previously created

4. Under the “Additional Features” tab, locate “Retry Policies” and update the Max Delivery Attempts to “30” and Event Time to Live to “23 hours”.

5. Click the “Create” button to complete the creation of your Event Subscription.

Navigate to the Event Grid System Topics to locate your new parent topic and confirm the queue.

Configure InsightCloudSec

After completing the setup within the Azure Console, you're ready to configure an Azure EDH consumer within InsightCloudSec.

🚧

Prerequisites

Before you can add an Azure ServiceBus Consumer to InsightCloudSec, you will need the following on hand:

• The resource group and namespace name for the Service Bus (created in Create a New Service Bus)
• The queue name for the queue created within the Service Bus (created in Create a Queue for Your Service Bus)
• The name of the subscription that contains the Service Bus (created in Azure Setup - Single Cloud)

1. Login to your InsightCloudSec platform and click "Clouds" in the left-hand navigation menu.

• Click "Consumers".
• Click "EDH Configuration".
• From the drop-down menu, click "Azure Service Bus Consumer".

2. Update the configuration for the necessary information.

• Select the Azure Subscription that contains the service bus.
• Provide the service bus ID in the format of <resource_group_name>|<namespace_id>|<queue_name>, ensuring you replace the placeholders with the appropriate values.
• Click "Configure".

Post Setup Information

Congratulations on setting up Event Driven Harvesting for your Azure Subscription(s) within InsightCloudSec. Below you'll find some important links about EDH in general.

• EDH Event Summaries and Reports
• Harvesting Failure Messages