GCP Event-Driven Harvesting
InsightCloudSec has the capability to augment standard polling-based harvesting with Event-Driven Harvesting (EDH). For Google Cloud Platform (GCP), InsightCloudSec subscribes to real-time notifications about resource and policy changes using a Cloud Asset Inventory feed, which triggers targeted harvesting via Pub/Sub.
This dynamic approach to data collection both improves InsightCloudSec's cadence for providing resource visibility and opportunities for remediation as well as enriches the data with lifecycle changes that enable auditing capabilities. With EDH-provided data, identifying how a resource entered a noncompliant state becomes much easier at scale.
Understanding EDH Concepts
EDH relies on a relationship between a Producer and Consumer. InsightCloudSec requires a way to pull event data out of GCP Projects and with minimal infrastructure required. This is done with Pub/Sub and a Cloud Asset Inventory feed. If the Project has a subscription hooked up to a feed, InsightCloudSec automatically labels the GCP Project as an EDH Consumer.
- Topic: Method for getting events out of the Project and into InsightCloudSec
- Consumer: Project that contains a Pub/Sub subscription associated with the topic
- Producer: Feed configuration that sends events to the topic
- Service Account: credentials used to pull events from the consumer
For a single GCP Project, InsightCloudSec consumes events from a Pub/Sub subscription, which in turn subscribes to events from the Cloud Asset Inventory feed. For each additional GCP Project, a feed and topic can be configured to forward events to the initial subscription, allowing for InsightCloudSec to consume events from a single subscription.
EDH Supported Resources
For a complete list of supported GCP resources, refer to the EDH - Supported Resources (GCP) page.
Configuration Details
For details on setting up EDH for GCP, check out our EDH - GCP Setup documentation.