GCP Event-Driven Harvesting

An Introduction to the InsightCloudSec Feature for Event-Driven Harvesting for Google Cloud Platform (GCP)

InsightCloudSec has the capability to augment standard polling-based harvesting with Event-Driven Harvesting (EDH). For Google Cloud Platform (GCP), InsightCloudSec subscribes to real-time notifications about resource and policy changes using a Cloud Asset Inventory feed, which triggers targeted harvesting via Pub/Sub.

This dynamic approach to data collection both improves InsightCloudSec's cadence for providing resource visibility and opportunities for remediation as well as enriches the data with lifecycle changes that enable auditing capabilities. With EDH-provided data, identifying how a resource entered a noncompliant state becomes much easier at scale.

Understanding EDH Concepts

EDH relies on a relationship between a Producer and Consumer. InsightCloudSec requires a way to pull event data out of GCP Projects and with minimal infrastructure required. This is done with Pub/Sub and a Cloud Asset Inventory feed. If the Project has a subscription hooked up to a feed, InsightCloudSec automatically labels the GCP Project as an EDH Consumer.

  • Topic: Method for getting events out of the Project and into InsightCloudSec
  • Consumer: Project that contains a Pub/Sub subscription associated with the topic
  • Producer: Feed configuration that sends events to the topic
  • Service Account: credentials used to pull events from the consumer

For a single GCP Project, InsightCloudSec consumes events from a Pub/Sub subscription, which in turn subscribes to events from the Cloud Asset Inventory feed. For each additional GCP Project, a feed and topic can be configured to forward events to the initial subscription, allowing for InsightCloudSec to consume events from a single subscription.

EDH Supported Resources

For a complete list of supported GCP resources, refer to the EDH - Supported Resources (GCP) page.

Configuration Details

For details on setting up EDH for GCP, check out our EDH - GCP Setup documentation.