Managing External Data Resources

📘

Using the API Documentation

This API documentation is currently offered “as-is” and as such we want to provide the following recommendations:

If you are not familiar with our API, or are working with these capabilities for the first time, we strongly recommend that you coordinate with your CSM or our support team.

We make this recommendation because some use cases may require additional clarification and we are here to help. Working with us directly will ensure that you are able to use our API effectively for whatever goals you have.

As part of our commitment to a great customer experience we are actively working on productizing our API. This includes outlining a hardened and repeatable standard for future endpoints and identifying common/high-impact use cases for verification and possibly revision/versioning.

Also, where possible, there are example requests and responses for the documented endpoints. These are available in the "EXAMPLES" drop-down menus within the "REQUEST" and "RESPONSE" sections on the right-hand side of an endpoint page. You can also check out some high-level Getting Started information here.

If you have questions or concerns regarding the content here, or need support using our API reach out to us through the Customer Support Portal.

Configuring External Data File

Before InsightCloudSec can utilize the external data file as an External Data Source, you'll need to pick existing resources in InsightCloudSec to gather more information from and you'll need to upload the data file to an S3 bucket that InsightCloudSec can access.

Picking Resources

The external data file must contain a JSON object for each AWS ARN that extra information will be harvested from. The AWS ARN can be found in InsightCloudSec via the Resources interface:

1. Log into the desired InsightCloudSec instance.

2. From the navigation menu on the left, click "Resources".

3. Navigate to the desired AWS resource type.

4. From the list of resources, click "Open Resource Properties" next to the desired resource.

5. Copy the "Namespace Id" to somewhere on your computer that you can easily access.

Creating and Uploading the File

Currently, only .json files are supported; the support value types are string, boolean, integer, list, and object. The file should follow a similar format to the following example, ensuring the Namespace IDs you copied in the section above are pasted into the file where necessary:

[
  {
    "arn": "<resource-1-arn>",
    "data": {
      "<parameter1>": <integer-value>,
      ...
      "<parameterN>": "<string-value>"
    }
  },
  ...
  {
    "arn": "<resource-N-arn>",
    "data": {
      "<parameter1>": [
        "<list-value1>",
        "<list-value2>"
      ],
      ...
      "<parameterN>": {
        "<some-key>": "<some-value>"
      }
    }
  }
]

Once the file is ready, upload it to an S3 bucket that InsightCloudSec recognizes or can recognize.

🚧

Duplicate Resources and Attributes

• Two external data sources cannot write to the same resource, so ensure the external data file references each resource ARN only once.
• If there is already an attribute in the file associated with a resource from some other source file and it's been processed by InsightCloudSec, the same attribute and resource pair detected from any subsequent sources will be ignored
• If there are two ore more occurrences of an attribute with a given resource in the same source file, the last pair to be read in the file will overwrite the previous occurrences.

Creating the External Data Source

Follow the API documentation found on Create Data Source to properly create an external data source. Once the data source has been created, visit the "System Administration > Background Jobs" page in InsightCloudSec and enqueue an ResourceExternalDataFetcher job to pull the data.

📘

External Data Fetcher Cadence

This job runs everyday for all enabled data sources across all InsightCloudSec organizations. If there are changes to attributes and resource pairs in an existing source file that has been fetched previously, the changes will be applied in the subsequent fetching.

Once the external data is available, you can use Jinja2 templating to access the data using Bots. Review Jinja2 - References for more information on Jinja2 templating.