Take Me to the Docs!API ReferenceRelease NotesSupport Portal
Contact Us

Installing Remote Plugins

Information on Installing Remote Plugins for InsightCloudSec

Suggest Edits

📘

Expanded Plugin Documentation

InsightCloudSec supports extension via a comprehensive plugin system. Customers can add new content and workflows to support their unique requirements by writing their own extensions. This capability requires access to our source code and is only available to customers.

Reach out to us through the the Customer Support Portal for access to both the repo and the additional documentation and instructions surrounding plugins.

To ease plugin deployment, you can configure InsightCloudSec to download plugins from a remote location on system startup. The application can download these files from S3 or from arbitrary HTTP hosting, optionally with password protection.

Prerequisites

You must have InsightCloudSec plugins in .zip files. Each .zip archive should either contain exactly one plugin or a directory consisting of multiple plugins.

That .zip file must be hosted in:

  • an S3 bucket that your InsightCloudSec installation already has access to via the s3:GetObject permission
  • an S3 bucket that is either protected by basic auth or is open to the world
  • an arbitrary HTTP-accessible location, protected by basic auth or open to the world
  • access to the all EC2 instances hosting the Interface Server, Scheduler and Worker containers

📘

DivvyCloud vs. InsightCloudSec

Note that examples, variables, and database components may still refer to DivvyCloud vs. InsightCloudSec - the functionality is the same.

Interface

The current interface consists of a collection of environment variables, shown below:

NameDescriptionDefault
DIVVY_PLUGIN_REMOTEThis contains all remote plugin configuration settings as a JSON string. See data format below.

Note: If this environment variable is set, the application will automatically use scheduler-coordinated plugin loading, unless the DIVVY_PLUGIN_LEGACY_LOADING or DIVVY_PLUGIN_DEV environments are set to True.		{}
DIVVY_PLUGIN_DEPLOY_DIREnvironment variable which can be used to set the deployment directory when using remote plugin syncing.<SystemTmp>/divvy/deployed_plugins>

The DIVVY_REMOTE_PLUGIN environment variable is a list of JSON objects, each object representing a remote plugin store configuration described in this table:

NameRequiredDescriptionDefault
urlYesFull URL to a compressed plugin(s). The application will use the prefix on the URL to identify the specific remote resource as described below.N/A
multi_pluginsNoIf True, InsightCloudSec will extract the file in the url, and inspect each sub-folder as a unique plugin.False
usernameNoOptional username to use if required by the urlN/A
passwordNoOptional password if required to access the urlN/A

AWS S3 Bucket:

If the prefix of the url starts with s3:// the application will support the following behavior.

If no username or password are specified, the application will assume that the InsightCloudSec scheduler is properly set up to support Instance AssumeRole to access that bucket.

If the username and password are specified, the application will attempt to authenticate with those credentials where username is the API Key and password is the Secret Key.

HTTP/HTTPS storage

If the url starts with http:// or https:// and a username/password are supplied, the application will use basic auth to attempt to authenticate with the endpoint. This means that the application will support fetching a private git repo.

Example

For example, you could specify the following:

export DIVVY_PLUGIN_REMOTE='
[{"url":"s3://bucket_name/plugin.zip"},
{"url":"https://test/many_compressed_plugins.zip", "multi_plugins":true}, 
{"url": "http://github.com/example/Plugins/archive/master.zip", "multi_plugins":true, "username": "xxxxxx", "password": "yyyyyy"}]
'

Use this configuration:

  • fetches an archive of a single plugin from S3 using Instance AssumeRole ({"url":"s3://bucket_name/plugin.zip"}),
  • fetches, unauthenticated, a zipfile consisting of many plugins, from a non-GitHub endpoint ({"url":"https://test/many_compressed_plugins.zip", "multi_plugins":true}),
  • fetches, using basic authentication, an archive of many plugins hosted in a GitHub repo ({"url": "http://github.com/example/Plugins/archive/master.zip", "multi_plugins":true, "username": "xxxxxx", "password": "yyyyyy"}).

Steps to Implementing

Note: This change should be added to prod.env on all EC2 instances

SSH to the EC2 instance, locate the directory where prod.env is saved, stop the docker container before updating:

docker-compose down

Ensure the containers are stopped.
Update/Modify the prod.env file:

vi prod.env

# MySQL 5.7 Primary database
DIVVY_DB_HOST=mysql
DIVVY_DB_PORT=3306
DIVVY_DB_USERNAME=divvy
DIVVY_DB_PASSWORD=divvy

# MySQL 5.7 Secure database
DIVVY_SECRET_DB_HOST=mysql
DIVVY_SECRET_DB_PORT=3306
DIVVY_SECRET_DB_USERNAME=divvy
DIVVY_SECRET_DB_PASSWORD=divvy

# Redis
DIVVY_REDIS_HOST=redis
DIVVY_REDIS_PORT=6379

becomes

# MySQL 5.7 Primary database
DIVVY_DB_HOST=mysql
DIVVY_DB_PORT=3306
DIVVY_DB_USERNAME=divvy
DIVVY_DB_PASSWORD=divvy

# MySQL 5.7 Secure database
DIVVY_SECRET_DB_HOST=mysql
DIVVY_SECRET_DB_PORT=3306
DIVVY_SECRET_DB_USERNAME=divvy
DIVVY_SECRET_DB_PASSWORD=divvy

# Redis
DIVVY_REDIS_HOST=redis
DIVVY_REDIS_PORT=6379
DIVVY_PLUGIN_REMOTE='[{"url":"s3://bucket_name/plugin.zip"}]'

Start up the containers to apply the change:

docker-compose up

Verify the variables have been set in the container:

docker ps

CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS                         PORTS                     NAMES
0a512082abfe        divvycloud/divvycloud:latest   "/entrypoint.sh divv…"   35 minutes ago 

docker exec -it 0a512082abfe bash

[email protected]:/# env | grep DIVVY_PLUGIN_REMOTE
DIVVY_PLUGIN_REMOTE=[{"url":"s3://bucket_name/plugin.zip"}]

Advanced Configuration for Plugin Developers

The following environment variables are for use if you are developing plugins locally:

NameDescriptionDefault
DIVVY_PLUGIN_DEVControls if InsightCloudSec should be in Plugin Development Mode. Currently this has the same affect as setting the DIVVY_PLUGIN_LEGACY_LOADING variable.

Will evaluate to True if the value is anything other than "" or "False".		False
DIVVY_PLUGINSThis environment variable contains the path to “locally” installed plugins.“plugins” folder at the project/deploy root folder.

Updated over 1 year ago