Installing Remote Plugins
Information on Installing Remote Plugins for InsightCloudSec
Expanded Plugin Documentation
InsightCloudSec supports extension via a comprehensive plugin system. Customers can add new content and workflows to support their unique requirements by writing their own extensions. This capability requires access to our source code and is only available to customers.
Reach out to us through the the Customer Support Portal for access to both the repo and the additional documentation and instructions surrounding plugins.
To ease plugin deployment, you can configure InsightCloudSec to download plugins from a remote location on system startup. The application can download these files from S3 or from arbitrary HTTP hosting, optionally with password protection.
Prerequisites
You must have InsightCloudSec plugins in .zip
files. Each .zip
archive should either contain exactly one plugin or a directory consisting of multiple plugins.
That .zip
file must be hosted in:
- an S3 bucket that your InsightCloudSec installation already has access to via the
s3:GetObject
permission - an S3 bucket that is either protected by basic auth or is open to the world
- an arbitrary HTTP-accessible location, protected by basic auth or open to the world
- access to the all EC2 instances hosting the Interface Server, Scheduler and Worker containers
DivvyCloud vs. InsightCloudSec
Note that examples, variables, and database components may still refer to DivvyCloud vs. InsightCloudSec - the functionality is the same.
Interface
The current interface consists of a collection of environment variables, shown below:
Name | Description | Default |
---|---|---|
DIVVY_PLUGIN_REMOTE | This contains all remote plugin configuration settings as a JSON string. See data format below. Note: If this environment variable is set, the application will automatically use scheduler-coordinated plugin loading, unless the DIVVY_PLUGIN_LEGACY_LOADING or DIVVY_PLUGIN_DEV environments are set to True . | {} |
DIVVY_PLUGIN_DEPLOY_DIR | Environment variable which can be used to set the deployment directory when using remote plugin syncing. | <SystemTmp>/divvy/deployed_plugins> |
The DIVVY_REMOTE_PLUGIN
environment variable is a list of JSON objects, each object representing a remote plugin store configuration described in this table:
Name | Required | Description | Default |
---|---|---|---|
url | Yes | Full URL to a compressed plugin(s). The application will use the prefix on the URL to identify the specific remote resource as described below. | N/A |
multi_plugins | No | If True, InsightCloudSec will extract the file in the url, and inspect each sub-folder as a unique plugin. | False |
username | No | Optional username to use if required by the url | N/A |
password | No | Optional password if required to access the url | N/A |
AWS S3 Bucket:
If the prefix of the url starts with s3://
the application will support the following behavior.
If no username or password are specified, the application will assume that the InsightCloudSec scheduler is properly set up to support Instance AssumeRole to access that bucket.
If the username and password are specified, the application will attempt to authenticate with those credentials where username is the API Key and password is the Secret Key.
HTTP/HTTPS storage
If the url starts with http://
or https://
and a username/password are supplied, the application will use basic auth to attempt to authenticate with the endpoint. This means that the application will support fetching a private git repo.
Example
For example, you could specify the following:
export DIVVY_PLUGIN_REMOTE='
[{"url":"s3://bucket_name/plugin.zip"},
{"url":"https://test/many_compressed_plugins.zip", "multi_plugins":true},
{"url": "http://github.com/example/Plugins/archive/master.zip", "multi_plugins":true, "username": "xxxxxx", "password": "yyyyyy"}]
'
Use this configuration:
- fetches an archive of a single plugin from S3 using Instance AssumeRole (
{"url":"s3://bucket_name/plugin.zip"}
), - fetches, unauthenticated, a zipfile consisting of many plugins, from a non-GitHub endpoint (
{"url":"https://test/many_compressed_plugins.zip", "multi_plugins":true}
), - fetches, using basic authentication, an archive of many plugins hosted in a GitHub repo (
{"url": "http://github.com/example/Plugins/archive/master.zip", "multi_plugins":true, "username": "xxxxxx", "password": "yyyyyy"}
).
Steps to Implementing
Note: This change should be added to prod.env on all EC2 instances
SSH to the EC2 instance, locate the directory where prod.env is saved, stop the docker container before updating:
docker-compose down
Ensure the containers are stopped.
Update/Modify the prod.env file:
vi prod.env
# MySQL 5.7 Primary database
DIVVY_DB_HOST=mysql
DIVVY_DB_PORT=3306
DIVVY_DB_USERNAME=divvy
DIVVY_DB_PASSWORD=divvy
# MySQL 5.7 Secure database
DIVVY_SECRET_DB_HOST=mysql
DIVVY_SECRET_DB_PORT=3306
DIVVY_SECRET_DB_USERNAME=divvy
DIVVY_SECRET_DB_PASSWORD=divvy
# Redis
DIVVY_REDIS_HOST=redis
DIVVY_REDIS_PORT=6379
becomes
# MySQL 5.7 Primary database
DIVVY_DB_HOST=mysql
DIVVY_DB_PORT=3306
DIVVY_DB_USERNAME=divvy
DIVVY_DB_PASSWORD=divvy
# MySQL 5.7 Secure database
DIVVY_SECRET_DB_HOST=mysql
DIVVY_SECRET_DB_PORT=3306
DIVVY_SECRET_DB_USERNAME=divvy
DIVVY_SECRET_DB_PASSWORD=divvy
# Redis
DIVVY_REDIS_HOST=redis
DIVVY_REDIS_PORT=6379
DIVVY_PLUGIN_REMOTE='[{"url":"s3://bucket_name/plugin.zip"}]'
Start up the containers to apply the change:
docker-compose up
Verify the variables have been set in the container:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a512082abfe divvycloud/divvycloud:latest "/entrypoint.sh divv…" 35 minutes ago
docker exec -it 0a512082abfe bash
root@1b5bb7bebc37:/# env | grep DIVVY_PLUGIN_REMOTE
DIVVY_PLUGIN_REMOTE=[{"url":"s3://bucket_name/plugin.zip"}]
Advanced Configuration for Plugin Developers
The following environment variables are for use if you are developing plugins locally:
Name | Description | Default |
---|---|---|
DIVVY_PLUGIN_DEV | Controls if InsightCloudSec should be in Plugin Development Mode. Currently this has the same affect as setting the DIVVY_PLUGIN_LEGACY_LOADING variable.Will evaluate to True if the value is anything other than "" or "False" . | False |
DIVVY_PLUGINS | This environment variable contains the path to “locally” installed plugins. | “plugins” folder at the project/deploy root folder. |
Updated almost 2 years ago