Kubernetes Security Guardrails
Details on Working with the InsightCloudSec Kubernetes Guardrails Feature.
Kubernetes Security Guardrails (K8s Guardrails) helps you harden your production environment by auditing your Kubernetes cluster, nodes, and pods configuration. This ensures that your cluster is tuned and runs according to security best practices and internal guidelines. Kubernetes Guardrails provides actionable recommendations for risk mitigation. You can easily shift-left to resolve security and compliance issues in the development stage before moving to production.
For any questions about setup, configuration, or support reach out to your CSM or to support through our Customer Support Portal.
Getting Started
Prerequisites
Before getting started with Kubernetes Security Guardrails you will need to have the following:
- A functioning InsightCloudSec Platform installation (must be at least v. 21.7.0)
- InsightCloud Sec Admin permissions (Domain or Org Admin)
- Familiarity using Helm
- Familiarity using
kubectl
- Access to the license server (e.g. if you do not see the "Manage Kubernetes API Key" option on the Clouds page this feature is not enabled)
- Reach out to your CSM or to support through the Customer Support Portal to request access


Manage Kubernetes API Key
If you are not experienced with the tools identified above we recommend working closely with us or coordinating with your own DevOps resources for support.
What is Supported?
- Multi-cluster vulnerability scanner
- A summary of your cluster’s compliance and security status
- A detailed list of compliance and security issues, followed by recommendation for a quick remediation
- Baseline your clusters profile and easily monitor degradations
- Note: Our current setup can support up to two API keys to enable key rotation. The clusters will be installed as a single Organization (within InsightCloudSec).
Working with Kubernetes Security Guardrails
This documentation provides instructions on the following:
- Using the InsightCloudSec User Interface or the InsightCloudSec API to update discovery of any clusters that are not currently reflected in your InsightCloudSec platform
- Verifying your local configuration for deployment using Helm and kubectl
Managed vs. Unmanaged Clusters
Managed Kubernetes Clusters are Kubernetes Clusters where the control plain is installed and managed by the cloud providers. (Examples are AWS EKS, GCP EKE, Azure AKS, etc.)
Unmanaged Kubernetes Clusters are Kubernetes Clusters where the control plain is installed and managed by the user.
Kubernetes Security guardrails can be use for both managed and unmanaged clusters. The only difference is the cluster ID that is used:
- For managed clusters, the cluster ID should be the one that was harvested by ICD from the cloud account
- For unmanaged clusters the cluster ID is any string that is chosen by the user.
Generating an API Key
Generating an API Key is required to identify and authenticate the Guardrails scanners (one on each cluster) and allow the scanner to report inventory and assessment findings to the InsightCloudSec platform.
1. Navigate to your InsightCloudSec installation and click to open "Cloud --> Clouds".


Add a Kubernetes API Key
2. On the top of the top of the page click the "Add Kubernetes API Key" button.
3. Provide a name for your API key and click "Create API Key"
4. Copy the newly generated API key and store it in a safe place. *Note: This will be your only opportunity to save this information.


Save your API Key
Manage Existing API Keys
After you have generated an API Key the button dynamically updates to read "Manage Kubernetes API Key". Our current setup supports up to two API keys for API key rotation. The clusters will be installed as a single Organization (within InsightCloudSec).
Clicking on this button will enable you to generate new API Keys, manage their status (activated, deactivated) and delete unused keys.


Managing your Kubernetes API Keys
Verifying Configuration Requirements
Before using the InsightCloudSec Kubernetes Security Guardrails feature you will need to verify that your local machine is set up with helm
and kubectl
.
To do this you can run the helm
and kubectl
commands (individually) to set the correct context against your Kubernetes cluster. Helm is required to install the Guardrails scanner.
Setup for kubectl
If you do not have an existing kubectl
setup refer to the following in order to connect to your Kubernetes cluster:
- To install
kubectl
on OSX - To install
kubectl
on Linux - To install
kubectl
on Windows
Setup for Helm
If you do not have an existing heml
install setup refer to the following in order to connect your Kubernetes cluster:
1. Download and install Helm.
Steps 2- 6 below should be executed on all designated clusters.
2. Connect to the cluster context that you would like to install k8s guardrails.
- For GKE clusters
- For EKS clusters
- For AKS clusters
- For local clusters such as kind, minikube, or kubeadm
3. Add the K8s guardrails Helm repo by issuing the following commands:
- Refer to the Notes for Helm Install Commands for details.
helm repo add helm-repo https://helm.rapid7.com/cloudsec
helm search repo
helm install k8s-guardrails helm-repo/k8s-guardrails -n rapid7 --create-namespace \
--set K8sGuardrails.ApiToken=<InsightCloudSec-API-token> \
--set Config.ClusterName=<InsightCloudSec-Cluster-Name> \
--set Config.Labels=<InsightCloudSec-Cluster-Badges> \
--set CronSchedule=<k8sGuardrails-CronSchedule> \
--set Config.BaseUrl=<InsightCloudSec-Base-URL> \
--set Config.ClusterId=<InsightCloudSec-Cluster-ID> \
--set devopscurlSpec.SelfSignedCert.Enabled=<Enable-Self-Cert> \
--set devopscurlSpec.SelfSignedCert.CertSecretName=<Self-Cert-Secret-Name>
The last --set
command can also be replaced with the following:
--set devopscurlSpec.SelfSignedCert.CertPem=<Self-Cert-Pem-Base64>
Specifying Resource Limits
InsightCloudSec includes the ability to specify resource limits and requests for Guardrails containers.
The helm key to set should start with the following YAML hierarchy:
"<container spec>.Resources."
following the wanted resources requests/limits, where <container spec>
is one of "advisorSpec/mergerSpec/inventoryscannerSpec/exporterSpec/devopscurlSpec"
advisorSpec.Resources.requests.cpu=200m
advisorSpec.Resources.requests.memory=100Mi
advisorSpec.Resources.limits.cpu=1
advisorSpec.Resources.limits.memory=1Gi
For more info and how to configure refer to Kubernetes documentation on Resource Management for Pods and Containers
Notes for Helm Install Commands
Property | Description | Instructions |
---|---|---|
| Mandatory. | If unknown, API URL can be retrieved from the InsightCloudSec interface:
This URL should be used alongside the path to the endpoint: See example: |
| Mandatory. Note: Ensure you generate a secure API token. | In the interface, navigate to Select “Add k8s API Key”. Enter the API Key to be used to link with your Kubernetes clusters. |
| Mandatory. | |
| Mandatory. Must match the ARN field of the discovered cluster in order to correlate correctly and to be able to generate coverage reports. | Navigate to the Resource page or use API to get “Kubernetes Cluster Without Guardrails Report”. Reports contain ARN for each cluster. Refer to Discovery for Existing Clusters for details. |
| Optional. The cluster badges, if provided, will be translated into cloud account (cluster) badges that you can user later on to navigate/filter Insight findings. | Example of Cluster-Badges: |
| Optional. Creates periodic and recurring tasks to run the Guardrails scanner. Default scanning schedule (if not specified) is once an hour. | For CronJob Scheduling refer to the following information. |
| Mandatory in order to enable feature. | Boolean type. |
| Optional Can be replaced by Self-Cert-Pem-Base64. | Create a secret in the same namespace and pass the secret name. |
| Optional Can be replaced by Self-Cert-Secret-Name. | Pass a base64 encoded certificate. If this option is used, ensure value passed via an inline paramater using the --set flag and not hardcoded in the values.yaml file. |
4. To verify that k8s-guardrails works successfully, you will need to trigger a job manually, using the following command.
kubectl create job --from=cronjob/k8s-guardrails -n rapid7 k8s-guardrails-manual-001
5. Verify that the pod is in the completed status.
- Time to completion status will depend on the size of the cluster.
kubectl get pods -nrapid7 | grep k8s-guardrails-manual-001
6. Verify that the cluster is marked as “monitored” and that resources appear with findings on them.
Discovery for Existing Clusters
To identify clusters that are not currently covered you will need to refer to the following steps. Below are instructions to identify clusters using the InsightCloudSec UI or the InsightCloudSec API.
Discovery of Clusters Using the UI
1. From your InsightCloudSec platform installation, navigate to "Resource --> Resources" and select the "Containers" tab.
2. (Optionally) Use the "Scopes" button at the top of the page to narrow the scope (e.g., cloud accounts, resource groups) to use when scanning for clusters that are not yet included in your InsightCloudSec setup.
3. From the Containers tab, select “Clusters” to see a list of all of the clusters included in the selected scope.
4. Navigate to “Filters” and search for/select the “Kubernetes Cluster Without Guardrails Report”.


InsightCloudSec Platform - Filter for Clusters without Guardrails
- Selecting this filter will update the display to include clusters that have not been scanned.
- The cluster ID field that displays will be used when deploying Guardrails to a specific cluster.
5. Locate the "Cluster ID" column and note the Cluster you want to deploy Guardrails in.
- Note: You will have to scroll to the right to see all of the columns


Cluster ID for Containers
Cluster ID
InsightCloudSec uses the Cluster ID to identify clusters. Using the Cluster ID allows us to correlate between clusters discovered via the InsightCloudSec platform (either through the UI or API) vs. clusters onboarded through the Alcide scanning capability.
Discovery of Clusters Using the API
For information on using the InsightCloudSec API refer to the Getting Started.
1. Login to the InsightCloudSec (DivvyCloud) API using your username and password in the request body in a POST
to v2/public/user/login
.
2. Use the session_id
from the response in the X-Auth-Token header
. Use the following request body in a POST
to v2/public/resource/query:
{
"selected_resource_type": "containercluster",
"filters": [{
"name": "divvy.query.kubernetes_cluster_without_guardrails_report",
"config": {}
}],
"offset": 0,
"limit": 100
}
3. The resources
list will display clusters that have not been scanned; the ARN
field will be used when deploying Guardrails to a specific cluster.
- Save the ARN details for clusters where you want to configure Kubernetes Security Guardrails.
Using Kubernetes Security Guardrails
These steps assume that you have enabled Kubernetes Security Guardrails feature via Helm.
Refer to the instructions below to identify and use Insights that apply to the findings generated for your Kubernetes cluster by Guardrails
- Refer to our Insights documentation for detailed information on this feature.
- You can also find detailed instructions on Creating a Custom Pack
- If you have questions or need assistance reach out to us through the Customer Support Portal.
View Insights Related to Kubernetes Security Guardrails
1. From InsightCloudSec navigate to "Security → Insights".
2. Select the logo from the list of cloud providers.
- Note: If you have existing Kubernetes clusters there may be two logos. The one with the dot to the top right is for Insights that support Kubernetes Security Guardrails.
3. Only Insights related to the Kubernetes cluster security will display.
- Further filtering using the Scopes function will allow you to narrow your search to identify issues with specific clusters or namespaces.
- Note that namespaces will be available as a "type" of Resource Group, and as such will appear within the navigation panel that includes Resource Groups.


View Insights for Kubernetes Security Guardrails
Create a Custom Pack for Kubernetes Security Guardrails
Many of these Insights are included in the “Center for Internet Security (CIS) - Kubernetes (1.6.0)” Compliance Packs. However, it is also possible to create Custom Packs.
1. Navigate to "Security → Insights".
2. Select "Custom Packs" and click the “CREATE PACK” button.
- Provide a name, description, and complete any other options you desire.
3. Click “Submit”.
4. From "Security --> Insights", select Library
- Select the logo from the list of cloud logos
- Select the Insights you wish to add to your custom pack with the checkbox icon
5. Click the “ACTIONS” button and choose “Add to pack” from the menu
6. Locate your custom pack and select to add the selected Insights.
Viewing K8s Details in the Compliance Scorecard
In addition to viewing information about your clusters and namespaces through Insights, you can also view this information in the Compliance Scorecard. Simply select Cluster/Namespace (this is a filter mode that is specific to the Kubernetes Security Guardrails feature) or Cloud/Cluster (to include *all Clusters - those previously added and anything new) and filter as desired.
*Important Note: The Cluster/Namespace filter mode will only display namespaces within the Cluster selected, if you select a single Cluster.*
- Take a look at our Compliance Scorecard documentation for complete details on this feature.


Using the Compliance Scorecard with K8s
Updated 24 days ago