Kubernetes Security Guardrails

Kubernetes Security Guardrails (K8s Guardrails) helps you harden your production environment by auditing your Kubernetes cluster, nodes, and pods configuration. This ensures that your cluster is tuned and runs according to security best practices and internal guidelines. Kubernetes Guardrails provides actionable recommendations for risk mitigation. You can easily shift-left to resolve security and compliance issues in the development stage before moving to production.

For any questions about setup, configuration, or support reach out to your CSM or to support through our Customer Support Portal.

Getting Started

Prerequisites

Before getting started with Kubernetes Security Guardrails you will need to have the following:

• A functioning InsightCloudSec Platform installation (must be at least v. 21.7.0)
• InsightCloud Sec Admin permissions (Domain or Org Admin)
• Familiarity using Helm
• Familiarity using kubectl
• Access to the license server (e.g. if you do not see the "Manage Kubernetes API Key" option on the Clouds page this feature is not enabled)
  • Reach out to your CSM or to support through the Customer Support Portal to request access
• Clusters to be monitored by Guardrails must have network connectivity to the ICS server IP and Port

If you are not experienced with the tools identified above we recommend working closely with us or coordinating with your own DevOps resources for support.

What is Supported?

• Multi-cluster vulnerability scanner
• A summary of your cluster’s compliance and security status
• A detailed list of compliance and security issues, followed by recommendation for a quick remediation
• Baseline your clusters profile and easily monitor degradations
• Note: Our current setup can support up to two API keys to enable key rotation. The clusters will be installed as a single Organization (within InsightCloudSec).

Working with Kubernetes Security Guardrails

This documentation provides instructions on the following:

• Using the InsightCloudSec User Interface or the InsightCloudSec API to update discovery of any clusters that are not currently reflected in your InsightCloudSec platform
• Verifying your local configuration for deployment using Helm and kubectl

Managed vs. Unmanaged Clusters

Managed Kubernetes Clusters are Kubernetes Clusters where the control plane is installed and managed by the cloud providers. (Examples are AWS EKS, GCP EKE, Azure AKS, etc.)

Unmanaged Kubernetes Clusters are Kubernetes Clusters where the control plane is installed and managed by the user.

Kubernetes Security guardrails can be use for both managed and unmanaged clusters. The only difference is the cluster ID that is used:

• For managed clusters, the cluster ID should be the one that was harvested by ICD from the cloud account
• For unmanaged clusters the cluster ID is any string that is chosen by the user.

Generating an API Key

Generating an API Key is required to identify and authenticate the Guardrails scanners (one on each cluster) and allow the scanner to report inventory and assessment findings to the InsightCloudSec platform.

1. Navigate to your InsightCloudSec installation and click to open "Cloud --> Clouds".

2. On the top of the top of the page click the "Add Kubernetes API Key" button.

3. Provide a name for your API key and click "Create API Key"

4. Copy the newly generated API key and store it in a safe place. *Note: This will be your only opportunity to save this information.

Manage Existing API Keys

After you have generated an API Key the button dynamically updates to read "Manage Kubernetes API Key". Our current setup supports up to two API keys for API key rotation. The clusters will be installed as a single Organization (within InsightCloudSec).

Clicking on this button will enable you to generate new API Keys, manage their status (activated, deactivated) and delete unused keys.

📘

Applying New API Keys

Assuming you have applied the suggested naming convention for the Helm repository and installation, the command for updating your Kubernetes Scanner deployment for new API keys should look like:

helm upgrade k8s-guardrails helm-repo/k8s-guardrails -n rapid7 --set K8sGuardrails.ApiToken=<new token>

Verifying Configuration Requirements

Before using the InsightCloudSec Kubernetes Security Guardrails feature you will need to verify that your local machine is set up with helm and kubectl.

To do this you can run the helm and kubectl commands (individually) to set the correct context against your Kubernetes cluster. Helm is required to install the Guardrails scanner.

Setup for kubectl

If you do not have an existing kubectl setup refer to the following in order to connect to your Kubernetes cluster:

• To install kubectl on OSX
• To install kubectl on Linux
• To install kubectl on Windows

Setup for Helm

If you do not have an existing Helm install setup refer to the following in order to connect your Kubernetes cluster:

1. Download and install Helm.

📘

Cluster Network Access Requirements

Steps 2- 6 below should be executed on all designated clusters.

Every cluster must have network access to the InsightCloudSec server's IP and Port.

For SaaS customers, your cluster(s) will be making requests to your InsightCloudSec installation on one of two IP addresses specific to your installation. It is possible to obtain your IP addresses by performing a DNS lookup on your installation’s domain name.

2. Connect to the cluster context that you would like to install k8s guardrails.

• For GKE clusters
• For EKS clusters
• For AKS clusters
• For local clusters such as kind, minikube, or kubeadm

3. Add the K8s guardrails Helm repo by issuing the following commands:

• Refer to the Notes for Helm Install Commands for details.

helm repo add helm-repo https://helm.rapid7.com/cloudsec

helm search repo

helm install k8s-guardrails helm-repo/k8s-guardrails -n rapid7 --create-namespace \
--set K8sGuardrails.ApiToken=<InsightCloudSec-API-token> \
--set Config.ClusterName=<InsightCloudSec-Cluster-Name> \
--set Config.Labels=<InsightCloudSec-Cluster-Badges> \
--set CronSchedule=<k8sGuardrails-CronSchedule> \
--set Config.BaseUrl=<InsightCloudSec-Base-URL>  \
--set Config.ClusterId=<InsightCloudSec-Cluster-ID> \
--set devopscurlSpec.SelfSignedCert.Enabled=<Enable-Self-Cert> \
--set devopscurlSpec.SelfSignedCert.CertSecretName=<Self-Cert-Secret-Name>

The last --set command can also be replaced with the following:
--set devopscurlSpec.SelfSignedCert.CertPem=<Self-Cert-Pem-Base64>

Specifying Resource Limits

InsightCloudSec includes the ability to specify resource limits and requests for Guardrails containers.

The helm key to set should start with the following YAML hierarchy:
"<container spec>.Resources." following the wanted resources requests/limits, where <container spec> is one of "advisorSpec/mergerSpec/inventoryscannerSpec/exporterSpec/devopscurlSpec"

advisorSpec.Resources.requests.cpu=200m 
advisorSpec.Resources.requests.memory=100Mi 
advisorSpec.Resources.limits.cpu=1 
advisorSpec.Resources.limits.memory=1Gi

For more info and how to configure refer to Kubernetes documentation on Resource Management for Pods and Containers

Using a Self-Signed Certificate

Accessing InsightCloudSec from the Kubernetes scanner is done over TLS. While in most cases a public Certificate Authority is used, some organizations use a private Certificate Authority that requires the Kubernetes scanner to be configured with a self-signed certificate.

Configuring a self-signed certificate is done by providing additional parameters to the helm chart installation indicating the use of a self-signed certificate and providing the certificate in the format of base64 encoding.

An example is below:

helm install k8s-guardrails helm-repo/k8s-guardrails -n rapid7 --create-namespace --set K8sGuardrails.ApiToken=token --set Config.ClusterName="Cluster name" --set CronSchedule="30 * * * *" --set Config.BaseUrl=https://self-sign-cert-ics.com --set Config.ClusterId="cluster-id" --set devopscurlSpec.SelfSignedCert.Enabled=true --set devopscurlSpec.SelfSignedCert.CertPem=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUIrVENDQVorZ0F3SUJBZ0lSQUpMYjRhZUxHNVR3VHZPbFVnMVdGaVF3Q2dZSUtvWkl6ajBFQXdJd1FURUwKTUFrR0ExVUVDaE1DY2pjeE1qQXdCZ05WQkFNVEtYTmxiR1l0YzJsbmJpMWpaWEowTFhOaGJtUmliM2d1WkdWMgpMbU56Y0M1eVlYQnBaRGN1WTI5dE1CNFhEVEl4TVRFeU9URXpOVFF5T0ZvWERUSXlNREl5TnpFek5UUXlPRm93ClFURUxNQWtHQTFVRUNoTUNjamN4TWpBd0JnTlZCQU1US1hObGJHWXRjMmxuYmkxalpYSjBMWE5oYm1SaWIzZ3UKWkdWMkxtTnpjQzV5WVhCcFpEY3VZMjl0TUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFNXVzaQp5MnJJMlI3cXpwTlZrTm0zNTZsTDBHMGdKSEtVcGp1Zkl1U1JiT2Q3eldHYnBwa2R2bnVlS2pUNFlEWkhDT2xICllQSHZ1MUpqMEZTMkwyZWxMcU40TUhZd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCL3dRRk1BTUIKQWY4d0hRWURWUjBPQkJZRUZCOUppUExWeUZxV3RiYTc1RkFyRlFSeHFoUUtNRFFHQTFVZEVRUXRNQ3VDS1hObApiR1l0YzJsbmJpMWpaWEowTFhOaGJtUmliM2d1WkdWMkxtTnpjQzV5WVhCcFpEY3VZMjl0TUFvR0NDcUdTTTQ5CkJBTUNBMGdBTUVVQ0lRRFIyQ1NLd1MvaWxlaWoybGdsdFRqQ1NYN2NMQ0tJaTNyV1pTdEdTbmcrL0FJZ0lYeTYKcjVIRCtvZDZhbWNVOEt3UTJRdCttTVNRUER0L3hMT2w5TnhjM3ljPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t

Notes for Helm Install Commands

4. To verify that k8s-guardrails works successfully, you will need to trigger a job manually, using the following command.

kubectl create job --from=cronjob/k8s-guardrails -n rapid7 k8s-guardrails-manual-001

5. Verify that the pod is in the completed status.

• Time to completion status will depend on the size of the cluster.

kubectl get pods -nrapid7 | grep k8s-guardrails-manual-001

6. Verify that the cluster is marked as “monitored” and that resources appear with findings on them.

Discovery for Existing Clusters

To identify clusters that are not currently covered you will need to refer to the following steps. Below are instructions to identify clusters using the InsightCloudSec UI or the InsightCloudSec API.

Discovery of Clusters Using the UI

1. From your InsightCloudSec platform installation, navigate to "Resource --> Resources" and select the "Containers" tab.

2. (Optionally) Use the "Scopes" button at the top of the page to narrow the scope (e.g., cloud accounts, resource groups) to use when scanning for clusters that are not yet included in your InsightCloudSec setup.

3. From the Containers tab, select “Clusters” to see a list of all of the clusters included in the selected scope.

4. Navigate to “Filters” and search for/select the “Kubernetes Cluster Without Guardrails Report”.

• Selecting this filter will update the display to include clusters that have not been scanned.
• The cluster ID field that displays will be used when deploying Guardrails to a specific cluster.

5. Locate the "Cluster ID" column and note the Cluster you want to deploy Guardrails in.

• Note: You will have to scroll to the right to see all of the columns

📘

Cluster ID

InsightCloudSec uses the Cluster ID to identify clusters. Using the Cluster ID allows us to correlate between clusters discovered via the InsightCloudSec platform (either through the UI or API) vs. clusters onboarded through the Alcide scanning capability.

Discovery of Clusters Using the API

For information on using the InsightCloudSec API refer to the Getting Started.

1. Login to the InsightCloudSec (DivvyCloud) API using your username and password in the request body in a POST to v2/public/user/login.

2. Use the session_id from the response in the X-Auth-Token header. Use the following request body in a POST to v2/public/resource/query:

{
    "selected_resource_type": "containercluster",
    "filters": [{
        "name": "divvy.query.kubernetes_cluster_without_guardrails_report",
    "config": {}
    }],
    "offset": 0,
    "limit": 100
}

3. The resources list will display clusters that have not been scanned; the ARN field will be used when deploying Guardrails to a specific cluster.

• Save the ARN details for clusters where you want to configure Kubernetes Security Guardrails.

Using Kubernetes Security Guardrails

These steps assume that you have enabled Kubernetes Security Guardrails feature via Helm.

Refer to the instructions below to identify and use Insights that apply to the findings generated for your Kubernetes cluster by Guardrails

• Refer to our Insights documentation for detailed information on this feature.
• You can also find detailed instructions on Creating a Custom Pack
• If you have questions or need assistance reach out to us through the Customer Support Portal.

View Insights Related to Kubernetes Security Guardrails

1. From InsightCloudSec navigate to "Security → Insights".

2. Select the logo from the list of cloud providers.

• Note: If you have existing Kubernetes clusters there may be two logos. The one with the dot to the top right is for Insights that support Kubernetes Security Guardrails.

3. Only Insights related to the Kubernetes cluster security will display.

• Further filtering using the Scopes function will allow you to narrow your search to identify issues with specific clusters or namespaces.
  • Note that namespaces will be available as a "type" of Resource Group, and as such will appear within the navigation panel that includes Resource Groups.

Create a Custom Pack for Kubernetes Security Guardrails

Many of these Insights are included in the “Center for Internet Security (CIS) - Kubernetes (1.6.0)” Compliance Packs. However, it is also possible to create Custom Packs.

1. Navigate to "Security → Insights".

2. Select "Custom Packs" and click the “CREATE PACK” button.

• Provide a name, description, and complete any other options you desire.

3. Click “Submit”.

4. From "Security --> Insights", select Library

• Select the logo from the list of cloud logos
• Select the Insights you wish to add to your custom pack with the checkbox icon

5. Click the “ACTIONS” button and choose “Add to pack” from the menu

6. Locate your custom pack and select to add the selected Insights.

Viewing K8s Details in the Compliance Scorecard

In addition to viewing information about your clusters and namespaces through Insights, you can also view this information in the Compliance Scorecard. Simply select Cluster/Namespace (this is a filter mode that is specific to the Kubernetes Security Guardrails feature) or Cloud/Cluster (to include *all Clusters - those previously added and anything new) and filter as desired.

*Important Note: The Cluster/Namespace filter mode will only display namespaces within the Cluster selected, if you select a single Cluster.*

• Take a look at our Compliance Scorecard documentation for complete details on this feature.