Kubernetes Remote Scanner

InsightCloudSec’s new Kubernetes Remote Scanner expands our existing Kubernetes Security Guardrails and Container Vulnerability Assessment capabilities by improving InsightCloudSec’s agentless approach for better usability and simplified operation of harvesting Kubernetes entities that exist within different Kubernetes clusters running across different cloud accounts. This solution currently only works with managed clusters.

Feature Overview

• InsightCloudSec harvests the Kubernetes services via the cloud provider API and creates a matching cloud account for each Kubernetes cluster.
• Cluster access is generated using the account access credentials provided by the user.
• Cluster resources are harvested and associated with the parent cloud account that is created to model the Kubernetes service.
• InsightCloudSec automatically performs a scan, repeating scans periodically at a defined interval.
• InsightCloudSec automatically synchronizes the badges associated with the cloud account that contains the Kubernetes service and the tags associated with the Kubernetes cluster into the cloud account that represents the cluster. This allows the user to filter the Kubernetes resources and insights using these badges and tags.
• To migrate between scanners, refer to the instructions on the Clusters Account Setup & Management page.

Prerequisites & Deployment

The Kubernetes Remote Scanner requires the appropriate configuration and permissions for your desired Cloud Service Provider (CSP) accounts (e.g., EKS, AKS) in order to harvest and populate cloud accounts built on the data associated with your clusters. Before getting started with the Kubernetes Remote Scanner you will need to ensure you have the following:

• An existing InsightCloudSec installation (version 22.10.26 or later)
• Appropriate permissions for your desired Cloud Service Providers in order to harvest the resource information specific to each provider
• An existing Kubernetes cluster with network access available to InsightCloudSec
  • Ports 443 and 6443 are used for communications from InsightCloudSec to Kubernetes clusters
  • In general, we recommend that Network access be provided through a dedicated access list for InsightCloudSec

Configuring Permissions

The following permissions must be provisioned by the user to enable the Kubernetes Remote Scanner.

AWS

The following permissions are required for InsightCloudSec to access the Elastic Kubernetes Service (EKS):

 
1
"eks:DescribeCluster",
2
"eks:ListClusters"

Azure

The following roles are required for InsightCloudSec to access the Azure Kubernetes Service (AKS):

• Azure Kubernetes Service Cluster User Role
• Azure Kubernetes Service RBAC Reader

The following write permission is also required to create subjectaccessreviews:

• Microsoft.Kubernetes/connectedClusters/authorization.k8s.io/subjectaccessreviews

GCP

The following permissions/roles are needed for a service account in order to scan a Kubernetes cluster in the Google Kubernetes Engine (GKE):

• Kubernetes Engine Cluster Viewer (roles/container.clusterViewer)
  • Provides access to get and list GKE clusters and get cluster credentials.
  • Note: This role is a subset of the Kubernetes Engine Viewer role (roles/container.viewer)
  • Provides permissions: container.clusters.get, container.clusters.list, resourcemanager.projects.get, resourcemanager.projects.list
• Kubernetes Engine Viewer (roles/container.viewer)
  • Provides read-only access to resources within GKE clusters, such as nodes, pods, and GKE API objects
  • Review the GCP documentation for more information about the permissions this role provides
• Permission needed for secrets visibility and validation: container.secrets.list
• Permission needed for subjectaccessreview creation: container.subjectAccessReviews.create

Configuring Kubernetes Access

Currently, InsightCloudSec offers support for the following CSPs and their Kubernetes service equivalents:

• AWS (Elastic Kubernetes Service(EKS))
• Azure (Azure Kubernetes Service (AKS))
• GCP (Google Kubernetes Engine (GKE))

AWS (EKS)

To enable access for EKS you will need to either:

• Update your existing AWS IAM Role/Read Access Permissions (InsightCloudSec Harvester role) to include EKS

OR

• Configure a new (dedicated, separate) AWS role for InsightCloudSec that provides read access to EKS

The majority of customers will use their existing AWS IAM Role for InsightCloudSec (the "harvesting" role) and update the configuration to provide read access to EKS clusters resources. To provide the role access to the Kubernetes clusters you will need to configure the role ARN in the cloud account and allow InsightCloudSec the permission to assume this role. Otherwise, you will create a new role and provide the harvester role with read access permissions to the EKS cluster.

For both options the role needs to be configured in the aws-auth ConfigMap in each EKS cluster. The specific permissions are listed above.

Update Existing Role

Provisioning access permission for reading the Kubernetes cluster resources from EKS requires the following steps:

• Define the access permissions in the EKS cluster RBAC:

  • Create a ClusterRole using the provided YAML file that allows read access to all Kubernetes resources
  • Create a ClusterRoleBinding to a Kubernetes RBAC group; this will bind the role to all service accounts that are under the provided group

• MAP an IAM role using eksctl or by directly editing the ConfigMap. Refer to the AWS Docs for details

1. Open a command shell within the desired directory and run the following command to connect to your Kubernetes cluster in the command line, ensuring you replace the placeholder values appropriately: aws eks --region <region-name> update-kubeconfig --name <cluster-name>.
   Review the AWS EKS Documentation for more information.

2. Copy the following example file to a local file to the desired directory and save it as ClusterRole-Create.yaml:

   yaml
   1
   apiVersion: rbac.authorization.k8s.io/v1
   2
   kind: ClusterRole
   3
   metadata:
   4
     name: cluster-reader-role
   5
   rules:
   6
     - apiGroups:
   7
         - '*'
   8
       resources:
   9
         - '*'
   10
       verbs:
   11
         - get
   12
         - list
   13
     - apiGroups:
   14
         - authorization.k8s.io
   15
       resources:
   16
         - subjectaccessreviews
   17
       verbs:
   18
         - create

3. Within the desired directory, run this command to create the cluster role in Kubernetes: kubectl apply -f ClusterRole-Create.yaml

4. In the same directory, copy the following example file to a local file and save it as ClusterRoleBinding-Apply.yaml:

   yaml
   1
   apiVersion: rbac.authorization.k8s.io/v1
   2
   kind: ClusterRoleBinding
   3
   metadata:
   4
     name: cluster-reader-binding
   5
   roleRef:
   6
     apiGroup: rbac.authorization.k8s.io
   7
     kind: ClusterRole
   8
     name: cluster-reader-role
   9
   subjects:
   10
     - kind: Group
   11
       name: cluster-reader-group
   12
       apiGroup: rbac.authorization.k8s.io

5. In the same command shell, run this command to create the cluster role in Kubernetes: kubectl apply -f ClusterRoleBinding-Apply.yaml

The final step is to map the existing IAM role to the cluster-reader-group by configuring aws-auth ConfigMap. There are two ways to perform this configuration:

• Using eksctl, an open source tool that simplifies the configuration of EKS.
• Directly editing the ConfigMap

Using eksctl

1. If you haven't already, download and install ekstctl.
2. Use the following command to perform the configuration:

shell
1
eksctl create iamidentitymapping \
2
    --cluster my-cluster \
3
    --region=region-code \
4
    --arn arn:aws:iam::111122223333:role/rapid7-readonly \
5
    --group cluster-reader-group \
6
    --no-duplicate-arns

Directly edit the ConfigMap

1. Run the following command to open the ConfigMap for editing: kubectl edit -n kube-system configmap/aws-auth.
   This will open vim by default; if you prefer to use a different editor, like nano, use the following command instead: KUBE_EDITOR="nano" kubectl edit -n kube-system configmap/aws-auth
2. Adjust the maproles section to look something like the example below, ensuring you replace the rolearn with the harvesting role mentioned above:

yaml
1
apiVersion: v1
2
data:
3
  mapRoles: |
4
    - groups:
5
      - cluster-reader-group
6
      rolearn: arn:aws:iam::111122223333:role/rapid7-readonly
7
      username: my-role

(Optional) Configure a New Role

The following steps outline the details for the configuration of a separate role that provides access to EKS.

Note that the completion of these steps only create the new separate role and you will still need to grant access through the process outlined above.

1. From InsightCloudSec navigate to Cloud>Clouds and select your target account from the Clouds Listing tab.
2. Select the Settings tab and click the Update EKS Scanner Role button.
3. Select or create your new separate role as desired and complete the form with the required account details.
4. Click OK to finalize your changes.
5. Refer to the steps outlined in the Provide an existing IAM Role With Read Access Permissions to EKS section and complete them as outlined for your new role.

Once you complete the steps to provide Read Access Permissions to EKS, Kubernetes access will be configured and the cluster will begin harvesting (assuming it is not in a paused state within InsightCloudSec).

Azure (AKS)

To enable access for AKS you will need to update your existing InsightCloudSec Azure Application (from Azure - Onboarding) to include the permissions/roles mentioned in Configuring Permissions.

1. Login as an Admin to the Azure Dashboard for the account that hosts your InsightCloudSec Application.

2. Open the Subscription Access Control Dashboard.

   • Navigate to the Subscriptions page.
   • From the desired subscription's menu panel on the left, select Access control (IAM).

3. Attribute the Azure Kubernetes Service RBAC Reader role to the existing InsightCloudSec-associated Application registration.

   • From the Access control (IAM) panel, click Add > Add role assignment.
   • Search for Azure Kubernetes Service.
   • Select the Azure Kubernetes Service RBAC Reader role.
   • Click Next.
   • Leave the Assign access to field as the default value (User, group, or service principal).
   • Next to Members, click + Select members.
   • In the Select panel, begin typing the name of the InsightCloudSec Azure Application. Select that application once it appears, then click Select.
   • Click Review + assign to add the role.

4. Repeat the previous steps for the other required role.

   • From the Access control (IAM) panel, click Add>Add role assignment.
   • Search for Azure Kubernetes Service.
   • Select the Azure Kubernetes Service Cluster User Role role.
   • Click Next.
   • Leave the Assign access to field as the default value (User, group, or service principal).
   • Next to Members, click + Select members.
   • In the Select panel, begin typing the name of the InsightCloudSec Azure Application. Select that application once it appears, then click Select.
   • Click Review + assign to add the role.

Once you complete the steps to provide Read Access Permissions to AKS, Kubernetes access will be configured and the cluster will begin harvesting (assuming it is not in a paused state within InsightCloudSec).

GCP (GKE)

To enable access for GKE you will need to update your existing InsightCloudSec GCP Project (from GCP - Onboarding) to include the permissions/roles mentioned in Configuring Permissions.

1. Within your GCP console navigate into the project that is associated with InsightCloudSec.
2. Navigate to IAM & Admin>Roles.
3. Edit the role associated with InsightCloudSec.
   • Find the role and click the vertical three dots to expose the menu.
   • Click Edit.
   • Click + ADD PERMISSIONS.
   • Filter for the "container.secrets.list" permission and select the checkbox next to it.
   • Click ADD.

Once you complete the steps to provide Read Access Permissions to GKE, Kubernetes access will be configured and the cluster will begin harvesting (assuming it is not in a paused state within InsightCloudSec).

What's Next?

Refer to the Kubernetes Security Guardrails Overview page for an overview of this feature and a summary of the prerequisites.

Jump to the Using Kubernetes Security Guardrails page to view details on using the feature and exploring the imported data.