- Introduced fallback logic for AWS S3 tags, if we are unable to obtain the tags due to provider API issues
- DV2019-301 Improved error handling for empty/missing snapshot listings on GCP
- DV2019-322 Fixed false positives on hookpoint matches from index
- Associate newly added AWS, Azure and GCP accounts with the default strategy for that cloud if a strategy ID is not supplied
- DV2019-309 Converted AWS CloudFront harvesting to a global call to reduce the number of API calls made to the service
- DV2019-313 Properly store AWS routes pointing to NAT/Transit gateways
- DV2019-294 Added the ability to skip previously identified resources when posting resource data to a remote URL
- Affix a permanent ResourceProperty
divvy.creatorwhich stores the user who provisioned AWS resources and can be used to enrich resource tagging. This only works for AWS customers who leverage EDH
- DV2019-289 GCP Organization harvesting now no longer filters projects by parent ID which allows support for iterating Folders
- Don't throw hookpoints on SNS and SQS policy documents which have not changed
- DV2019-287 Leverage the OrphanedResourceCleanup processor to more efficiently removal resources/cloud accounts pending deletion
- DV2019-284 Support additional DNS record types (NAPTR/CAA) within AWS Route53
- DV2019-257 Properly store the OssBucketName that's associated with Alibaba Cloud API accounting resources
- DV2019-248 Fixed a harvesting issue that prevented GCP Database Snapshots from harvesting for multi-node clusters
- Improved equality checking for the IAM assume role policy document
- DV2019-244 Added Request Timeout For Kubernetes
- DV2019-167 Added the ability to sort Database Clusters by the lifecycle state
- DV2019-243 Restored the ability to create Bots that run against DNS Zone resources
- DV2019-221 Do not harvest AWS Organization information from AWS GovCloud or AWS China as it's not supported by the provider
- DV2019-222 Added Namespace Search Filter for Kubernetes
- DV2019-218 Do not include password in email for external auth users
- DV2019-195 ECS Commands Not Showing
- DV2019-266 Fixed an issue that prevented Bots from updating S3 IAM bucket policies
- DV2019-266 Rephrased the ID field and made it a string input, not textarea
- DIVVY-4488 Added NSX LoadBalancer Support
- Added the ability for domain administrators to reset job scheduler stats within System Settings
- Restored Instance CPU and Database connection harvesting in AWS
- Fixed an edge case that prevented harvesting of Azure storage accountsd
- Fixed an issue where missing images/job names were displaying in the Slowest Jobs panel
- Restored multiprocessing on select resource types where it was enabled in 18.6
- Added new Google Cloud Platform Filters to identify resources running in legacy/default networks
- DV2019-204 Improved the tagging action, "Assign Tag to Resource" to prevent a possible loop from occurring
- DV2019-194 Fixed an issue with Cloud Service Cost jobs failing to start
- DV2019-191 Added Queue name to worker logs
- DV2019-192: Restored the ability to create new custom Insights by setting the proper default value for badge scopes
- Changed the cadence of the license harvester to run every 6 hours instead of every hour
- Fixed an issue that resulted in GCP clouds being put into a paused state
- Fixed an issue that would prevent scheduled events from firing under certain conditions
- DV2019-178 - Fixed an edge case that prevented tag explorer configurations from being saved
- DV2019-179 - Added wildcard and exclusion capabilities to the the API Key By User Name filter
- DV2019-158 - Defaulted to newly created Insight to leverage OR vs AND when multiple badges are associated with an Insight
- DV2019-168 - Removed duplicate deletion protection column from the Database Clusters view
- DV2019-167 - Fixed DatabaseCluster region sorting
- DV2019-172 - AWS EFS visibility into eu-west-2 (London), Workspace visibility into AWS GovCloud us-west-1, and Kinesis Firehose visibility into AWS China (cn-north-1)
- DV2019-159 - Immediately enqueued jobs for a newly added cloud
- DIVVY-4557 - Expanded the Resource Recently Modified filter to support all resource types
- DV2019-162- Fixed an incorrect version formatting of backoffice api url
- DV2019-175 - Fixed an issue were we did not actually remove dead keys from sets
Before upgrading your DivvyCloud installation, please take a moment to view our
Administrator and Developer Notes at the bottom of this page.
Setup Harvest Strategy after you Upgrade to 18.7
It's very important that you set up your harvesting strategies after you upgrade to 18.7. More information on configuring Harvesting Strategies can be found here.
Contact email@example.com for questions.
18.7 introduces a new scheduler with major performance enhancements on the back end. The scheduler is now much faster and more stable, especially at scale. Start up time is a fraction of what it was prior to 18.7; any refresh or loading times via either the UI or API are also reduced. Not only is the scheduler faster, it also allows for a very low CPU footprint. After deploying 18.7, customers can check hardware to either downsize extra large instances, or even eliminate some worker nodes.
Accessible by clicking on an account card in the Cloud Compliance view, the Pack Compliance page displays: a compliance time-series graph, a listing of compliance per every rule in the pack, and a breakdown of noncompliance by resource type.
A new tab, 'Organizations', has been introduced to the Clouds page. Within the page, a cloud organization resource can be added to the system to enable automatic addition of all cloud accounts under the organization into DivvyCloud system. Only Google Cloud Platform organizations will be supported in this release. We will be expanding this feature to AWS Organizations in early 2019.
Harvesting strategies and overrides are new items introduced in this release which will help customize the cadence of resource harvesters at a granular level. There will be a default strategy for AWS, Google, and Azure, and any existing cloud accounts will be assigned the respective default strategy. Create new strategies or configure overrides for existing strategies for the suitable harvesting needs. Admins can access the Harvesting Strategy page via the main navigation panel. More information on configuring Harvesting Strategies can be found here.
There's now a new domain user classification, Domain Viewer, which allows a user to be given full read-only access to the entire installation; however, users of this type cannot take any lifecycle operations on cloud resources, create Insights, Bots, or any other administrative function within the tool. This feature is especially useful for customers running multiple organizations.
Amazon Web Services
* Support for Container Registries and Images * Support for Account Level S3 Bucket Access Controls * Support for EC2 instance hibernation * Support for new A1 and C5n instance types * Event driven harvesting (EDH) support for VPC Flow Logs, Dedicated Hosts, Network Peers, Memcache and Elasticsearch Instances, and RDS Aurora clusters. * Enhanced visibility and lifecycle support for RDS Aurora Clusters * Storage of the resource ARN across all resource types * Support for AWS Cloudwatch Logs * Added support for us-gov-east-1 region * Add visibility into whether or not an instance is a spot instance * Support for SageMaker Notebooks * Add support for tags for IAM users/roles * Add ability to suspend and resume processes for Autoscaling Groups * Support for the new Stockholm region (eu-north-1)
Google Cloud Platform
* Support for walking a GCP Organization and auto-adding all projects * Support for Autoscaling Groups * Support for Cloud DNS * Add the ability to start/stop SQL databases
* Support for Container Registries * Support for harvesting of IAM Service Principals with role assignments * Support for harvesting of App Service Plans and Apps * Support for harvesting Azure Functions
* Support for creating a new VM from an existing instance snapshot
- Job Backlog data can now be exported to CloudWatch, configured via SystemSettings table.
- Event driven harvesting producer configuration can be updated within the tool.
- Added cross account inspection for Service Encryption Keys and Elasticsearch Instances.
- Added pagination support for KMS encryption keys, enabling visibility into regions with over 1,000 keys.
- Split Database Instances and Database Clusters into two separate resource types for more clarity.
- Added encryption type specification support to the Storage Containers With Default Encryption resource filter.
divvy.badge prefix to
system.to add clarity that these are system level badges which cannot be modified/removed.
- Added ability to track the state of Internet Gateways.
The following AWS events have been added:
We released over 100 filters in 18.7 and we wanted to highlight a few that we thought were interesting (see below).
To view a complete list all the new filters in 18.7, please check out the Filters Page in your DivvyCloud account.
Identifies AWS accounts without specified bucket policy option ''enabled’.
Match DNS zones which have DNSSEC (DNS Security) disabled.
Over 40 new filters have been added for Kubernetes Cloud Accounts.
Identify service detectors that do not have any of the supplied accounts as their master accounts. The master account invites other accounts to become associated with it. Those accounts are called member accounts. A master account can view and manage its member accounts.
Match cloud roles with a trust relationship which contains access 'from a wildcard role name.
Identify storage containers with an IAM policy that contain the supplied statement ID.
Identify log groups which do not have encryption enabled.
Identify Serverless Functions which have an HTTP binding that does not require authentication.
Identify instances which are of type Spot Instance.
Identify instances which are not of type Spot Instance.
Identify instances for which detailed monitoring is enabled. In clouds such as Amazon Web Services, detailed monitoring has an additional cost, so this filter can help identify opportunities for cost savings.
Identify instances for which detailed monitoring is disabled. Detailed monitoring can be important for production systems where additional monitoring capabilities are required.
Identify instances which are missing one or more required extensions. The supplied extension(s) must be present on the target instances and provisioned in a successful state.
Identify ML instances in a particular lifecycle state.
Identify ML instances that are directly accessible from the Internet.
Identify ML instances that are not part of a subnet.
Match web apps with the selected remote debugging state.
Match web apps with the selected web sockets state.
Match web apps with the selected FTP configuration state.
Match web apps with the selected HTTPS configuration state.
Identify Serverless Functions which have an HTTP bindings that does not require authentication. This only applies to Microsoft Azure Functions.
Match Internet Gateways which are associated with a parent network
Match NAT Gateways which are in the specified lifecycle state(s).
Suspend automatic scaling processes for an autoscaling group. You can specify certain processes or do all at once.
Resume automatic scaling processes for an autoscaling group. You can specify certain processes or do all at once.
Execute a delete lifecycle command against an AWS RDS Cluster. Note that this lifecycle command will permanently destroy the master write node as well as all read-only replicas.
Execute a start lifecycle command against an AWS RDS Cluster. Note that this lifecycle command will start the master write node as well as all read-only replicas.
Execute a stop lifecycle command against an AWS RDS Cluster. Note that this lifecycle command will stop the master write node as well as all read-only replicas.
Remove all permissions for a particular GCP User from a project. This action can be taken manually or via a Bot.
Remove all permissions for a particular GCP Group from a project. This action can be taken manually or via a Bot.
Remove all permissions for a particular GCP Role (Service Account) from a project. This action can be taken manually or via a Bot.
Enable detailed monitoring for AWS compute instances. When enabled, performance data is available in 1-minute periods for an additional cost. For those instances for which you've enabled detailed monitoring, you can also get aggregated data across groups of similar instances.
Disable detailed monitoring for AWS compute instances. When disabled, performance data is available in 5-minute periods without additional cost. Disable monitoring to save money on those instances which do not require additional granularity in performance data.
- Fixed a visibility bug which was preventing plugins from displaying their load status in the UI (DIVVY-4299)
- Fixed an edge case that prevented sorting within the Tag Explorer (DIVVY-3869)
- Insight CSV download now takes cloud filtering into consideration (DIVVY-3939)
- Delivery Streams no longer trigger modification hookpoints on every single harvest (DIVVY-4464)
For Amazon Web Service customers, the role/user policy associated with each connected account will need to be adjusted to include the permissions below. Without these permissions, visibility into the newly supported AWS services will not be possible.
iam:ListInstanceProfiles sagemaker:ListNotebookInstances sagemaker:DescribeNotebookInstance sagemaker:ListTags s3:GetPublicAccessBlock s3control:GetAccountPublicAccessBlock autoscaling:SuspendProcesses autoscaling:ResumeProcesses
For those interested in modifying tags for AWS IAM users and roles, the
following permissions will need to be included in your policy.
iam:TagUser iam:TagRole iam:UntagUser iam:UntagRole ecr:DescribeRepositories ecr:DescribeImages
For Microsoft Azure customers using custom roles, the role definitions associated with each connected account will need to be adjusted to include the permissions below. Without
these permissions, visibility into newly supported Microsoft Azure services will not be possible.
// For the read-only Standard User role Microsoft.ContainerRegistry/registries/read Microsoft.ContainerService/managedClusters/read Microsoft.Storage/storageAccounts/read Microsoft.Web/serverfarms/read Microsoft.Web/sites/*/read Microsoft.Web/sites/config/list/Action // For the Power User role Microsoft.ContainerRegistry/* Microsoft.ContainerService/* Microsoft.Storage/* Microsoft.Web/*
DivvyCloud v18.7 includes support for Google Analytics in order for us to improve navigation and the overall customer experience.