Increase Scheduler Task Memory on ECS

Instructions for Increasing Scheduler Task Memory on ECS-based Deployments



The content on this page is deprecated per our Production Services team. If you have questions regarding this content reach out to the Docs team through Slack.


Value Names (DivvyCloud vs. InsightCloudSec)

Some components use our former product name (DivvyCloud vs. InsightCloudSec). Updates to the naming of these components will be communicated when changes are made, but note that the name difference does not affect functionality within the product.

The following documentation provides instructions on how to increase the memory available to the InsightCloudSec scheduler in ECS. The recommended amount of memory for the scheduler is 1GB. Older deployments may only have 512MB allocated to the scheduler.

Note: The content/steps provided on this page applies to self-hosted customers. For hosted customers we recommend that you contact your CSM or through the Customer Support Portal with any questions or concerns.

Updating via Terraform

1. Locate the "aws_ecs_task_definition" named "scheduler" in your Terraform configuration files. Update the value for "memory" to 1024 and apply.

  • An example is provided below, on line 51.
resource "aws_ecs_task_definition" "scheduler" {
    depends_on = [aws_ecs_cluster.DivvyCloud-ECS-Cluster]
    container_definitions    = jsonencode(
                command                = [
                cpu                    = 0
                dnsSearchDomains       = []
                dnsServers             = []
                dockerLabels           = {}
                dockerSecurityOptions  = []
                entryPoint             = []
                environment            = local.environment
                essential              = true
                extraHosts             = []
                image                  = "${var.divvycloud_version}"
                links                  = []
                linuxParameters        = {
                    capabilities = {}
                    devices      = []
                logConfiguration       = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = var.ecs_log_group_name
                        awslogs-region        = var.region
                        awslogs-stream-prefix = "scheduler"
                mountPoints            = []
                name                   = "scheduler"
                portMappings           = []
                privileged             = false
                pseudoTerminal         = false
                readonlyRootFilesystem = false
                ulimits                = []
                volumesFrom            = []
    cpu                      = "256"
    execution_role_arn       = local.ecs_exec_role
    family                   = "scheduler"
    memory                   = "1024"
    network_mode             = "awsvpc"
    requires_compatibilities = [
    tags                     = {}
    task_role_arn            = local.ecs_task_role

Updating via AWS Cloud Formation Template

1. Locate "schedulerTask" in the Compute CFT. Update the "Memory" value to 1024 and deploy in Cloud Formation.

  • An example is provided below, on line 10.
    Type: 'AWS::ECS::TaskDefinition'
      Family: 'scheduler'
        Fn::ImportValue: !Sub '${ParameterIAMStackName}-DivvyCloudStandardRoleARN'   
        Fn::ImportValue: !Sub '${ParameterIAMStackName}-AmazonECSTaskExecutionRoleARN'
      Cpu: '256'
      Memory: '1024'
      NetworkMode: awsvpc
        - FARGATE
        # scheduler
        - Image: !Ref ParameterDivvyCloudVersion
          Name: scheduler
            - Name: DIVVY_REDIS_SSL_ENABLED
              Value: true
            - Name: VIRTUAL_ENV
              Value: /
            - Name: DIVVY_ENV
              Value: prod
            - Name: DIVVY_DB_NAME
              Value: !FindInMap [Vars, db, name]
            - Name: DIVVY_SECRET_DB_NAME
              Value: !FindInMap [Vars, db, secureName]
              Value: !Join
               - ''
               -  - 'AWSAssumeRole,region='
                  - !Ref AWS::Region
                  - ',secret_name='
                  - Fn::ImportValue: !Sub '${ParameterDataStackName}-DivvyCloudDBSecretName'
            - Name: DIVVY_REDIS_HOST
                Fn::ImportValue: !Sub '${ParameterDataStackName}-DivvyCloudRedis'
            - Name: DIVVY_REDIS_PORT
              Value: !FindInMap [Vars, redis, port]
            LogDriver: awslogs
              awslogs-group: DivvyCloud-ECS
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: scheduler
            - divvycloud
            - scheduler 
            - --db-upgrade
            - --enqueue-immediately
            - --distribution-window
            - '1440'

Updating via AWS Console

1. In the AWS Console, located ECS and browse to "Task Definitions".

2. Click on your scheduler definition.

3. Select the latest task definition revision.

4. Click "Create new revision".


AWS Console - Amazon ECS - Create New Revision

5. In the new revision change the Task Memory to 1GB, scroll down and save.


Task Memory Update

5b. You can deregister the older revision (optional).


Deregiser Old Task (Optional)

Now you will need to update the cluster service to use this new task definition.

6. Browse to your ECS Cluster.

7. Select the scheduler service and click "Update Cluster".


Update Cluster with New Task Definition

8. Change the revision to the new version you created earlier, scroll down and click through until you save the service changes.


Update Service

9. Once you have saved, check the service for the tasks to restart. You should see any [INACTIVE] tasks shut down and the new tasks running after a few minutes.


Verify Updated Tasks

Did this page help you?