Increase Scheduler Task Memory on ECS

Instructions for Increasing Scheduler Task Memory on ECS-based Deployments

🚧

INTERNAL NOTE

The content on this page is deprecated per our Production Services team. If you have questions regarding this content reach out to the Docs team through Slack.

📘

Value Names (DivvyCloud vs. InsightCloudSec)

Some components use our former product name (DivvyCloud vs. InsightCloudSec). Updates to the naming of these components will be communicated when changes are made, but note that the name difference does not affect functionality within the product.

The following documentation provides instructions on how to increase the memory available to the InsightCloudSec scheduler in ECS. The recommended amount of memory for the scheduler is 1GB. Older deployments may only have 512MB allocated to the scheduler.

Note: The content/steps provided on this page applies to self-hosted customers. For hosted customers we recommend that you contact your CSM or through the Customer Support Portal with any questions or concerns.

Updating via Terraform

1. Locate the "aws_ecs_task_definition" named "scheduler" in your Terraform configuration files. Update the value for "memory" to 1024 and apply.

  • An example is provided below, on line 51.
resource "aws_ecs_task_definition" "scheduler" {
    depends_on = [aws_ecs_cluster.DivvyCloud-ECS-Cluster]
    container_definitions    = jsonencode(
        [
            {
                command                = [
                    "divvycloud",
                    "scheduler",
                    "--db-upgrade",
                    "--enqueue-immediately",
                    "--distribution-window",
                    "1440"
                ]
                cpu                    = 0
                dnsSearchDomains       = []
                dnsServers             = []
                dockerLabels           = {}
                dockerSecurityOptions  = []
                entryPoint             = []
                environment            = local.environment
                essential              = true
                extraHosts             = []
                image                  = "${var.divvycloud_version}"
                links                  = []
                linuxParameters        = {
                    capabilities = {}
                    devices      = []
                }
                logConfiguration       = {
                    logDriver = "awslogs"
                    options   = {
                        awslogs-group         = var.ecs_log_group_name
                        awslogs-region        = var.region
                        awslogs-stream-prefix = "scheduler"
                    }
                }
                mountPoints            = []
                name                   = "scheduler"
                portMappings           = []
                privileged             = false
                pseudoTerminal         = false
                readonlyRootFilesystem = false
                ulimits                = []
                volumesFrom            = []
            },
        ]
    )
    cpu                      = "256"
    execution_role_arn       = local.ecs_exec_role
    family                   = "scheduler"
    memory                   = "1024"
    network_mode             = "awsvpc"
    requires_compatibilities = [
        "FARGATE",
    ]
    tags                     = {}
    task_role_arn            = local.ecs_task_role
}

Updating via AWS Cloud Formation Template

1. Locate "schedulerTask" in the Compute CFT. Update the "Memory" value to 1024 and deploy in Cloud Formation.

  • An example is provided below, on line 10.
schedulerTask:
    Type: 'AWS::ECS::TaskDefinition'
    Properties:
      Family: 'scheduler'
      TaskRoleArn: 
        Fn::ImportValue: !Sub '${ParameterIAMStackName}-DivvyCloudStandardRoleARN'   
      ExecutionRoleArn: 
        Fn::ImportValue: !Sub '${ParameterIAMStackName}-AmazonECSTaskExecutionRoleARN'
      Cpu: '256'
      Memory: '1024'
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      ContainerDefinitions:
        # scheduler
        - Image: !Ref ParameterDivvyCloudVersion
          Name: scheduler
          Environment:
            - Name: DIVVY_REDIS_SSL_ENABLED
              Value: true
            - Name: VIRTUAL_ENV
              Value: /
            - Name: DIVVY_ENV
              Value: prod
            - Name: DIVVY_DB_NAME
              Value: !FindInMap [Vars, db, name]
            - Name: DIVVY_SECRET_DB_NAME
              Value: !FindInMap [Vars, db, secureName]
            - Name: DIVVY_SECRETS_PROVIDER_CONFIG
              Value: !Join
               - ''
               -  - 'AWSAssumeRole,region='
                  - !Ref AWS::Region
                  - ',secret_name='
                  - Fn::ImportValue: !Sub '${ParameterDataStackName}-DivvyCloudDBSecretName'
            - Name: DIVVY_REDIS_HOST
              Value:
                Fn::ImportValue: !Sub '${ParameterDataStackName}-DivvyCloudRedis'
            - Name: DIVVY_REDIS_PORT
              Value: !FindInMap [Vars, redis, port]
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: DivvyCloud-ECS
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: scheduler
          Command:
            - divvycloud
            - scheduler 
            - --db-upgrade
            - --enqueue-immediately
            - --distribution-window
            - '1440'

Updating via AWS Console

1. In the AWS Console, located ECS and browse to "Task Definitions".

2. Click on your scheduler definition.

3. Select the latest task definition revision.

4. Click "Create new revision".

16041604

AWS Console - Amazon ECS - Create New Revision

5. In the new revision change the Task Memory to 1GB, scroll down and save.

16041604

Task Memory Update

5b. You can deregister the older revision (optional).

16041604

Deregiser Old Task (Optional)

Now you will need to update the cluster service to use this new task definition.

6. Browse to your ECS Cluster.

7. Select the scheduler service and click "Update Cluster".

16041604

Update Cluster with New Task Definition

8. Change the revision to the new version you created earlier, scroll down and click through until you save the service changes.

13421342

Update Service

9. Once you have saved, check the service for the tasks to restart. You should see any [INACTIVE] tasks shut down and the new tasks running after a few minutes.

16041604

Verify Updated Tasks


Did this page help you?