Mateusz Biel's Blog

Say "NO" to Beanstalk! Use CodeDeploy instead pt.2

Photo by Markus Spiske on Unsplash

Say "NO" to Beanstalk! Use CodeDeploy instead pt.2

Mateusz Biel's photo
Mateusz Biel
·

17 min read

Introduction

AWS CodeDeploy is a service for automating application deployment, it's possible to use this service to deploy Lambda functions, ECS Services or EC2 Instances, here I describe only the last option.

CodeDeploy is responsible only for uploading the application bundle and running scripts listed in appspec.yml, all resources, environment variables, logging tools and process management need to be provisioned separately.

This approach offers a great flexibility but the whole process becomes complex. Because of that this deployment option may be considered only if other ones don't provide enough features.

I created a sample Terraform module that tries to replicate AWS Beanstalk features, it's available HERE, feel free to use it.

I describe here resources that Beanstalk provisions and all of those should be created if you want to imitate this setup.

Beanstalk Resources

Beanstalk Environment is a collection of AWS resources:

If the property LoadBalancerType from aws:elasticbeanstalk:environment namespace is set to classic Beanstalk creates following resource:

AWS::ElasticLoadBalancing::LoadBalancer (aws_elb)

for application and network values of LoadBalancerType the Environment contains such resources:

For security reasons Beanstalk creates two Security Groups: one for a load balancer and one for EC2 instances.

Configuration of these security groups:

Security GroupIngressEgress
Load BalancerTCP port 80 src: 0.0.0.0/0TCP port 80 dest: 0.0.0.0/0
EC2 InstancesTCP port 80 src: Load Balancer SGAll traffic dest: 0.0.0.0/0

So the Security Group for EC2 Instances allows inbound traffic only from the Load Balancer Security Group, it's a good practice that everyone should follow. Load Balancer Egress could have also been restricted to the proper VPC CIDR block. In Beanstalk Environment the Load Balancer proxies the traffic to the instance listening on port 80, in the presented solution nginx can use any port higher than 1024 so the security group can be parametrized based on port that nginx listens on.

Also, it's worth to mention that AWS recommends to use AWS::EC2::LaunchTemplate instead of AWS::AutoScaling::LaunchConfiguration, because it provides the newest EC2 features, Launch Templates can also be versioned, there is no need to recreate the resource when update is required, on the other hand Launch Configurations need to be recreated every property update.

My AutoScaling Group has health_check_type set to ELB so every instance that will not pass the Load Balancer health check will be terminated, the Load Balancer will monitor only instances that run some application version, I describe it further. For the heath check target I use HTTP protocol and the same port that nginx uses (for me, it's 8080).

AutoScaling Group Instances Refresh

Even though aws_autoscaling_group uses the latest version of the aws_launch_template:

  launch_template {
    id = aws_launch_template.app-launch-template.id
    version = aws_launch_template.app-launch-template.latest_version
  }

every update of the template WILL NOT affect running EC2 instances, AutoScaling Group applies it only to new instances, it will not replace the existing ones. For instances replacement AWS introduced instance refresh, instances can be refreshed using StartInstanceRefresh (start_instance_refresh) action, the replacement process can be monitored using AWS Console or with DescribeInstanceRefreshes (describe_instance_refreshes) action.

Terraform is able to trigger instance refresh when the launch template has been changed, but it WILL NOT wait until the process is finished. When the infrastructure update is finished the deployment pipeline should detect if there is any pending instance refresh and wait until it's finished before starting the deployment.

Instance refresh configuration of an aws_autoscaling_group resource:

  instance_refresh {
    strategy = "Rolling"
    preferences {
      min_healthy_percentage = 99
    }
  }

Instances can be replaced using Rolling option, so there will be no downtime. When the process is triggered by Terraform you can wait it to finish by using AWS SDK, describe_instance_refreshes can return the status for every instance being replaced.

AutoScaling Group Load Balancer

It's important to not attach a Load Balancer to an AutoScaling Group using load_balancers or target_group_arns properties of aws_autoscaling_group, because the Load Balancer should monitor the instances AFTER the first deployment, otherwise it would terminate instances that don't have any version deployed yet. The Load Balancer is configured for an aws_codedeploy_deployment_group, and it will be attached after the successful deployment.

Components

In this section I present some important parts of my environment, including AMI, logging tools, EC2 security configuration, environment variables provisioning and other tools that are required to create a solution stack similar to one created by Beanstalk.

Environment AMI

The image can be provisioned by Packer and should consist of all tools required by a specific environment (JRE for Java, Python interpreter, some native libraries) together with nginx that will be responsible for buffering incoming connections. You should provision every tool that your chosen language stack requires. It's important to configure and install as many tools as possible in this state, because it will speed up the scale-up process, the newly provisioned instances during high-load will need to download only the application bundle.

An example install script for a Python3 environment can be found HERE.

Of course the image should provide CodeDeploy Agent that is responsible for unpacking the application bundle and perform deployment actions described in appspec.yml file.

AWS provides different S3 buckets with CodeDeploy binary for every region so the current region can be fetched during AMI provisioning process:

TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
INSTANCE_IDENTITY=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/dynamic/instance-identity/document)
REGION=$(jq -r '.region' <<< "$INSTANCE_IDENTITY")
CODE_DEPLOY_URL="https://aws-codedeploy-$REGION.s3.$REGION.amazonaws.com/latest/install"
wget "$CODE_DEPLOY_URL"

IAM and Security Groups

EC2 Security Group needs to allow HTTPS (tcp port 443) outbound traffic to let the CodeDeploy Agent download the application bundle from S3. There is no need for any additional inbound traffic rule (except the one for Load Balancer of course).

Security Group for EC2 Instances:

resource "aws_security_group" "instance-security-group" {
  vpc_id = var.vpc_id
  ingress {
    security_groups = [aws_security_group.lb-security-group.id]
    from_port = var.nginx-port
    to_port = var.nginx-port
    protocol = "tcp"
  }
  egress {
    cidr_blocks = ["0.0.0.0/0"]
    from_port = 443
    to_port = 443
    protocol = "tcp"
  }
}

The IAM Role Policy for the instance is:

data "aws_iam_policy_document" "ec2-policy" {
  version = "2012-10-17"
  statement {
    sid = "AllowDownloadBundle"
    effect = "Allow"
    actions = [
      "s3:List*",
      "s3:Get*"
    ]
    resources = ["arn:aws:s3:::${local.app-name}-artifacts-${local.region}-${local.account_id}/*"]
  }
  statement {
    sid = "AllowCloudWatchLogs"
    effect = "Allow"
    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents",
      "logs:TagLogGroup",
      "logs:DescribeLogGroups",
      "logs:DescribeLogStreams"
    ]
    resources = [
      "arn:aws:logs:${local.region}:${local.account_id}:log-group:/${local.app-name}/*",
      "arn:aws:logs:${local.region}:${local.account_id}:log-group:/${local.app-name}/*:log-stream:*"
    ]
  }
}

s3:List* and s3:Get* are listed HERE as required. These actions are restricted only to the S3 bucket with application artifacts.

appspec.yml

appspec.yml is a file required by CodeDeploy Agent to copy proper files and run lifecycle hooks. The file should be provisioned as part of the application bundle. It's the core of the CodeDeploy deployment process as it describes every action that is required to install the application.

Source of the appspec.yml:

version: 0.0
os: linux
files:
  - source: systemd/app.service
    destination: /etc/systemd/system
  - source: src
    destination: /home/app
  - source: nginx.conf
    destination: /etc/nginx
  - source: requirements.txt
    destination: /home/app
  - source: fluent-bit/app.conf
    destination: /etc/fluent-bit/fluent-bit.conf.d
  - source: systemd/journald.conf
    destination: /etc/systemd
  - source: web
    destination: /home/proxy/web
file_exists_behavior: OVERWRITE
hooks:
  BeforeInstall:
    - location: scripts/stop_services.sh
    - location: scripts/setup_env_var.sh
  AfterInstall:
    - location: scripts/init_venv.sh
    - location: scripts/restart_journald.sh
    - location: scripts/start_services.sh
  ValidateService:
    - location: scripts/test_proxy.sh

I use CodeDeploy to copy all required files to the destination folders, it can be used to copy config files, source code, compiled binaries or web pages.

Because the application, nginx and fluent-bit are managed by systemd they should be stopped first, then enabled or restarted when the configuration is changed, it can be done by lifecycle hooks. Hooks can also be used to verify application state, script that exits with error code fails the deployment.

Source of scripts/test_proxy.sh

#!/bin/bash

RETRIES=10

while ((RETRIES > 0)); do
  RESPONSE=$(curl --write-out '%{http_code}' --silent --output /dev/null http://localhost:8080/health)
  if [ "$RESPONSE" = "200" ]; then
    exit 0
  fi
  RETRIES=$((RETRIES - 1))
  sleep 2
done

exit 1

CodeDeploy hooks are not only restricted to Bash scripts. It can be for example a Python script:

#!/usr/bin/python3

print("Hello")

But it needs to start with a #! followed by a path to the interpreter.

Systemd

Same as in the original Beanstalk environment systemd can be used as an application process supervisor, handle failures, run the application after EC2 instance restart and send logs to systemd-journald.

For every environment the service file may look quite different, the basic configuration may consist of ExecStart property and Type=simple but it all depends on requirements.

Here is the sample app.service file for a Python3 environment:

[Unit]
Wants=network-online.target
After=network-online.target cloud-final.service

[Service]
User=app
Group=app

Type=simple
ExecStart=/home/app/env/bin/gunicorn --bind :5000 --workers 3 --threads 2 app:app --log-level debug
RuntimeDirectory=gunicorn
WorkingDirectory=/home/app
Restart=on-failure
StandardOutput=journal
StandardError=journal
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=5
PrivateTmp=true
EnvironmentFile=/etc/app/*.env

[Install]
WantedBy=cloud-init.target

This file is deployed as part of the application bundle and copied into /etc/systemd/system by the CodeDeploy Agent as defined in appspec.yml.

Fluent-Bit

As CloudWatch Agent isn't able to collect logs from systemd-journald other logs collector should be used, I used fluent-bit because it's quite popular, the binary is very small, and it's able to fetch logs from journald and send to CloudWatch, it also offers basic logs processing with Filters and Streams.

Fluent-Bit configuration can be deployed as part of the application bundle, the fluent-bit.service needs to be restarted by the AfterInstall hook script to apply newly copied configuration.

Fluent-Bit can also monitor the environment BEFORE the first deployment, the /etc/fluent-bit.conf file can be provided during an image creation, if the file contains such line:

@INCLUDE fluent-bit.conf.d/*.conf

The additional configuration can be fetched from /etc/fluent-bit.conf.d directory, these configuration can be deployed using CodeDeploy. to apply it fluent-bit.service restart is required.

Source of /etc/fluent-bit.conf:

[SERVICE]
    Flush 1
    Log_Level info
    Parsers_File parsers.conf

[INPUT]
    Name tail
    Tag codedeploy-agent-${INSTANCE_ID}
    path /var/log/aws/codedeploy-agent/codedeploy-agent.log

[OUTPUT]
    Name cloudwatch_logs
    Match codedeploy-agent-*
    region ${REGION}
    log_stream_prefix logs-
    log_group_name /${APP_NAME}/codedeploy-agent-log-group
    auto_create_group On

[INPUT]
    Name systemd
    Tag fluent-bit-${INSTANCE_ID}
    Systemd_Filter _SYSTEMD_UNIT=fluent-bit.service

[OUTPUT]
    Name cloudwatch_logs
    Match fluent-bit-*
    region ${REGION}
    log_stream_prefix logs-
    log_group_name /${APP_NAME}/fluent-bit-log-group
    auto_create_group On

@INCLUDE fluent-bit.conf.d/*.conf

Source of /etc/fluent-bit.conf.d/app.conf that is provisioned by CodeDeploy:

[INPUT]
    Name systemd
    Tag app-${INSTANCE_ID}
    Systemd_Filter _SYSTEMD_UNIT=app.service

[OUTPUT]
    Name cloudwatch_logs
    Match app-*
    region ${REGION}
    log_stream_prefix logs-
    log_group_name /${APP_NAME}/app-log-group
    auto_create_group On

[INPUT]
    Name systemd
    Tag nginx-${INSTANCE_ID}
    Systemd_Filter _SYSTEMD_UNIT=nginx.service

[OUTPUT]
    Name cloudwatch_logs
    Match nginx-*
    region ${REGION}
    log_stream_prefix logs-
    log_group_name /${APP_NAME}/nginx-log-group
    auto_create_group On

Environment variables INSTANCE_ID, APP_NAME and REGION are provided by a fluent-bit-init-sh script started by a fluent-bit-init.service:

[Unit]
Wants=network-online.target
Before=fluent-bit.service
After=network-online.target

[Service]
Type=oneshot
ExecStart=/opt/fluent-bit-init.sh

[Install]
WantedBy=fluent-bit.service

This service needs to be started Before the fluent-bit.service to generate /etc/fluent-bit/variables.env environment file. Unfortunately it's not possible to fetch the variables by a script started with ExecStartPre of a fluent-bit.service because systemd requires the EnvironmentFile to exist before the command is started. The Type property is set to oneshot to be sure, that fluent-bit.service defined as dependent will not be started until the fluent-bit-init.service is finished.

Part of install.sh script started by Packer:

cp /tmp/fluent-bit-init.service /usr/lib/systemd/system
cp /tmp/fluent-bit-init.sh /opt
chmod +x /opt/fluent-bit-init.sh

mkdir -p /usr/lib/systemd/system/fluent-bit.service.d

cat <<EOT > /usr/lib/systemd/system/fluent-bit.service.d/00_env.conf
[Service]
EnvironmentFile=/etc/fluent-bit/variables.env
EOT

systemctl enable fluent-bit
systemctl enable fluent-bit-init

Source of fluent-bit-init.sh:

#!/bin/bash

TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
INSTANCE_IDENTITY=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/dynamic/instance-identity/document)
REGION=$(jq -r '.region' <<< "$INSTANCE_IDENTITY")
INSTANCE_ID=$(jq -r '.instanceId' <<< "$INSTANCE_IDENTITY")
APP_NAME=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/tags/instance/Name)

cat <<EOT > /etc/fluent-bit/variables.env
REGION=${REGION}
INSTANCE_ID=${INSTANCE_ID}
APP_NAME=${APP_NAME}
EOT

IMPORTANT NOTE: This script fetches the application name from instance tags, this feature is disabled by default, to have an access to instance tags using meta-data URL the following configuration should be added to the aws_launch_template:

  metadata_options {
    http_endpoint = "enabled"
    http_tokens = "required"
    http_protocol_ipv6 = "disabled"
    http_put_response_hop_limit = 1
    instance_metadata_tags = "enabled"
  }

The fluent-bit-init.sh and fluent-bit-init.service are provisioned by Packer:

build {
  sources = ["source.amazon-ebs.main"]
  provisioner "file" {
    source = "../fluent-bit/fluent-bit.conf"
    destination = "/tmp/fluent-bit.conf"
  }
  provisioner "file" {
    source = "fluent-bit-init.service"
    destination = "/tmp/fluent-bit-init.service"
  }
  provisioner "file" {
    source = "fluent-bit-init.sh"
    destination = "/tmp/fluent-bit-init.sh"
  }
  provisioner "shell" {
    script = "install.sh"
    execute_command = "echo 'packer' | sudo -S sh -c '{{ .Vars }} {{ .Path }}'"
  }
}

Journald configuration

As mention in the previous part of these series the default journald config allows only 1000 log messages within 30 seconds, by using CodeDeploy the entire journald.conf can be provisioned as part of the application bundle.

Example jorunald.conf that will be copied into /etc/systemd during the deployment process:

[Journal]
RateLimitInterval=30s
RateLimitBurst=10000

Then it can be restarted by a simple script configured as AfterInstall CodeDeploy hook

Source of scripts/restart_journald.sh:

#!/bin/bash

systemctl restart systemd-journald

Nginx

AWS Beanstalk uses Nginx as a proxy that forwards data to the application. It's recommended to use such proxy for connection buffering, it can also serve static content for the application. Nginx installed on Beanstalk instances binds to port 80, I think, that it's not required as binding to the first 1024 ports requires special capabilities. Instead, I created a new user called proxy that doesn't have any permissions and bind Nginx to port 8080, AWS Load Balancers can forward data to any port, so using Well Known Ports is not required.

My example nginx configuration is available HERE. It proxies traffic to the application and serves static content. The proxy also sets the same headers as Beanstalk:

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

In this setup the application needs to listen on port 5000.

Environment Variables

Environment Variables are very important part of the deployment infrastructure as they are the easiest way to provide URLs to Database, SQS Queues, S3 Bucket or some other services used by the application. Because of that it would be nice to be able to provide environment variables as a parameter of the code-deploy Terraform module:

module "code-deploy" {
  source = "./code_deploy"
  app-subnets = [local.first_subnet, local.second_subnet]
  elb-subnets = [local.first_subnet, local.second_subnet]
  environment_variables = {
    S3_BUCKET_ARN = aws_s3_bucket.demo-app-data-bucket.arn,
    QUEUE_URL = aws_sqs_queue.task-queue.url,
    AWS_DEFAULT_REGION = data.aws_region.current.name
  }
  vpc_id = data.aws_vpc.default-vpc.id
  app-name = "demo-app"
  app-image-name = "demo-app-image"
  app-health-path = "/health"
  max-instances = 1
  min-instances = 1
  deployment-type = "ALL_AT_ONCE"
  instances-update-policy = "ONE_AT_A_TIME"
  role-id = aws_iam_role.app-role.id
  allow-ssh = true
}

To achieve this I use EC2 UserData (which I usually use for running init script, but not this time), it can be then fetched during the deployment process with BeformInstall hook:

#!/bin/bash

mkdir -p /etc/app
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
USER_DATA=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/user-data)
echo "$USER_DATA" > /etc/app/00_infra_variables.env

The UserData can be fetched by the instance using the special IP address, then the fetched environment variables are saved into configuration file that is used by systemd to provide them to the application:

Part of the /etc/systemd/system/app.service

[Service]
EnvironmentFile=/etc/app/*.env

Of course when the UserData is used to provision environment variables there is no option to run the init script, everything needs to be provisioned during AMI creation process.

When the EnvironmentFile path is set the systemd requires it to be in format:

VAR1=VALUE1
VAR2=VALUE2

To convert provided Terraform map into the properly formatted UserData the following code can be used:

Part of the aws_launch_template resource definition:

user_data = base64encode(join("\n", [for key, value in var.environment_variables : "${key}=${value}"]))

CodeDeploy resources

The main resource that bound every other in CodeDeploy is the aws_codedeploy_app:

resource "aws_codedeploy_app" "app" {
  compute_platform = "Server"
  name = var.app-name
}

It generally configures the application name and compute_platform that for EC2 should be set to Server.

The aws_codedeploy_deployment_group configures everything related to the instances. There the AutoScaling Group is attached with autoscaling_groups field. Also, this is the right place to configure Load Balancer with load_balancer_info config and target_group_info or elb_info depending on the Load Balancer type that has been used. By providing it here and NOT in the AutoScaling Group you can be sure that EC2 Instances will be attached to the Load Balancer only after the deployment, so the AutoScaling Group will not terminate instances marked as Unhealthy by the Load Balancer because the initial version has not been deployed yet.

NOTE: As mentioned HERE CodeDeploy integrates with configured AutoScaling Group and every time a new instance is added because of Scale-Up action the current application version is deployed on this instance.

With aws_codedeploy_deployment_config you can configure deployment policies similar to ones defined by Beanstalk, like: ALL_AT_ONCE or ROLLING:

resource "aws_codedeploy_deployment_config" "app-deployment-config" {
  deployment_config_name = "${var.app-name}-deployment-config"
  minimum_healthy_hosts {
    type = "HOST_COUNT"
    value = var.deployment-type == "ALL_AT_ONCE" ? 0 : var.minimum-healthy-hosts
  }
}

The entire procedure is described HERE, it contains some good examples how this configuration affects the entire process (Even if the value is set to 0 at least one instance must succeed for the overall deployment to succeed).

Worker Environment

Besides of a typical Web Stack with a Load Balancer in front of the application Beanstalk offers Worker Environment that creates the following resources:

The queue is used to provide tasks for a Worker Application, every EC2 instance managed by AWS Beanstalk has sqsd installed, a serviced that performs the SQS queue polling and redirects fetched messages to an application with a proper HTTP endpoint configured.

It's also possible to run tasks periodically using cron expressions by providing cron.yaml file inside application bundle. These CRON mechanism also triggers configured HTTP endpoints.

To be honest I don't see any point of using Worker Environment, typical microservice with an HTTP interface also can perform background tasks fetched from an SQS queue by provisioning a separated Thread Pool, there is no need to separate such logic into another deployment stack. Even if the application is designed only to perform some tasks fetched from an SQS it still needs to use a Web Framework to create HTTP endpoints that sqsd may trigger, and if so it can be also deployed a Web Environment with one /health endpoint that a Load Balancer may test. Also, any REST API can be easily added to such service, because the infrastructure is already provided.

The CRON functionality provided by Beanstalk sends periodic tasks using SQS queue, these functionality can be provided using EventBridge (or CloudWatch Events) that can send notifications to SQS:

resource "aws_sqs_queue" "task-queue" {
  visibility_timeout_seconds = 60
}

data "aws_iam_policy_document" "sqs-allow-events" {
  statement {
    effect = "Allow"
    actions = ["sqs:SendMessage"]
    principals {
      identifiers = ["events.amazonaws.com"]
      type = "Service"
    }
    resources = ["*"]
  }
}

resource "aws_sqs_queue_policy" "allow-events" {
  policy = data.aws_iam_policy_document.sqs-allow-events.json
  queue_url = aws_sqs_queue.task-queue.url
}

resource "aws_cloudwatch_event_rule" "schedule-rule" {
  schedule_expression = "rate(1 minute)"
}

resource "aws_cloudwatch_event_target" "sqs-event-target" {
  arn = aws_sqs_queue.task-queue.arn
  rule =aws_cloudwatch_event_rule.schedule-rule.name
}

Blue/Green Deployment

AWS CodeDeploy offers Blue/Green deployment option for those who prefer to have an immutable deployment. It can provision a new AutoScaling Group for every deployment or discover the existing one. Terraform documentation for the aws_codedeploy_deployment_group suggest using DISCOVER_EXISTING policy, because otherwise Terraform will not be able to manage the newly created AutoScaling Group. The downside of this approach is you pay for IDLE resources.

The Blue/Green deployment can be configured by adding the following config to the aws_codedeploy_deployment_group resource:

autoscaling_groups = [aws_autoscaling_group.app-group[0].id]
blue_green_deployment_config {
  green_fleet_provisioning_option {
    action = "DISCOVER_EXISTING"
  }
  deployment_ready_option {
    action_on_timeout = "CONTINUE_DEPLOYMENT"
  }
  terminate_blue_instances_on_deployment_success {
    action = "KEEP_ALIVE" // TERMINATING removes ASG, not instances
  }
}

lifecycle {
  ignore_changes = [autoscaling_groups]
}

This configuration is fascinating, first question you may ask: "If DISCOVER_EXISTING is configured why there is only one ASG provided?", BECAUSE THE OTHER ONE IS PROVIDED WITH create_deployment API CALL, the targetInstances map should contain a list autoScalingGroups and this is the GREEN deployment group. So the BLUE group is initially configured with autoscaling_groups property and the GREEN group needs to be passed as an argument to the API call that triggeres the deployment. That's really confusing.

Also, the autoscaling_groups field needs to be ignored, because CodeDeploy UPDATES this field after the deployment, the AutoScaling Group that has just been provided using the API call is used to set this property internally, so Terraform will try to change it to the original one during next run. There is no possibility to omit the autoscaling_groups field, when it's omitted the deployment fails (so the field is not required but if missing then every deployment fails).

So if you expected CodeDeploy to manage the BLUE/GREEN deployment state internally and choose the proper AutoScaling Group for every triggered deployment you may be disappointed, you need to do it by yourself, fetch all configured groups, check which one is currently used and use the other one during the next deployment. It can be done easily with a script, but it should be provided by the CodeDeploy service itself.

The other thing is terminate_blue_instances_on_deployment_success, you may want to terminate the old instances, but it needs to be done with an API call, setting action to TERMINATE will remove the AutoScaling Group created by Terraform. AutoScaling Group will start some new instances after termination, but CodeDeploy will not deploy any version on these, so the instances will just run without any application.

Conclusion

The presented setup is complicated and requires some AWS knowledge, it should be used only if the environment needs to be fully customized. Also, because of its complexity it should be well documented to let the newcomers understand it, that is the main benefit of the AWS Beanstalk, it can be considered as a framework that usually works fine, some minor issues with Beanstalk can be solved by workarounds that I presented in the previous part.

There is also a third option: If you use docker for a local development (everyone does these days) you may consider using ECS, if you create a docker image with your application anyway (for example using docker-maven-plugin) it sounds like a natural choice. ECS provides multiple features for creating a microservice environment, but the ecs agent can be also used to provision a single docker application. There is nothing wrong with having an ECS Cluster with only one small EC2 instance running a single container.

AWSLinux