Dead simple Terraform configuration for creating RabbitMQ cluster on AWS

What it does ?

1. Uses official RabbitMQ docker image.
2. Creates N nodes in M subnets
3. Creates Autoscaling Group and ELB to load balance nodes
4. Makes sure nodes can talk to each other and create cluster
5. Make sure new nodes attempt to join the cluster at startup
6. Configures / vhost queues in High Available (Mirrored) mode with automatic synchronization ("ha-mode":"all", "ha-sync-mode":"3")
7. Installs and configures Datadog Agent to gather metrics and logs for RabbitMQ

How to use it ?

Copy and paste into your Terraform configuration:

module "rabbitmq" {
  source                            = "github.com/smartrent/terraform-aws-rabbitmq"
  vpc_id                            = var.vpc_id
  ssh_key_name                      = var.ssh_key_name
  subnet_ids                        = var.subnet_ids
  elb_additional_security_group_ids = [var.cluster_security_group_id]
  min_size                          = "3"
  max_size                          = "3"
  desired_size                      = "3"
  dd_env                            = var.env_name
  dd_site                           = var.datadog_site
  kms_key_arn                       = var.kms_key_id
  ecr_registry_id                   = var.ecr_registry_id
  rabbitmq_image                    = var.rabbitmq_image
  ami_id                            = var.ami_id
}

Need an AMI ID? Use empty string ("") to allow the module to get the latest for you. Remember to replace the AMI_ID with a static value as latest changes frequently and will ultimately cause perpetual drift.

Then run terraform init, terraform plan and terraform apply.

Are 3 node not enough ? Update sizes to 5 and run terraform apply again, it will update Autoscaling Group and add 2 nodes more. Dead simple.

Node becomes unresponsive ? Autoscaling group and ELB Health Checks will automatically replace it with new one, without data loss.

Note: The VPC must have enableDnsHostnames = true and enableDnsSupport = true for the private DNS names to be resolvable for the nodes to connect to each other.

Debugging

If you can SSH onto one of the nodes you can run: docker exec rabbitmq rabbitmqctl cluster_status to see the cluster status of that node.

Upgrading

Sometimes we will need to do a hot restart of a node in the cluster in order to preform some maintenance, upgrade, or infrastructure improvements. To do this graceful we need to remove the current node from the cluster. This helps keep the node in sync and organized correctly. To do this we need to stop the app and reset the node as follows:

docker exec rabbitmq rabbitmqctl cluster_status
docker exec rabbitmq rabbitmqctl stop_app
docker exec rabbitmq rabbitmqctl reset
docker exec rabbitmq rabbitmqctl cluster_status

README.md updated successfully

Requirements

Name	Version
terraform	>= 1
aws	>= 4.0

Providers

Name	Version
aws	>= 4.0
cloudinit	n/a
random	n/a

Modules

No modules.

Resources

Name	Type
aws_autoscaling_group.rabbitmq	resource
aws_elb.elb	resource
aws_iam_instance_profile.iam_profile	resource
aws_iam_policy.ssm_managed_instances	resource
aws_iam_role.iam_role	resource
aws_iam_role_policy.iam_policy	resource
aws_iam_role_policy_attachment.inspector2_managed_cis_policy	resource
aws_iam_role_policy_attachment.ssm_managed_instance_core	resource
aws_launch_template.rabbitmq	resource
aws_security_group.rabbitmq_elb	resource
aws_security_group.rabbitmq_nodes	resource
aws_ssm_parameter.datadog_api_key	resource
aws_ssm_parameter.datadog_user_password	resource
aws_ssm_parameter.rabbit_admin_password	resource
aws_ssm_parameter.rabbit_password	resource
aws_ssm_parameter.secondary_datadog_user_password	resource
aws_ssm_parameter.secondary_rabbit_admin_password	resource
aws_ssm_parameter.secondary_rabbit_password	resource
aws_ssm_parameter.secret_cookie	resource
random_password.admin_password	resource
random_password.datadog_password	resource
random_password.rabbit_password	resource
random_password.secondary_admin_password	resource
random_password.secondary_datadog_password	resource
random_password.secondary_rabbit_password	resource
random_password.secret_cookie	resource
aws_ami.amazon_linux_2	data source
aws_ami.amazon_linux_2_latest	data source
aws_iam_policy_document.policy_doc	data source
aws_iam_policy_document.policy_permissions_doc	data source
aws_iam_policy_document.ssm_managed_instances	data source
aws_region.current	data source
cloudinit_config.user_data	data source

Inputs

Name	Description	Type	Default	Required
access_log_bucket	optional bucket name to use for access logs	string	"bucketname"	no
access_log_bucket_prefix	optional prefix to use for access logs	string	""	no
access_log_interval	How often for the ELB to publish access logs in minutes	number	60	no
access_logs_enabled	Whether or not to enable access logging on the ELB	bool	false	no
ami_id	The AMI ID to use for the ec2 instance. Empty string uses a pre-configured latest.	string	""	no
aws_session_manager_enabled	Whether or not the ec2 instances in this cluster should allow session manager permissions	bool	false	no
dd_env	The environment the app is running in	string	n/a	yes
dd_site	The Datadog site url	string	n/a	yes
desired_size	Desired number of RabbitMQ nodes	number	2	no
ecr_registry_id	The ECR registry ID	string	n/a	yes
elb_additional_security_group_ids	List of additional ELB security group ids	list(string)	[]	no
enable_password_rotation	Whether to configure secondary users to facilitate password rotations.	bool	false	no
encrypted_ebs_instance_volume	Whether to encrypt the instance ebs volume	bool	true	no
health_check_grace_period	The ASG health check grace period	number	400	no
health_check_interval	The ELB health check interval in seconds	number	30	no
healthy_threshold	The ELB health check healthy threshold count	number	2	no
inspector2_cis_scanning_enabled	Whether to attach required policies for inspector 2 ec2 CIS scans	bool	true	no
instance_type	The EC2 instance type to use	string	"m5.large"	no
instance_volume_iops	The amount of provisioned iops	number	0	no
instance_volume_size	The size of the instance volume in gigabytes	number	0	no
instance_volume_type	The instance volume type to use (standard, gp2, gp3, st1, sc1, io1)	string	"standard"	no
kms_key_arn	The KMS key arn to use for encrypting and decrypting SSM parameters	string	n/a	yes
max_size	Maximum number of RabbitMQ nodes	number	2	no
min_size	Minimum number of RabbitMQ nodes	number	2	no
name	The name of the RabbitMQ cluster	string	"main"	no
nodes_additional_security_group_ids	List of additional node security group ids	list(string)	[]	no
rabbitmq_image	The RabbitMQ docker image	string	n/a	yes
session_manager_kms_encryption_enabled	Whether to enable session manager kms encryption to protect session data	bool	false	no
session_manager_kms_key_arn	The kms key arn to use for session manager session encryption	string	""	no
ssh_key_name	The ssh key to provide the instance to use for ssh login	string	n/a	yes
subnet_ids	Subnets for RabbitMQ nodes	list(string)	n/a	yes
tags	Optional additional Tags to add onto resources this module creates	map(string)	{}	no
timeout	The ELB health check length of time before timeout in seconds	number	3	no
unhealthy_threshold	The ELB health check unhealthy threshold count	number	10	no
upgrading_minor_version_number	The value of the minor version you are upgrading to.	number	0	no
use_secondary_users	Whether the instance should use the secondary users.	bool	false	no
vpc_id	n/a	any	n/a	yes

Outputs

Name	Description
admin_password	n/a
amqp_url	n/a
iam_profile_arn	n/a
iam_role	n/a
rabbit_password	n/a
rabbitmq_elb_dns	n/a
secondary_admin_password	n/a
secondary_amqp_url	n/a
secondary_rabbit_password	n/a
secret_cookie	n/a
ssm_managed_instances_policy_arn	n/a