Matross is our collection of opinionated Capistrano recipes. We made a bunch of additions and customizations. Below we list the most relevant ones.
- Foreman by default: we use
foremanto environment variables, init scripts, task definitions and more. - Custom foreman upstart template: we also leverage
foreman's templates to build a custom upstart template that enablesconsole log, allowinglogrotateto work properly.
Put matross in the :development group of your Gemfile:
group :development do
gem 'matross'
endRun bundle exec capify . in the project root folder:
$ bundle exec capify .Find a full example down this README.
We have our opinions, but don't know everything. What works for us, may not fit your needs since each app is a unique snowflake. To take care of that, matross allows you to define your own templates instead of the built in ones. Look at the included ones in lib/matross/templates to see how we think things should go.
Foreman has freed us of the tedious task of writing init and Upstart scripts. Some of our matross recipes automatically add processes - such as the unicorn server - to the Procfile.
If you have an application Procfile with custom daemons defined, such as Rake task, they will be concatenated with all the processes defined in matross, resulting in one final Procfile-matross file that will be used to start your application and export init scrips.
You can specify the number of each instance defined in Procfile-matross using the foreman_procs variable.
Suppose you have a process called dj and want to export 3 instances of it:
set :foreman_procs, {
dj: 3
}We also modified the default upstart template to log through upstart instead of just piping stdout and stderr into files. Goodbye nocturnal logexplosion. (Like all templates you can override it!).
If you have custom tasks that should also be started, simply list them in the Procfile in the root of your application. They will be appended to the recipe's task definitions (eg.: unicorn).
custom_task: bundle exec rake custom_task
If there are any environment variables that you want to use, just set them in a .env file in the root of your application. Please note that RAILS_ENV is properly set during foreman tasks.
CUSTOM_TASK_ENV=boost
If you require different values for these environment variables across deployment stages, define them separately in .env-#{stage} files, for example .env-production.
Requires having foreman available in the application. As mentioned before, we use foreman in production to save us from generating upstart init scripts. As a bonus we get sane definition of environment variables.
Overwritable template: process.conf.erb
Variables
| Variable | Default value | Description |
|---|---|---|
:foreman_user |
{ user } - The user defined in Capistrano |
The user which should run the tasks defined in the Procfile |
:foreman_bin |
'bundle exec foreman' |
The foreman command |
:foreman_procs |
{} - Defaults to one per task definition |
Number of processes for each task definition in the Procfile |
Tasks
| Task | Description |
|---|---|
foreman:pre_setup |
Creates the upstart folder in the shared_path |
foreman:setup |
Merges all partial Procfiles and .envs, including the appropriate RAILS_ENV |
foreman:export |
Export the task definitions as Upstart scripts |
foreman:symlink |
Symlink .env-matross and Procfile-matross to current_path |
foreman:log |
Symlink Upstart logs to the log folder in shared_path |
foreman:stop |
Stop all of the application tasks |
foreman:restart |
Restart or start all of the application tasks |
foreman:remove |
Remove all of the application tasks from Upstart |
Requires having unicorn available in the application. By loading our unicorn recipe, you get our default configuration.
Overwritable template: unicorn.rb.erb
Procfile task: web: bundle exec unicorn -c <%= unicorn_config %> -E <%= rails_env %>
Variables
| Variable | Default value | Description |
|---|---|---|
:unicorn_config |
"#{shared_path}/config/unicorn.rb" |
Location of the configuration file |
:unicorn_log |
"#{shared_path}/log/unicorn.log" |
Location of unicorn log |
:unicorn_workers |
Number of cores specified by /proc/cpuinfo | Number of unicorn workers |
Tasks
| Task | Description |
|---|---|
unicorn:setup |
Creates the unicorn.rb configuration file in the shared_path |
unicorn:procfile |
Defines how unicorn should be run in a temporary Procfile |
This recipes creates and configures the virtual_host for the application. [This virtual host] has some sane defaults, suitable for most of our deployments (non-SSL). The file is created at /etc/nginx/sites-available and symlinked to /etc/nginx/sites-enabled. These are the defaults for the Nginx installation in Ubuntu. You can take a look at our general nginx.conf.
Overwritable template: nginx_virtual_host_conf.erb
Variables
| Variable | Default value | Description |
|---|---|---|
:htpasswd |
None | htpasswd user:password format |
:nginx_default_server |
false |
Sets the vhost for the specified stage as the default |
Tasks
| Task | Description |
|---|---|
nginx:setup |
Creates the virtual host file |
nginx:reload |
Reloads the Nginx configuration |
nginx:lock |
Sets up the a basic http auth on the virtual host |
nginx:unlock |
Removes the basic http auth |
Requires having mysql2 available in the application. In our MySQL recipe we dynamically generate a database.yml based on the variables that should be set globally or per-stage.
The backup routine requires s3cmd installed and properly configured. The key pair must have write access to the backup bucket on S3.
Overwritable template: database.yml.erb
Variables
| Variable | Default value | Description |
|---|---|---|
:database_config |
"#{shared_path}/config/database.yml" |
Location of the configuration file |
:mysql_host |
None | MySQL host address |
:mysql_database |
None | MySQL database name. Dashes - are gsubed for underscores _ |
:mysql_user |
None | MySQL user |
:mysql_passwd |
None | MySQL password |
:mysql_backup_script |
"#{shared_path}/matross/mysql_backup.sh" |
MySQL backup script location |
:mysql_backup_cron_schedule |
'30 3 * * *' |
Cron schedule for the backup script |
:mysql_backup_bucket |
None | Bucket used to store the dumps |
:mysql_backup_prefix |
None | Flat file prefix for the backup files |
Tasks
| Task | Description |
|---|---|
mysql:setup |
Creates the database.yml in the shared_path |
mysql:symlink |
Creates a symlink for the database.yml file in the current_path |
mysql:create |
Creates the database if it hasn't been created |
mysql:schema_load |
Loads the schema if there are no tables in the DB |
mysql:backup:setup |
Creates the backup script and configures the user's cron - Not hooked to any other task |
mysql:dump:do |
Dumps the application database |
mysql:dump:get |
Downloads a copy of the last generated database dump |
mysql:dump:apply |
Applies the latest dump generated stored in 'dumps' locally |
mysql:dump:post |
Uploads a copy of the last generated database dump |
mysql:dump:apply_remotely |
Applies the latest uploaded dump in the remote server |
Requires having pg available in the application. In our PostgreSQL recipe we dynamically generate a database.yml based on the variables that should be set globally or per-stage.
Please note that we rely on local peer connection, hence, remote connections should use a custom template for database.yml.erb. The PG cluster must have UTF8 encoding.
Overwritable template: database.yml.erb
Variables
| Variable | Default value | Description |
|---|---|---|
:database_config |
"#{shared_path}/config/database.yml" |
Location of the configuration file |
:postgresql_database |
None | PostgreSQL database name. Dashes - are gsubed for underscores _ |
:postgresql_user |
user |
PostgreSQL user |
Tasks
| Task | Description |
|---|---|
postgresql:setup |
Creates the database.yml in the shared_path, and the user if needed |
postgresql:symlink |
Creates a symlink for the database.yml file in the current_path |
postgresql:create |
Creates the database if it hasn't been created and loads the schema |
Requires having mongoid available in the application. In our Mongoid recipe we dynamically generate a mongoid.yml based on the variables that should be set globally or per-stage.
Overwritable template: mongoid.yml.erb
Variables
| Variable | Default value | Description |
|---|---|---|
:mongoid_config |
"#{shared_path}/config/mongoid.yml" |
Location of the mongoid configuration file |
:mongo_hosts |
None | List of MongoDB hosts |
:mongo_database |
None | MongoDB database name |
:mongo_user |
None | MongoDB user |
:mongo_passwd |
None | MongoDB password |
Tasks
| Task | Description |
|---|---|
mongoid:setup |
Creates the mongoid.yml in the shared_path |
mongoid:symlink |
Creates a symlink for the mongoid.yml file in the current_path |
Requires having delayed_job available in the application.
Procfile task: dj: bundle exec rake jobs:work or dj_<%= queue_name %>: bundle exec rake jobs:work QUEUE=<%= queue_name %>
Variables
| Variable | Default value | Description |
|---|---|---|
:dj_queues |
None | List of queues |
Tasks
| Task | Description |
|---|---|
delayed_job:procfile |
Defines how delayed_job should be run in a temporary Procfile |
Requires having fog available in the application. When we use fog, it is for interacting with Amazon services, once again very opinionated.
Overwritable template: fog_config.yml.erb
The configuration generated may be used by other gems, such as carrierwave. Here is an example of how we use it:
# config/initializers/carrierwave.rb
CarrierWave.configure do |config|
fog_config = YAML.load(File.read(File.join(Rails.root, 'config', 'fog_config.yml')))
config.fog_credentials = {
:provider => 'AWS',
:aws_access_key_id => fog_config['aws_access_key_id'],
:aws_secret_access_key => fog_config['aws_secret_access_key'],
:region => fog_config['region']
}
config.fog_directory = fog_config['directory']
config.fog_public = fog_config['public']
endVariables
| Variable | Default value | Description |
|---|---|---|
:fog_config |
"#{shared_path}/config/fog_config.yml" |
Location of the fog configuration file |
:fog_region |
'us-east-1' |
AWS Region |
:fog_public |
false |
Bucket policy |
:fog_aws_access_key_id |
None | AWS Access Key Id |
:fog_aws_secret_access_key |
None | AWS Secret Access Key |
Tasks
| Task | Description |
|---|---|
fog:setup |
Creates the fog_config.yml in the shared_path |
fog:symlink |
Creates a symlink for the fog_config.yml file in the current_path |
Requires having faye available in the application.
Overwritable templates: faye.ru.erb and faye_server.yml
Procfile task: faye: bundle exec rackup <%= faye_ru %> -s thin -E <%= rails_env %> -p <%= faye_port %>
Variables
| Variable | Default value | Description |
|---|---|---|
:faye_config |
"#{shared_path}/config/faye_config.yml" |
Location of the faye parameters configuration file |
:faye_ru |
"#{shared_path}/config/faye.ru" |
Location of the faye configuration file |
:faye_port |
None | Which port faye should listen on |
Tasks
| Task | Description |
|---|---|
faye:setup |
Creates faye_config.yml and faye.ru in the shared_path |
faye:symlink |
Creates a symlink for the faye_config.yml file in the current_path |
In order to deal with memory or CPU constrained production servers, this recipe overwrites the default assets precompilation by compiling them locally and then uploading the result to the server.
Rake precompiles assets in a production to properly generate file names with hashes. One of the gotchas to this is that by default Rails initializes the entire applicaction when executing the assets:precompile task. If you use different databases in production and development and run into database connection errors with this task you can override this behavior.
config/application.rb
module AwesomeApplication
class Application < Rails::Application
...
config.assets.initialize_on_precompile = false if Rails.env.production?
...
end
endBelow is a full example of how to use matross exhaustively. Do note that this would be an edge case as, for example, you don't normally run mongoid with mysql.
config/deploy.rb
set :stages, %w(production staging)
set :default_stage, 'staging'
require 'capistrano/ext/multistage'
require 'bundler/capistrano'
require 'matross'
load 'matross/local_assets'
load 'matross/nginx'
load 'matross/unicorn'
load 'matross/faye'
load 'matross/delayed_job'
load 'matross/fog'
load 'matross/mongoid'
load 'matross/mysql'
load 'matross/foreman'
set :application, 'awesome_application'
set :repository, '[email protected]:innvent/awesome_application.git'
set :ssh_options, { :forward_agent => true }
set :scm, :git
set :scm_verbose, true
set :deploy_via, :remote_cache
set :shared_children, %w(public/system log tmp/pids public/uploads)
default_run_options[:pty] = true
logger.level = Capistrano::Logger::DEBUG
after 'deploy:update', 'deploy:cleanup'
config/deploy/production.rb
set :user, 'ubuntu'
set :group, 'ubuntu'
set :use_sudo, false
set :branch, 'master'
set :rails_env, 'production'
set :deploy_to, "/home/#{user}/#{application}"
set :server_name, 'example.com'
set :mongo_hosts, [ 'localhost' ]
set :mongo_database, "#{application}_#{rails_env}"
set :mysql_host, 'localhost'
set :mysql_database, "#{application}_#{rails_env}"
set :mysql_user, "#{user}"
set :mysql_passwd, ''
set :faye_port, '9292'
set :faye_local, true
set :htpasswd, 'admin:$apr1$twOdKBdh$okL.giy91y9LzXsD5swUb0'
set :dj_queues, [ 'queue1', 'queue2' ]
set :fog_aws_access_key_id, 'AKIACS5E2GU9NND0NMD4'
set :fog_aws_secret_access_key, 'rNB38h5Y4ysUM3r10F3oehrnp2ZaBcUPtiOnJyLn'
set :fog_directory, 'awesome_application_production'
set :foreman_procs, { 'dj_queue1' => 2, 'dj_queue2' => 3 }
server '192.168.1.1', :app, :web, :dj, :faye, :db, :primary => true
set :default_environment, {
'PATH' => "/home/#{user}/.rbenv/shims:/home/#{user}/.rbenv/bin:$PATH"
}
Capfile
load 'deploy'
load 'deploy/assets'
load 'config/deploy'