Skip to content

Commit

Permalink
Merge pull request #1260 from jembi/emergency-backup-zip
Browse files Browse the repository at this point in the history
Emergency backup/restore cron job / pen testing setup / municipality support / hotfixes
  • Loading branch information
euanmillar authored Nov 26, 2019
2 parents c84e70e + 3866261 commit b46f7ae
Show file tree
Hide file tree
Showing 53 changed files with 1,205 additions and 501 deletions.
17 changes: 14 additions & 3 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
OpenCRVS was originally coded between 2017 and 2019 by the organisations
Plan International, Jembi Health Systems & Dynamic Solution Innovators
OpenCRVS has been produced by the funding and support from the organisations
Plan International, Jembi Health Systems & Vital Strategies,
with coding assistance from Dynamic Solution Innovators & Futurice.

https://plan-international.org/
https://www.jembi.org/
http://www.dsinnovators.com/
https://www.vitalstrategies.org/
https://www.futurice.com/

Here is an inevitably incomplete list of MUCH-APPRECIATED CONTRIBUTORS --
people who have submitted code and generally made OpenCRVS that much better:
Expand All @@ -12,7 +15,7 @@ Euan Millar https://github.com/euanmillar
Ryan Crichton https://github.com/rcrichton
Riku Rouvila https://github.com/rikukissa
Anik Mushraful Hoque https://github.com/mushrafulhoque-dsi
Asif Atick https://github.com/asifatick
Atiq Zaman https://github.com/atiqzaman-dsi
Tahmid Rahman https://github.com/tahmidrahman-dsi
Sahriar Toufiq https://github.com/sahriartoufiq
Yeasin Hossain https://github.com/yeasinhossain-dsi
Expand All @@ -27,6 +30,14 @@ The core product and design team, list of supporters and supporting organisation
can be found at https://www.opencrvs.org without who's leadership, funding and guidance
OpenCRVS would never have existed. THANK YOU!

OpenCRVS is the brainchild of our Product Manager Ed Duffus from Plan International.

Ed Duffus https://github.com/eduffus

Extra special thanks to our Plan International Product Owner Annina Wersun, and Design Lead Jonathan Pye-Finch

Design and Product thanks must also be extended to Umesh Pandya, Isa Jokela-Gomes & George Bevan.

This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at https://mozilla.org/MPL/2.0/.
Expand Down
34 changes: 24 additions & 10 deletions deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,12 @@
set -e

print_usage_and_exit () {
echo 'Usage: ./deploy.sh COUNTRY --clear-data=yes|no --restore-metadata=yes|no HOST VERSION'
echo 'Usage: ./deploy.sh COUNTRY --clear-data=yes|no --restore-metadata=yes|no HOST ENV VERSION'
echo " Script must receive a first COUNTRY parameter of 'bgd' or 'zmb' set as a supported alpha-3 country code e.g.: ./deploy.sh bgd"
echo " --clear-data must have a value of 'yes' or 'no' set e.g. --clear-data=yes"
echo " --restore-metadata must have a value of 'yes' or 'no' set e.g. --restore-metadata=yes"
echo ' HOST is the server to deploy to'
echo " ENV can be 'production' or 'development'"
echo " VERSION can be any docker image tag or 'latest'"
exit 1
}
Expand All @@ -40,13 +41,19 @@ if [ -z "$4" ] ; then
fi

if [ -z "$5" ] ; then
echo 'Error: Argument VERSION is required in postition 5.'
echo 'Error: Argument ENV is required in postition 5.'
print_usage_and_exit
fi

if [ -z "$6" ] ; then
echo 'Error: Argument VERSION is required in postition 6.'
print_usage_and_exit
fi

COUNTRY=$1
HOST=$4
VERSION=$5
ENV=$5
VERSION=$6
SSH_USER=${SSH_USER:-root}
SSH_HOST=${SSH_HOST:-$HOST}
LOG_LOCATION=${LOG_LOCATION:-/var/log}
Expand All @@ -55,16 +62,16 @@ echo
echo "Deploying version $VERSION to $SSH_HOST..."
echo

mkdir -p /tmp/compose/infrastructure/backups
mkdir -p /tmp/compose/infrastructure/default_backups

# Copy selected country config to public & infrastructure folder
cp packages/resources/src/$COUNTRY/config/client-config.prod.js /tmp/compose/infrastructure/client-config.js
cp packages/resources/src/$COUNTRY/config/login-config.prod.js /tmp/compose/infrastructure/login-config.js

# Copy selected country backups to infrastructure backups folder
cp packages/resources/src/$COUNTRY/backups/hearth-dev.gz /tmp/compose/infrastructure/backups/hearth-dev.gz
cp packages/resources/src/$COUNTRY/backups/openhim-dev.gz /tmp/compose/infrastructure/backups/openhim-dev.gz
cp packages/resources/src/$COUNTRY/backups/user-mgnt.gz /tmp/compose/infrastructure/backups/user-mgnt.gz
# Copy selected country default backups to infrastructure default_backups folder
cp packages/resources/src/$COUNTRY/backups/hearth-dev.gz /tmp/compose/infrastructure/default_backups/hearth-dev.gz
cp packages/resources/src/$COUNTRY/backups/openhim-dev.gz /tmp/compose/infrastructure/default_backups/openhim-dev.gz
cp packages/resources/src/$COUNTRY/backups/user-mgnt.gz /tmp/compose/infrastructure/default_backups/user-mgnt.gz

# Copy all infrastructure files to the server
rsync -rP docker-compose* infrastructure $SSH_USER@$SSH_HOST:/tmp/compose/
Expand All @@ -76,13 +83,20 @@ rsync -rP packages/resources/src/$COUNTRY/config/docker-compose* infrastructure
rsync -rP /tmp/compose/infrastructure $SSH_USER@$SSH_HOST:/tmp/compose

# Prepare docker-compose.deploy.yml and docker-compose.<COUNTRY>.yml file - rotate secrets etc
ssh $SSH_USER@$SSH_HOST '/tmp/compose/infrastructure/rotate-secrets.sh /tmp/compose/docker-compose.deploy.yml /tmp/compose/docker-compose.'$COUNTRY'.deploy.yml | tee -a '$LOG_LOCATION'/rotate-secrets.log'

if [[ "$ENV" = "development" ]]; then
ssh $SSH_USER@$SSH_HOST '/tmp/compose/infrastructure/rotate-secrets.sh /tmp/compose/docker-compose.deploy.yml /tmp/compose/docker-compose.'$COUNTRY'.deploy.yml | tee -a '$LOG_LOCATION'/rotate-secrets.log'
else
ssh $SSH_USER@$SSH_HOST '/tmp/compose/infrastructure/rotate-secrets.sh /tmp/compose/docker-compose.deploy.yml /tmp/compose/docker-compose.prod-deploy.yml /tmp/compose/docker-compose.'$COUNTRY'.deploy.yml | tee -a '$LOG_LOCATION'/rotate-secrets.log'
fi
# Setup configuration files and compose file for the deployment domain
ssh $SSH_USER@$SSH_HOST '/tmp/compose/infrastructure/setup-deploy-config.sh '$HOST' | tee -a '$LOG_LOCATION'/setup-deploy-config.log'

# Deploy the OpenCRVS stack onto the swarm
if [[ "$ENV" = "development" ]]; then
ssh $SSH_USER@$SSH_HOST 'cd /tmp/compose && COUNTRY='$COUNTRY' VERSION='$VERSION' docker stack deploy -c docker-compose.deps.yml -c docker-compose.yml -c docker-compose.deploy.yml -c docker-compose.'$COUNTRY'.deploy.yml --with-registry-auth opencrvs'
else
ssh $SSH_USER@$SSH_HOST 'cd /tmp/compose && COUNTRY='$COUNTRY' VERSION='$VERSION' docker stack deploy -c docker-compose.deps.yml -c docker-compose.yml -c docker-compose.deploy.yml -c docker-compose.prod-deploy.yml -c docker-compose.'$COUNTRY'.deploy.yml --with-registry-auth opencrvs'
fi

if [ $2 == "--clear-data=yes" ] || [ $3 == "--restore-metadata=yes" ] ; then
echo
Expand Down
102 changes: 53 additions & 49 deletions docker-compose.deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
#
# Copyright (C) The OpenCRVS Authors. OpenCRVS and the OpenCRVS
# graphic logo are (registered/a) trademark(s) of Plan International.
version: '3.3'
version: "3.3"

services:
# Configure reverse proxy for public endpoints
Expand All @@ -16,8 +16,8 @@ services:
traefik:
image: traefik:1.7.16
ports:
- '80:80'
- '443:443'
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /data/traefik/acme.json:/acme.json
Expand All @@ -36,7 +36,7 @@ services:
mongo1:
command: --replSet rs0
volumes:
- '/data/mongo:/data/db'
- "/data/mongo:/data/db"
deploy:
replicas: 1
placement:
Expand All @@ -50,7 +50,7 @@ services:
restart: unless-stopped
command: --replSet rs0
volumes:
- '/data/mongo:/data/db'
- "/data/mongo:/data/db"
deploy:
replicas: 1
placement:
Expand All @@ -64,7 +64,7 @@ services:
restart: unless-stopped
command: --replSet rs0
volumes:
- '/data/mongo:/data/db'
- "/data/mongo:/data/db"
deploy:
replicas: 1
placement:
Expand Down Expand Up @@ -94,7 +94,10 @@ services:
# Configure elasticsearch
elasticsearch:
volumes:
- '/data/elasticsearch:/usr/share/elasticsearch/data'
- "/data/elasticsearch:/usr/share/elasticsearch/data"
- "/backups/elasticsearch:/backups/elasticsearch"
- "/tmp/compose/infrastructure/elasticsearch/elasticsearch.yml:/config/elasticsearch.yml"
- "/tmp/compose/infrastructure/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml"
deploy:
replicas: 1
placement:
Expand All @@ -106,7 +109,8 @@ services:
# Configure influxdb
influxdb:
volumes:
- '/data/influxdb:/var/lib/influxdb'
- "/data/influxdb:/var/lib/influxdb"
- "/backups/influxdb:/backups/influxdb"
deploy:
replicas: 1
placement:
Expand All @@ -122,12 +126,12 @@ services:
target: /usr/share/nginx/html/config.js
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: register.{{hostname}},{{hostname}}'
- 'traefik.frontend.redirect.regex=^https?://{{hostname}}/(.*)'
- 'traefik.frontend.redirect.replacement=https://register.{{hostname}}/$${1}'
- 'traefik.port=80'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: register.{{hostname}},{{hostname}}"
- "traefik.frontend.redirect.regex=^https?://{{hostname}}/(.*)"
- "traefik.frontend.redirect.replacement=https://register.{{hostname}}/$${1}"
- "traefik.port=80"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
networks:
- overlay_net
Expand All @@ -138,21 +142,21 @@ services:
target: /usr/share/nginx/html/config.js
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: performance.{{hostname}}'
- 'traefik.port=80'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: performance.{{hostname}}"
- "traefik.port=80"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
networks:
- overlay_net

styleguide:
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: styleguide.{{hostname}}'
- 'traefik.port=6060'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: styleguide.{{hostname}}"
- "traefik.port=6060"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 1
networks:
- overlay_net
Expand All @@ -163,10 +167,10 @@ services:
target: /usr/share/nginx/html/config.js
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: login.{{hostname}}'
- 'traefik.port=80'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: login.{{hostname}}"
- "traefik.port=80"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
networks:
- overlay_net
Expand All @@ -181,10 +185,10 @@ services:
- CERT_PUBLIC_KEY_PATH=/run/secrets/jwt-public-key.{{ts}}
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: auth.{{hostname}}'
- 'traefik.port=4040'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: auth.{{hostname}}"
- "traefik.port=4040"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
networks:
- overlay_net
Expand Down Expand Up @@ -217,10 +221,10 @@ services:
- CERT_PUBLIC_KEY_PATH=/run/secrets/jwt-public-key.{{ts}}
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: gateway.{{hostname}}'
- 'traefik.port=7070'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: gateway.{{hostname}}"
- "traefik.port=7070"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
networks:
- overlay_net
Expand Down Expand Up @@ -254,10 +258,10 @@ services:
- TEST_USER_PASSWORD=test
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: resources.{{hostname}}'
- 'traefik.port=3040'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: resources.{{hostname}}"
- "traefik.port=3040"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
networks:
- overlay_net
Expand Down Expand Up @@ -302,24 +306,24 @@ services:
- mongo3
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.docker.network=opencrvs_overlay_net'
- 'traefik.openhim-api.frontend.rule=Host: openhim-api.{{hostname}}'
- 'traefik.openhim-api.protocol=https'
- 'traefik.openhim-api.port=8080'
- 'traefik.router.frontend.rule=Host: api.{{hostname}}'
- 'traefik.router.port=5001'
- "traefik.enable=true"
- "traefik.docker.network=opencrvs_overlay_net"
- "traefik.openhim-api.frontend.rule=Host: openhim-api.{{hostname}}"
- "traefik.openhim-api.protocol=https"
- "traefik.openhim-api.port=8080"
- "traefik.router.frontend.rule=Host: api.{{hostname}}"
- "traefik.router.port=5001"
replicas: 2
networks:
- overlay_net

openhim-console:
deploy:
labels:
- 'traefik.enable=true'
- 'traefik.frontend.rule=Host: openhim.{{hostname}}'
- 'traefik.port=80'
- 'traefik.docker.network=opencrvs_overlay_net'
- "traefik.enable=true"
- "traefik.frontend.rule=Host: openhim.{{hostname}}"
- "traefik.port=80"
- "traefik.docker.network=opencrvs_overlay_net"
replicas: 2
configs:
- source: openhim-console-conf.{{ts}}
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.deps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
#
# Copyright (C) The OpenCRVS Authors. OpenCRVS and the OpenCRVS
# graphic logo are (registered/a) trademark(s) of Plan International.
version: '3.3'
version: "3.3"

services:
mongo1:
Expand Down
43 changes: 43 additions & 0 deletions docker-compose.prod-deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at https://mozilla.org/MPL/2.0/.
#
# OpenCRVS is also distributed under the terms of the Civil Registration
# & Healthcare Disclaimer located at http://opencrvs.org/license.
#
# Copyright (C) The OpenCRVS Authors. OpenCRVS and the OpenCRVS
# graphic logo are (registered/a) trademark(s) of Plan International.
version: '3.3'

services:
gateway:
environment:
- NODE_ENV=production

workflow:
environment:
- NODE_ENV=production

search:
environment:
- NODE_ENV=production

metrics:
environment:
- NODE_ENV=production

auth:
environment:
- NODE_ENV=production

user-mgnt:
environment:
- NODE_ENV=production

notification:
environment:
- NODE_ENV=production

resources:
environment:
- NODE_ENV=production
Loading

0 comments on commit b46f7ae

Please sign in to comment.