Backups Strategy

Every application should be configured to generate daily backup tar/zip files and store them in /opt/backups folder; make sure the user backup-s3 can read and write those files.

Every day at 4am UTC, a cron task ran by user backup-s3 will upload all files in /opt/backups to AWS S3 (s3://openmrs-backups/<hostname>), server-side encrypted with AWS KMS. S3 is configured to archive to glacier after 30 days, and delete after 6 months (glacier is more expensive to retrieve).

Files are deleted from filesystem after a successful upload. Cron task logs can be found in /home/backup-s3/backup.logs

AWS credentials are unique per server, and should not be shared. That user has only permission to write files under hostname folder in S3.

Configuring backups

Make sure the terraform stack as either 'has_backup=true' or has module "backup-user". When applying the stack, you should receive the AWS backup credentials for that server.

You can now go to ansible:

• Add the machine to 'backup' group
• Add AWS credentials to host vars (make sure they are encrypted in vars
• Deploy the other cron tasks or relevant tasks to generate files in /opt/backups
• Add the following variable to the host: backup_tag: 'configured'

*** Exception are talk/discourse and wordpress/site. They are configured to upload their backups straight to S3, bucket openmrs-talk-backups and openmrs-site-backups.

Download and verify backups

• Login to AWS console, search for 'S3'
• Download file
• Profit!

In datadog, you can group machines by their backup condition:

• non-applicable (no state to have a backup)
• bootstrapped (scripts to upload to S3 in place, applications not yet configured to generate tar files)
• configured (backups working as expected)

How to upload manual backups

For all manually uploaded backups, use S3 bucket openmrs-manual-backup.

• On AWS console, create a new key pair for your user. Go to IAM Users -> {Your User} -> Security Credentials -> Create access key. Download the csv file, and keep it safe!
• On AWS console, go to S3 -> 'openmrs-manual-backup'. Verify there's a folder for the product you are uploading the backups. Otherwise, create a folder now.
• Install aws cli on a machine with the backups pip install awscli
• Run 'aws configure' on the machine containing the file(s) to be backed up. Add the access key created before, and region 'us-west-2'.
• Run aws cli to upload files to the s3 bucket. For example: aws s3 cp backup-2016-09-03.tgz s3://openmrs-manual-backup/nexus/backup-2016-09-03.tgz to upload a file to the folder nexus.
• After the uploads, please deactivate the access key from the amazon console. You should always activate an access key every time there's a desire to upload something and deactivate it afterwards.