Feature/enhanced offsiting

CHANGELOG.md

@@ -1,6 +1,21 @@
 # Change Log
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).
 
+## 0.0.5 - 2017-01-27
+### Added
+- Created new function to offsite SnapShots based upon presence of 'DestinationRegion' Tag
+- All Tags on the Source snapShot will be applied to the copy
+- Ability to exclude individual EBS Volumes
+- Linked original and copied snapshots by Tagging with the other's respective Id
+- Added YAML CFN to create Lambda Functions from 'ebs-snapshot-creator', 'ebs-snapshot-manager' & 'ebs-snapshot-offsiter' along with IAM Roles and CloudWatch Cron triggers
+
+### Changed
+- Automatic determination of curent AWS Region
+- All variables obtained through Tags rather than hard-coded
+
+### Fixed
+- Nothing so far
+
 ## 0.0.4 [unreleased]
 ### Added
 - WIP: Out-of-region snapshot support

IDEAS.md

@@ -6,23 +6,23 @@
   - DONE: Copying a snapshot to an additional region should be possible within the creator (hardcoded in creator)
   - DONE: Enabling snapshot copying out-of-region should be easily configurable in the creator script (albeit still requiring a variable parameter change)
   - DONE: Only copy snapshot out of region if a copy_region is defined in the creator script
-  - The out-of-region/copy snapshot functionality should be in its own dedicated job 
+  - DONE: The out-of-region/copy snapshot functionality should be in its own dedicated job 
     - because snapshots can't be copied until they're in a completed state (and this enables getting closer to that)
     - Job/function is easy to understand (logical point of separation)
-  - Copies of snapshots in the additional region should be tagged in the same manner as in-region snapshots (Automated: Yes, expiration info, etc.)
-  - Enabling the copying (duplication) of a snapshot out-of-region should be configurable on a per instance basis
-  - Out-of-region snapshots should be managed (for expiration/retention) just like in-region snapshots
+  - DONE: Copies of snapshots in the additional region should be tagged in the same manner as in-region snapshots (Automated: Yes, expiration info, etc.)
+  - DONE: Enabling the copying (duplication) of a snapshot out-of-region should be configurable on a per instance basis
+  - DONE: Out-of-region snapshots should be managed (for expiration/retention) just like in-region snapshots
 
 ### P2
 - It should be possible to get automatically notified when the job (a Lambda function) emits an error
   - e.g. http://docs.aws.amazon.com/lambda/latest/dg/with-scheduledevents-example.html
-- The required minimum IAM role policy should be provided
+- DONE: The required minimum IAM role policy should be provided
 
 ### P3
 - It should be possible to configure multiple regions to copy (duplicate) snapshots into
 - It should be possible to trigger a web hook (optionally / if configured) every time the creator job runs 
   - e.g. to use with PagerDuty to monitor if a job doesn't check-in every N days/hours/whatever
-- Add JSON for IAM and/or CloudFormation and/or Terraform code and/or CLI/SH for deploying
+- ADDED CFN: Add JSON for IAM and/or CloudFormation and/or Terraform code and/or CLI/SH for deploying
 - It should be possible to trigger snapshots of instance volumes in other regions besides the one that the creator is running in (or should it?)
 
 ## ebs-snapshot-manager.py
@@ -41,7 +41,7 @@
 
 ### P3
 - Trigger a optional (if configured) web hook every time it runs (e.g. to use with PagerDuty to trigger if job doesn't check-in every N days/hours/whatever)
-- Add JSON for IAM and/or CloudFormation and/or Terraform code and/or CLI/SH for deploying
+- ADDED CFN: Add JSON for IAM and/or CloudFormation and/or Terraform code and/or CLI/SH for deploying
 
 ## ebs-snapshot-watcher.py
 

README.md

@@ -9,6 +9,18 @@ This is for managing AWS EC2 EBS volume snapshots. It consists of a snapshot cre
 - Ability to configure retention period on a per EC2 instance basis (applying to all volumes attached to said instance)
 - Ability to manually tag individual snapshots to be kept indefinitely (regardless of instance retention configuration)
 - Does not require a job/management instance; no resources to provision to run snapshot jobs (leverages AWS Lambda)
+- Ability to snapshot all Volumes attached to a given Instance (Default), and exclude on a per-Volume basis any indivdual Volume (Through the addition of `Backup = No` Tag to Volume)
+- Ability to replicate snapshot to a second AWS Region (As specified by Tag) and remove snapshot from source Region upon successful copy. Tags are replicated from source to destination snapshots
+
+## Tags Configuration
+
+- Instance Level
+	- `Backup` { Yes | No }
+	- `DestinationRegion` { us-west-1 | eu-west-1 | etc. }
+	- `RetentionDays` { 1..x }
+
+- Volume Level
+	- `Backup` { Yes | No } (Default if absent = 'Yes') : Overrides default to exclude a given Volume from snapshot
 
 ## Implementation Details
 
@@ -24,11 +36,40 @@ For the moment, read these links for documentation on how to setup/use. I've ext
 
 Ideas and To Do items are currently tracked in [IDEAS](IDEAS.md).
 
+## IAM Role
+
+The minimal IAM Role for these Lambda Functions is:
+
+```
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": [
+                "logs:CreateLogGroup",
+                "logs:CreateLogStream",
+                "logs:PutLogEvents",
+                "ec2:DescribeInstances",
+                "ec2:DescribeVolumes",
+                "ec2:CreateSnapshot",
+                "ec2:CreateTags",
+                "ec2:CopySnapshot",
+                "ec2:DescribeSnapshots",
+                "ec2:DeleteSnapshot"
+            ],
+            "Resource": "*"
+        }
+    ]
+}
+```
+
 ## Files:
 
 Each file implements a single AWS Lambda function.
 
 - ebs-snapshot-creator.py
+- ebs-snapshot-offsiter.py
 - ebs-snapshot-manager.py
 
 ## Related: