Skip to content

Commit

Permalink
wip! Remove credentials from config in favor of environment variables
Browse files Browse the repository at this point in the history
This is more standard (e.g. follows the 12 factor app pattern) and, as
Trevor noted¹, is encouraged by lots of services like Travis CI and
Heroku and EC2.  It also means credentials are not visible in the output
of `ps`, which is important for running on shared hosts.

Unresolved is how to programmatically describe the requirements on the
"envdir" module.  This is an unresolved question for build requirements
in general.²  It may be that we sidestep that by accessing envdir
through a nextstrain.build module that's always required to be
available, as described in a proposal for refactoring shared functions.³

¹ #4
² #3
³ #2
  • Loading branch information
tsibley committed Jul 2, 2018
1 parent 5c4f6f6 commit 2d14707
Show file tree
Hide file tree
Showing 4 changed files with 14 additions and 23 deletions.
14 changes: 4 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,16 +55,10 @@ filter:
This build starts by pulling sequences from our live [fauna][] database (a
RethinkDB instance). For data privacy and security reasons, you'll need to
provide credentials to the database yourself, using a `config_local.yaml`
snippet like so:

```yaml
---
credentials:
rethink:
host: ...
auth_key: ...
```
provide credentials to the database yourself using two files in the `envdir/`
directory: `envdir/RETHINK_HOST` and `envdir/RETHINK_AUTH_KEY`. Each file
should contain the appropriate value on a single line. These files are ignored
by git, so you can't (and shouldn't) commit them.

If you don't have access to our database, you can run the build using the
example data provided in this repository. Before running the build, copy the
Expand Down
10 changes: 5 additions & 5 deletions Snakefile
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ def field_map(fields):
]


# Environment variables
import envdir
envdir.open()


# Config
configfile: "config.yaml"

Expand All @@ -46,14 +51,9 @@ rule download:
build / "data/zika.fasta"
params:
fields = [ f[0] for f in fasta_fields ],

rethink_host = config["credentials"]["rethink"]["host"],
rethink_auth_key = config["credentials"]["rethink"]["auth_key"],
shell:
"""
env PYTHONPATH=../fauna \
RETHINK_HOST={params.rethink_host:q} \
RETHINK_AUTH_KEY={params.rethink_auth_key:q} \
python2 ../fauna/vdb/download.py \
--database vdb \
--virus zika \
Expand Down
8 changes: 0 additions & 8 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,6 @@
# feature definitions to analyze for amino acid changes.
reference: config/zika_outgroup.gb

# Credentials for pulling data from fauna/rethink. These are not provided here
# for data privacy and security reasons. Please provide them in your
# config_local.yaml file.
credentials:
rethink:
host: ""
auth_key: ""

# This defines the fields in the pipe-delimited FASTA header of each sequence.
# It is used both for downloading from fauna and processing in augur. Fields
# may be mapped from one name in fauna to another in augur using "- from: to".
Expand Down
5 changes: 5 additions & 0 deletions envdir/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# This file serves to ignore required-but-sensitive environment variables as
# well as to preserve the containing empty directory in the git repo.

/RETHINK_AUTH_KEY
/RETHINK_HOST

0 comments on commit 2d14707

Please sign in to comment.