Branch	Status
master
qa
production

Fedora Ingest Rails

This is a Rails port of the Java application FedoraIngest.

It has an endpoint that MMS hits (with an items' UUIDs as a parameter). It records the UUID in an internal database.

Then (via DelayedJob):

• Iterates through those UUIDS and asks MMS for the latest information.

  • Asks other services (like Filestore DB) for more information about the item.

• Turns around and posts that information to Fedora & RELS-EXT Solr

This decouples MMS from direct communication with Fedora in the event of Fedora API changes or downtime.

Installing & Running

This application uses docker-compose.yml to for most of what it needs. As time goes on, we'll Dockerize more dependencies and have docker-compose be one-stop shopping for running locally. You can edit code as on your machine and expect it to hot-reload like you usually would. Forget Docker is there.

Setup

1. Clone this repo.
2. Clone NYPL/fedoracommons-3.4.2-dockerized & NYPL/filestore_databases_docker in the directory above this. (make them siblings of this app)
3. In this app's root directory cp ./.env.example ./.env and fill it out. (See directions in .env.example)

Setting Up Databases (first run)

1. Run docker-compose up filestore-db postgres, wait, let the databases be created, and synched/mounted to ./database-data. The output will slow down after ~30 seconds.
2. Now, in another terminal run docker-compose run webapp, this will create the database and run the migrations.
3. Once the migrations end you can crtl-z and stop the services

Running

docker-compose up --scale worker=2

What Does Compose Spin Up?

It brings up the following services:

The App Itself

The app reachable at http://localhost:3000. It also spins up 2 workers.

PostgreSQL

The app's database persists in ./database-data/postgres of your machine.

Fedora

Our dockerized Fedora instance reachable at http://localhost:8080.

Filestore Databases

The app's database persists in ./database-data/postgres of your machine.

It brings up the moving & still image MySQL filestore databases. Change your .env file if you want to connect to a remote filestore.

Testing

Run tests through docker-compose:

docker-compose run webapp /bin/bash -c "cd /home/app/fedora_ingest_rails && bundle exec rspec"

Git Workflow & Deployment

Our branches (in order or stability are):

Branch	Environment	AWS Account
master	none	none
qa	qa	nypl-dams-dev
production	production	nypl-dams-prod

Cutting A Feature Branch

1. Feature branches are cut from master.
2. Once the feature branch is ready to be merged, file a pull request of the branch into master.
3. We 'promote' branches by merging from the less mature branch to the more mature branch. (master => qa => production)

Deploying

We use Travis for continuous deployment. Merging to certain branches automatically deploys to the environment associated to that branch.

Merge from	Into	Deploys to (after tests pass)
master	qa	qa env
qa	production	production env

For insight into how CD works look at .travis.yml and the provisioning/travis_ci_and_cd directory. The approach is inspired by this blog post (google cached version).

Amazon & ECS Deployment Configuration

See Amazon And ECS.

Debugging

You may want to start a rails console or hit an endpoint for debugging purposes.
See the debugging documentation.