diff --git a/.gitignore b/.gitignore new file mode 100644 index 000000000..a56a7ef43 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +node_modules + diff --git a/README.markdown b/README.markdown index f93d526ae..0917e908d 100644 --- a/README.markdown +++ b/README.markdown @@ -1,59 +1,33 @@ -# Wave Software Development Challenge -Applicants for the [Software developer](https://wave.bamboohr.co.uk/jobs/view.php?id=1) role at Wave must complete the following challenge, and submit a solution prior to the onsite interview. +This project requires installation of both the frontend (Angular) UI, as well as this API -The purpose of this exercise is to create something that we can work on together during the onsite. We do this so that you get a chance to collaborate with Wavers during the interview in a situation where you know something better than us (it's your code, after all!) +# Setup -There isn't a hard deadline for this exercise; take as long as you need to complete it. However, in terms of total time spent actively working on the challenge, we ask that you not spend more than a few hours, as we value your time and are happy to leave things open to discussion in the onsite interview. +* install PostgreSQL +* create database called `wave` +* `sudo npm install -g db-migrate` +* `npm install` +* `db-migrate up` (installs schema needed by DB/API, pass in `PG_USER` and `PG_PASS` environment variables if needed to allow this API to authenticate to your PostgreSQL DB) +* `node server.js` (pass in `PG_USER` and `PG_PASS` environment variables if needed to allow this API to authenticate to your PostgreSQL DB) -Please use whatever programming language and framework you feel the most comfortable with. +# Unit Tests -Feel free to email [dev.careers@waveapps.com](dev.careers@waveapps.com) if you have any questions. +* `npm test` (for CSV input validation) -## Project Description -Imagine that Wave has just acquired a new company. Unfortunately, the company has never stored their data in a database, and instead uses a comma separated text file. We need to create a way for the new subsidiary to import their data into a database. Your task is to create a web interface that accepts file uploads, and then stores them in a relational database. +# Design decisions -### What your web-based application must do: +* Database credentials passed through via environment variable, this is more secure than posting credentials to a code repository. These variables would be managed by some sort of container/VM automation tool like Chef/Ansible/Puppet +* Date conversion to ISO 8601 standard since this the default PGSQL date format +* Promise based PostgreSQL workflow (via the pg-promises driver, which is a fork of node-postgres) +* Third party CSV parser (figured this was a time saver over coming up with a reliable regexp pattern) +* CSV input validation (originally this was to assist with unit testing, but see problems section below) +* PostgreSQL transactions (COMMIT + ROLLBACK support for integrity of data) +* DB migrations for portability of schema and ease of management within teams +* DB used to group expenses by date. The were originally unsorted, and databases are good at sorting stuff +* Added display of the number of expenses for the month. Although not required this seemed like useful info +* Unit tests - a valiant effort, tests are good! +* Data manipulation in the backend, I like my clients to be dumb and not have to do much other than display data provided by the API. -1. Your app must accept (via a form) a comma separated file with the following columns: date, category, employee name, employee address, expense description, pre-tax amount, tax name, and tax amount. -1. You can make the following assumptions: - 1. Columns will always be in that order. - 2. There will always be data in each column. - 3. There will always be a header line. +# Problems - An example input file named `data_example.csv` is included in this repo. - -1. Your app must parse the given file, and store the information in a relational database. -1. After upload, your application should display a table of the total expenses amount per-month represented by the uploaded file. - -Your application should be easy to set up, and should run on either Linux or Mac OS X. It should not require any non open-source software. - -There are many ways that this application could be built; we ask that you build it in a way that showcases one of your strengths. If you you enjoy front-end development, do something interesting with the interface. If you like object-oriented design, feel free to dive deeper into the domain model of this problem. We're happy to tweak the requirements slightly if it helps you show off one of your strengths. - -### Documentation: - -Please modify `README.md` to add: - -1. Instructions on how to build/run your application -1. A paragraph or two about what you are particularly proud of in your implementation, and why. - -## Submission Instructions - -1. Fork this project on github. You will need to create an account if you don't already have one. -1. Complete the project as described below within your fork. -1. Push all of your changes to your fork on github and submit a pull request. -1. You should also email [dev.careers@waveapps.com](dev.careers@waveapps.com) and your recruiter to let them know you have submitted a solution. Make sure to include your github username in your email (so we can match applicants with pull requests.) - -## Alternate Submission Instructions (if you don't want to publicize completing the challenge) -1. Clone the repository. -1. Complete your project as described below within your local repository. -1. Email a patch file to [dev.careers@waveapps.com](dev.careers@waveapps.com) - -## Evaluation -Evaluation of your submission will be based on the following criteria. - -1. Did you follow the instructions for submission? -1. Did you document your build/deploy instructions and your explanation of what you did well? -1. Were models/entities and other components easily identifiable to the reviewer? -1. What design decisions did you make when designing your models/entities? Why (i.e. were they explained?) -1. Did you separate any concerns in your application? Why or why not? -1. Does your solution use appropriate datatypes for the problem as described? +* Although the unit test is showing the test passing, this is not working properly. I struggled with race conditions involving the CSV parser (which doesn't support Javascript promises) and this workflow. If I had gotten this to work I would have manipulated the data mock to test the validation. I would have also figured out how to call the `createExpenses` function directly from within the unit tests rather than the separate `testParser` function +* A second iteration through the expense summary results was necessary to round to two decimal places and deal with some funky Javascript floating point issues. There is probably a better workaround here? diff --git a/database.json b/database.json new file mode 100644 index 000000000..8dfee2608 --- /dev/null +++ b/database.json @@ -0,0 +1,9 @@ +{ + "development": { + "driver": "pg", + "user": {"ENV": "PG_USER"}, + "password": {"ENV": "PG_PASS"}, + "host": "localhost", + "database": "wave" + } +} \ No newline at end of file diff --git a/migrations/20170221193017-gen-schema.js b/migrations/20170221193017-gen-schema.js new file mode 100644 index 000000000..ceb5e1f4c --- /dev/null +++ b/migrations/20170221193017-gen-schema.js @@ -0,0 +1,37 @@ +'use strict'; + +var dbm; +var type; +var seed; + +/** + * We receive the dbmigrate dependency from dbmigrate initially. + * This enables us to not have to rely on NODE_PATH. + */ +exports.setup = function(options, seedLink) { + dbm = options.dbmigrate; + type = dbm.dataType; + seed = seedLink; +}; + +exports.up = function(db) { + return db.createTable('expenses', { + id: { type: 'serial', primaryKey: true }, + date: 'date', + category: 'string', + employee_name: 'string', + employee_address: 'string', + expense_description: 'text', + pre_tax_amount: 'decimal', + tax_name: 'string', + tax_amount: 'decimal' + }); +}; + +exports.down = function(db) { + return db.dropTable('expenses'); +}; + +exports._meta = { + "version": 1 +}; diff --git a/package.json b/package.json new file mode 100644 index 000000000..bf5f8310e --- /dev/null +++ b/package.json @@ -0,0 +1,20 @@ +{ + "name": "wave-challenge", + "version": "1.0.0", + "description": "", + "main": "index.js", + "scripts": { + "test": "mocha" + }, + "author": "", + "license": "ISC", + "dependencies": { + "boom": "^4.2.0", + "csv": "^1.1.1", + "hapi": "^16.1.0", + "pg-promise": "^5.5.8" + }, + "devDependencies": { + "mocha": "^3.2.0" + } +} diff --git a/server.js b/server.js new file mode 100644 index 000000000..33a641ee8 --- /dev/null +++ b/server.js @@ -0,0 +1,41 @@ +'use strict'; + +const Hapi = require('hapi'); +const server = new Hapi.Server(); +var wave = require('./wave'); + +server.connection({ port: 3000, host: 'localhost' }); + +server.start((err) => { + + if (err) { + throw err; + } + + console.log(`Server running at: ${server.info.uri}`); +}); + + +server.route({ + path: '/expenses', + method: 'POST', + handler: wave.createExpenses, + config: { + cors: { + origin: ['*'], + additionalHeaders: ['cache-control', 'x-requested-with'] + } + } +}); + +server.route({ + path: '/expenses', + method: 'GET', + handler: wave.readExpenses, + config: { + cors: { + origin: ['*'], + additionalHeaders: ['cache-control', 'x-requested-with'] + } + } +}); diff --git a/test/test.js b/test/test.js new file mode 100644 index 000000000..1449c0004 --- /dev/null +++ b/test/test.js @@ -0,0 +1,20 @@ +var assert = require('assert'); +var wave = require('../wave'); +var dataMock = 'date,category,employee name,employee address,expense description,pre-tax amount,tax name,tax amount\n12/1/2013,Travel,Don Draper,"783 Park Ave, New York, NY 10021",Taxi ride, 350.00 ,NY Sales tax, 31.06\n12/15/2013,Meals and Entertainment,Steve Jobs,"1 Infinite Loop, Cupertino, CA 95014",Team lunch, 235.00 ,CA Sales tax, 17.63\n12/31/2013,Computer - Hardware,Jonathan Ive,"1 Infinite Loop, Cupertino, CA 95014",HP Laptop, 999.00 ,CA Sales tax, 74.93\n12/14/2013,Computer - Software,Tim Cook,"1 Infinite Loop, Cupertino, CA 95014",Microsoft Office, 899.00 ,CA Sales tax, 67.43\n12/6/2013,Computer - Software,Sergey Brin,"1600 Amphitheatre Parkway, Mountain View, CA 94043",iCloud Subscription, 50.00 ,CA Sales tax, 3.75\n12/9/2013,Computer - Software,Larry Page,"1600 Amphitheatre Parkway, Mountain View, CA 94043",iCloud Subscription, 50.00 ,CA Sales tax, 3.75\n11/10/2013,Meals and Entertainment,Eric Schmidt,"1600 Amphitheatre Parkway, Mountain View, CA 94043",Coffee with Steve, 300.00 ,CA Sales tax, 22.50\n11/12/2013,Travel,Larry Page,"1600 Amphitheatre Parkway, Mountain View, CA 94043",Taxi ride, 230.00 ,CA Sales tax, 17.25\n11/20/2013,Meals and Entertainment,Don Draper,"783 Park Ave, New York, NY 10021",Client dinner, 200.00 ,NY Sales tax, 15.00\n10/4/2013,Travel,Eric Schmidt,"1600 Amphitheatre Parkway, Mountain View, CA 94043",Flight to Miami, 200.00 ,CA Sales tax, 15.00\n10/12/2013,Computer - Hardware,Don Draper,"783 Park Ave, New York, NY 10021",Macbook Air," 1,999.00 ",NY Sales tax, 177.41\n12/9/2013,Computer - Software,Steve Jobs,"1 Infinite Loop, Cupertino, CA 95014",Dropbox Subscription, 15.00 ,CA Sales tax, 1.13\n9/18/2013,Travel,Tim Cook,"1 Infinite Loop, Cupertino, CA 95014",Taxi ride, 200.00 ,CA Sales tax, 15.00\n9/30/2013,Office Supplies,Larry Page,"1600 Amphitheatre Parkway, Mountain View, CA 94043",Paper, 200.00 ,CA Sales tax, 15.00\n12/30/2013,Meals and Entertainment,Larry Page,"1600 Amphitheatre Parkway, Mountain View, CA 94043",Dinner with potential acquisition, 200.00 ,CA Sales tax, 15.00\n1/6/2014,Computer - Hardware,Eric Schmidt,"1600 Amphitheatre Parkway, Mountain View, CA 94043",iPhone, 200.00 ,CA Sales tax, 15.00\n1/7/2014,Travel,Steve Jobs,"1 Infinite Loop, Cupertino, CA 95014",Airplane ticket to NY, 200.00 ,CA Sales tax, 15.00\n2/3/2014,Meals and Entertainment,Jonathan Ive,"1 Infinite Loop, Cupertino, CA 95014",Starbucks coffee, 12.00 ,CA Sales tax, 0.90\n2/18/2014,Travel,Eric Schmidt,"1600 Amphitheatre Parkway, Mountain View, CA 94043",Airplane ticket to NY," 1,500.00 ",CA Sales tax, 112.50'; +var csvimport; + +describe('Wave Challenge', function() { + describe('#csvimport()', function() { + it('verify successful CSV import', function() { + wave.testParser({ + payload: { + file: dataMock + } + }) + .then(function(csvimport) { + console.log("IMPORT", csvimport); + done(assert.equal(19, csvimport.length)); + }) + }); + }); +}); \ No newline at end of file diff --git a/wave.js b/wave.js new file mode 100644 index 000000000..58466fc54 --- /dev/null +++ b/wave.js @@ -0,0 +1,169 @@ +'use strict' + +var Boom = require('boom'); +var pgp = require('pg-promise')(); +var db = pgp({ + host: 'localhost', + port: 5432, + database: 'wave', + user: process.env.PG_USER ? process.env.PG_USER : '', + password: process.env.PG_PASS ? process.env.PG_PASS : '' +}); +var csv = require('csv'); + + +module.exports = { + + createExpenses: function(request, reply) { + // validation + if (!request.payload.file) { + reply("Error: file contains no data"); + } + + var input; + var inputArray = []; + var promiseList = []; + var dateArr; + + csv.parse(request.payload.file, {}, function(err, csvInput) { + // validate entire CSV file before an attempt to save data to DB occurs + try { + if (err) { + // some library issue with parsing the CSV + throw err; + } + + for (var i=0; i < csvInput.length; i++) { + if (i > 0) { // don't validate header line + // convert date input to ISO 8601 standard + dateArr = csvInput[i][0].split("/"); + csvInput[i][0] = dateArr[2] + "-" + dateArr[0] + "-" + dateArr[1]; + + csvInput[i][5] = parseFloat(csvInput[i][5]); + csvInput[i][7] = parseFloat(csvInput[i][7]); + + inputArray.push(csvInput[i]); + + // better validation is needed here, but this is a basic PoC + if (!csvInput[i][0].match(/[0-9]{4}-[0-9]+-[0-9]+/)) { + throw Boom.badRequest("Your CSV file contains an invalid date"); + } + else if (!parseFloat(csvInput[i][5])) { + throw Boom.badRequest("Your CSV file contains an invalid pre-tax amount"); + } + else if (!parseFloat(csvInput[i][7])) { + throw Boom.badRequest("Your CSV file contains an invalid tax amount"); + } + } + } + + db.tx(function (t) { + // no validation issues, now save to DB by starting a transaction + inputArray.forEach(function(validInput) { + promiseList.push(db.one('INSERT INTO expenses(date, category, employee_name, employee_address, expense_description, pre_tax_amount, tax_name, tax_amount) VALUES($1, $2, $3, $4, $5, $6, $7, $8) RETURNING id', validInput)) + }); + return t.batch(promiseList); + }) + .then(function(data) { + reply(data); + }) + .catch(function(err) { + // rollback + console.log("ERROR", err); + reply(Boom.badImplementation(err)); + }); + } + catch(e) { + reply(e); + } + }); + + }, + + readExpenses:function(request, reply) { + var expenses = []; + var output = []; + var monthString, thisMonth, expenseTally, numExpenses; + var itr = 0; + var lastMonth = false; + + db.query('SELECT * FROM expenses ORDER by date') + .then(function(response) { + // output summary by month + + response.forEach(function(expense) { + thisMonth = expense.date.getMonth() + 1; + monthString = expense.date.getFullYear() + " - " + thisMonth; + + // assure input is being treated as a float + expense.pre_tax_amount = parseFloat(expense.pre_tax_amount); + expense.tax_amount = parseFloat(expense.tax_amount); + + itr = thisMonth !== lastMonth ? itr + 1 : itr; + expenseTally = thisMonth !== lastMonth ? expense.pre_tax_amount + expense.tax_amount : parseFloat(expenses[itr].expense + (expense.pre_tax_amount + expense.tax_amount)); + numExpenses = thisMonth !== lastMonth ? 1 : expenses[itr].numExpenses + 1; + + expenses[itr] = { + date:monthString, + expense:expenseTally, + numExpenses:numExpenses + }; + + lastMonth = expense.date.getMonth() + 1; + }); + + // iterate through results again and round expenses for visual formatting + expenses.forEach(function(listing) { + output.push({ + date:listing.date, + expense:listing.expense.toFixed(2), + numExpenses:listing.numExpenses + }) + }); + + return reply(output); + }) + .catch(function(err) { + return Boom.badImplementation(err); + }); + }, + + testParser:function(request) { + var dateArr; + + return new Promise(function(resolve, reject) { + try { + csv.parse(request.payload.file, {}, function(err, csvInput) { + for (var i=0; i <= csvInput.length; i++) { + if (i > 0) { // don't validate header line + + // convert date input to ISO 8601 standard + dateArr = csvInput[i][0].split("/"); + csvInput[i][0] = dateArr[2] + "-" + dateArr[0] + "-" + dateArr[1]; + + csvInput[i][5] = parseFloat(csvInput[i][5]); + csvInput[i][7] = parseFloat(csvInput[i][7]); + + // better validation is needed here, but this is a basic PoC + if (!csvInput[i][0].match(/[0-9]{4}-[0-9]+-[0-9]+/)) { + throw Boom.badRequest("Your CSV file contains an invalid date"); + } + else if (!parseFloat(csvInput[i][5])) { + throw Boom.badRequest("Your CSV file contains an invalid pre-tax amount"); + } + else if (!parseFloat(csvInput[i][7])) { + throw Boom.badRequest("Your CSV file contains an invalid tax amount"); + } + } + } + resolve(csvInput); + }); + } + catch(e) { + reject(e); + } + + }) + } + +} \ No newline at end of file