Backend Challenge Solution - Manan Habib #23
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # See https://help.github.com/ignore-files/ for more about ignoring files. | ||
| # dependencies | ||
| /node_modules | ||
| /.idea | ||
| /dist | ||
| /build | ||
| /.terraform | ||
| # misc | ||
| .DS_Store | ||
| .env* | ||
|
|
||
| npm-debug.log* | ||
| yarn-debug.log* | ||
| yarn-error.log* | ||
| .env | ||
| build | ||
| iac/deployment/test/terraform.tfstate | ||
| .terraform.lock.hcl | ||
| terraform.tfstate.backup | ||
| terraform.tfstate | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,86 +1,183 @@ | ||
| # Superformula Cloud Backend Test - Manan Habib | ||
|
|
||
| ## Table of contents | ||
|
|
||
| - [Technical stack](#tech-stack) | ||
| - [Architecture](#arch) | ||
| - [Project Structure](#proj-struct) | ||
| - [Environment Variables](#env-vars) | ||
| - [Building and running](#build-run) | ||
| - [API Reference](#api-ref) | ||
| - [Implementation Details](#impl-details) | ||
| - [Demo](#demo) | ||
|
|
||
| <a name="tech-stack"></a> | ||
| ## Technical stack | ||
|
|
||
| - [**Node.js**](https://nodejs.org/en/) as runtime | ||
| - [**Typescript**](https://www.typescriptlang.org/) as programming language | ||
| - [**API Gateway**](https://aws.amazon.com/api-gateway/) to expose API | ||
| - [**AWS Lambda**](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) as compute service | ||
| - [**AWS CloudWatch**](https://aws.amazon.com/cloudwatch) for logging | ||
| - [**GraphQL**](https://graphql.org/) as specification to develop API using **Apollo Server** | ||
| - [**Terraform**](https://www.terraform.io/) as IaC | ||
|
|
||
| <a name="arch"></a> | ||
| ## Architecture | ||
| As the image shows, we have serverless architecture of this project built upon amazon web services. | ||
| the first entry point for request is **Api Gateway** which redirects the request to corresponding lambda | ||
| function. After gateway, **AWS Lambda**, gets the request and this is the actual component where request gets | ||
| processed and response is generated for user. It is the building block of serverless architecture where on each | ||
| request a lambda gets triggered, serves the response and gets killed. While process of request in lambda, all the | ||
| logs and traces being generated get saved in **AWS Cloudwatch service**. | ||
|
|
||
| To get the coordinates against given address, **Google geocoding api** is being used. Lambda function directly communicates | ||
| with this service to fetch the location data. | ||
|
|
||
|  | ||
|
|
||
| <a name="proj-struct"></a> | ||
| ## Project Structure | ||
| Following is the directories arrangememnt in folder: | ||
| ``` | ||
| . | ||
| ├───coverage # Code coverage report. Jest produce results in this folder. | ||
| ├───iac # All the terraform files reside in this directory | ||
| │ ├───aws | ||
| │ ├───api_gateway # Contains .terraform files for api gateway | ||
| │ └───lambda # Contains tf files for aws lambda and its role | ||
| ├───src | ||
| │ ├───configs # Implementation around env files and global configs | ||
| │ ├───dataSources # Apollo server data sources implementation to wire requests to services | ||
| │ ├───resolvers # Graphql Query/Mutation resolvers implementation | ||
| │ │ └───coordinates | ||
| │ ├───schema # Implementation around graphql schema | ||
| │ ├───services # Contains services implementation which makes core business logic of app | ||
| │ └───utils # Contains helper/reusable functions | ||
| └───tests | ||
| ├───e2e # Contains End to end tests | ||
| └───unit # Contains unit tests | ||
|
|
||
| ``` | ||
| <a name="env-vars"></a> | ||
| ## Environment Variables | ||
| Following are the important environment variables to be set to run the application. For the scope of this project | ||
| I have added env files for dev, test and aws environments having variables set in it. (Although, env files shouldn't | ||
| be part of git repo and deployments variable should be in terraform.) | ||
| ``` | ||
| NODE_ENV # Environment of the app | ||
| GOOGLE_MAPS_KEY # Api key for maps api | ||
| LOG_LEVEL # Log level of the application | ||
|
|
||
| ``` | ||
| <a name="build-run"></a> | ||
| ## Building and running | ||
|
|
||
| #### Pre-Reqs | ||
| This project was developed on windows machine but a *package.linux.json* file is created for linux users. You just need to rename the file to *package.json* and delete or rename current one(which is for windows). Before building and running the project, you are supposed to install some 7zip cli tool and terraform. Use following commands to install 7zip: | ||
| ``` | ||
| sudo apt install p7zip-full | ||
| ``` | ||
| To install terraform use this [link](https://learn.hashicorp.com/tutorials/terraform/install-cli). After installing, initiate it using following command in project's root directory | ||
| ``` | ||
| terraform init | ||
| ``` | ||
| Set the following variable values in *./main.tf* | ||
| ``` | ||
| aws_access_key #AWS access key | ||
| aws_secret_key #AWS secret sey | ||
| ``` | ||
|
|
||
| To manage the build and ship, some npm commands have been implemented in *package.json* file. | ||
| #### Install required packages | ||
| In order to install required packages, run following command in project root directory: | ||
| ``` | ||
| npm install | ||
| ``` | ||
| This will generate assets in *./build* folder. | ||
| #### Build | ||
| In order to build the project and generate assets for deployment, run: | ||
| ``` | ||
| npm run build | ||
| ``` | ||
| This will generate assets in *./build* folder. | ||
|
|
||
| #### Run tests | ||
| In order to run the test and generate coverage report: | ||
| ``` | ||
| npm test | ||
| ``` | ||
| This will generate CLI report as well as report assets in *./coverage* folder. | ||
|
|
||
| #### Run locally | ||
| In order to run the app locally, hit: | ||
| ``` | ||
| npm start | ||
| ``` | ||
| This will host application locally on default 4000 port. | ||
| #### Deploy to aws | ||
| In order to run the app locally, hit: | ||
| ``` | ||
| npm run aws-deploy | ||
| ``` | ||
| This command will automate the whole building and shipping process using the CLI. On running this, | ||
| it will build the assets, run all the test suites, generate a coverage report, create the zipped dist package, | ||
| create/update the terraform plan and apply it to deploy it to aws. Upon successful deplpyment, you | ||
| shoudl be seeing lambda url in CLI output. | ||
|
|
||
| <a name="api-ref"></a> | ||
| ## API Reference | ||
|
|
||
| #### Get location coordinates | ||
|
|
||
| ``` Sample request | ||
| curl -X POST \ | ||
| http://localhost:4000/ \ | ||
| -H 'cache-control: no-cache' \ | ||
| -H 'content-type: application/json' \ | ||
| -H 'postman-token: 3e3c3fa8-b827-3b61-ced1-8e3a12f229ca' \ | ||
| -d '{"query":"query {\n getCoordinates(address: \"New York\"){\n ... on Coordinates{\n latitude\n longitude\n }\n ... on Error{\n code\n message\n }\n }\n \n \n}\n\n\n","variables":{"address":null}}' | ||
| ``` | ||
|
|
||
| | Parameter | Type | Description | | ||
| | :-------- | :------- | :------------------------- | | ||
| | `address` | `NonEmptyString` | **Required**. Arbitrary address to know the coordinates of | | ||
|
|
||
| #### Response | ||
|
|
||
| ```http | ||
| { | ||
| latitute: value, | ||
| longitude: value | ||
| } | ||
|
|
||
| //in case of error | ||
|
|
||
| { | ||
| code: <errorCOde> | ||
| message: <errorMessage> | ||
| } | ||
| ``` | ||
| <a name="impl-details"></a> | ||
| ## Implementation Details | ||
|
|
||
| - Most important aspect of implementation is that it isusing Graphql for API specification. Following are some details of how components are designed. | ||
| - **Schema** | ||
| - Schema is defined in .graphql file turned into *typedefs* by reading the schema file on runtime. | ||
| - A union of type `Coordinates` and `Error` is implemented to have clean error handling in business layer. | ||
| - There's a custom type being implemented to have validation of non empty address on first level of request. | ||
| **Note**: We could have add validation in business layer as well instead of implementing custom type but given the | ||
| fact that app is serverless and cost would be based on compute time, its intutive if we reject the request on schema validation level rather | ||
| than let it to go to business layer and have further compute. | ||
| - **Resolvers** | ||
| - Resolvers are organized on the basis of schema structure. Currently, resolvers just get the request and wire it don to data sources layer. No business logic is exposed in this layer. | ||
| - **Data Sources** | ||
| - This is a simple layer which uses dependency container to resolve the service component and make it available for resolver layer to call. | ||
| - **Services** | ||
| - Servcice layer is where all the business logic is implemented. Dependency container from `typedi` is being incorporated in this layer to make services implementation injectable and resolvable without hassle of managing the lifecycle of objects. Dependencies of each service class is also injected in DI container at start of app to be available to consume with in service. This makes whole solution extensible as well as testable. | ||
|
|
||
| <a name="demo"></a> | ||
| ## Demo | ||
|
|
||
| Link to the demo video: [Video Link](https://youtu.be/pToCMUI9dq8) | ||
| Link to lambda function: [Lambda Link](https://5puw69eq20.execute-api.us-east-1.amazonaws.com/staging/graphql) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| NODE_ENV="aws" | ||
|
|
||
| # Google Geocode API | ||
| GOOGLE_MAPS_KEY='' | ||
|
|
||
| # Debug | ||
| LOG_LEVEL='error' | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
|
There was a problem hiding this comment. What was the reason you decided to push the coverage folder? There was a problem hiding this comment. There is no major reason. I just had one thought to let reviewer quickly go over the html report(./coverage/lcov-report/index.html) that was generated before my push. |
||
| <coverage generated="1660173086173" clover="3.2.0"> | ||
| <project timestamp="1660173086173" name="All files"> | ||
| <metrics statements="56" coveredstatements="56" conditionals="11" coveredconditionals="11" methods="7" coveredmethods="7" elements="74" coveredelements="74" complexity="0" loc="56" ncloc="56" packages="6" files="7" classes="7"/> | ||
| <package name="dataSources"> | ||
| <metrics statements="3" coveredstatements="3" conditionals="0" coveredconditionals="0" methods="1" coveredmethods="1"/> | ||
| <file name="datasources.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\dataSources\datasources.ts"> | ||
| <metrics statements="3" coveredstatements="3" conditionals="0" coveredconditionals="0" methods="1" coveredmethods="1"/> | ||
| <line num="1" count="1" type="stmt"/> | ||
| <line num="2" count="1" type="stmt"/> | ||
| <line num="4" count="2" type="stmt"/> | ||
| </file> | ||
| </package> | ||
| <package name="resolvers"> | ||
| <metrics statements="6" coveredstatements="6" conditionals="2" coveredconditionals="2" methods="1" coveredmethods="1"/> | ||
| <file name="resolvers.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\resolvers\resolvers.ts"> | ||
| <metrics statements="6" coveredstatements="6" conditionals="2" coveredconditionals="2" methods="1" coveredmethods="1"/> | ||
| <line num="1" count="1" type="stmt"/> | ||
| <line num="2" count="1" type="stmt"/> | ||
| <line num="4" count="1" type="stmt"/> | ||
| <line num="6" count="2" type="cond" truecount="2" falsecount="0"/> | ||
| <line num="12" count="1" type="stmt"/> | ||
| <line num="14" count="1" type="stmt"/> | ||
| </file> | ||
| </package> | ||
| <package name="resolvers.coordinates"> | ||
| <metrics statements="2" coveredstatements="2" conditionals="0" coveredconditionals="0" methods="1" coveredmethods="1"/> | ||
| <file name="query.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\resolvers\coordinates\query.ts"> | ||
| <metrics statements="2" coveredstatements="2" conditionals="0" coveredconditionals="0" methods="1" coveredmethods="1"/> | ||
| <line num="1" count="1" type="stmt"/> | ||
| <line num="3" count="2" type="stmt"/> | ||
| </file> | ||
| </package> | ||
| <package name="schema"> | ||
| <metrics statements="17" coveredstatements="17" conditionals="4" coveredconditionals="4" methods="2" coveredmethods="2"/> | ||
| <file name="customScalars.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\schema\customScalars.ts"> | ||
| <metrics statements="12" coveredstatements="12" conditionals="4" coveredconditionals="4" methods="2" coveredmethods="2"/> | ||
| <line num="1" count="2" type="stmt"/> | ||
| <line num="4" count="2" type="stmt"/> | ||
| <line num="8" count="3" type="cond" truecount="1" falsecount="0"/> | ||
| <line num="9" count="1" type="stmt"/> | ||
| <line num="11" count="2" type="cond" truecount="1" falsecount="0"/> | ||
| <line num="12" count="1" type="stmt"/> | ||
| <line num="15" count="1" type="stmt"/> | ||
| <line num="18" count="7" type="cond" truecount="1" falsecount="0"/> | ||
| <line num="19" count="1" type="stmt"/> | ||
| <line num="21" count="6" type="cond" truecount="1" falsecount="0"/> | ||
| <line num="22" count="1" type="stmt"/> | ||
| <line num="25" count="5" type="stmt"/> | ||
| </file> | ||
| <file name="schema.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\schema\schema.ts"> | ||
| <metrics statements="5" coveredstatements="5" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/> | ||
| <line num="1" count="1" type="stmt"/> | ||
| <line num="2" count="1" type="stmt"/> | ||
| <line num="3" count="1" type="stmt"/> | ||
| <line num="5" count="1" type="stmt"/> | ||
| <line num="6" count="1" type="stmt"/> | ||
| </file> | ||
| </package> | ||
| <package name="services"> | ||
| <metrics statements="20" coveredstatements="20" conditionals="5" coveredconditionals="5" methods="2" coveredmethods="2"/> | ||
| <file name="locationService.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\services\locationService.ts"> | ||
| <metrics statements="20" coveredstatements="20" conditionals="5" coveredconditionals="5" methods="2" coveredmethods="2"/> | ||
| <line num="1" count="2" type="stmt"/> | ||
| <line num="2" count="2" type="stmt"/> | ||
| <line num="3" count="2" type="stmt"/> | ||
| <line num="5" count="2" type="stmt"/> | ||
| <line num="6" count="2" type="stmt"/> | ||
| <line num="9" count="2" type="stmt"/> | ||
| <line num="10" count="5" type="stmt"/> | ||
| <line num="11" count="5" type="stmt"/> | ||
| <line num="15" count="6" type="stmt"/> | ||
| <line num="16" count="6" type="stmt"/> | ||
| <line num="17" count="6" type="stmt"/> | ||
| <line num="22" count="6" type="stmt"/> | ||
| <line num="24" count="6" type="stmt"/> | ||
| <line num="26" count="6" type="stmt"/> | ||
| <line num="28" count="6" type="cond" truecount="5" falsecount="0"/> | ||
| <line num="29" count="2" type="stmt"/> | ||
| <line num="31" count="2" type="stmt"/> | ||
| <line num="33" count="2" type="stmt"/> | ||
| <line num="39" count="4" type="stmt"/> | ||
| <line num="41" count="4" type="stmt"/> | ||
| </file> | ||
| </package> | ||
| <package name="utils"> | ||
| <metrics statements="8" coveredstatements="8" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/> | ||
| <file name="helpers.ts" path="E:\Dev\Temp Repos\Superformula - Nodejs Challenge\cloud-backend-test\src\utils\helpers.ts"> | ||
| <metrics statements="8" coveredstatements="8" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/> | ||
| <line num="1" count="1" type="stmt"/> | ||
| <line num="2" count="1" type="stmt"/> | ||
| <line num="4" count="1" type="stmt"/> | ||
| <line num="5" count="1" type="stmt"/> | ||
| <line num="6" count="1" type="stmt"/> | ||
| <line num="7" count="1" type="stmt"/> | ||
| <line num="8" count="1" type="stmt"/> | ||
| <line num="11" count="1" type="stmt"/> | ||
| </file> | ||
| </package> | ||
| </project> | ||
| </coverage> | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Help me understand why you added just
errorlogs to the AWS lambda in production. What is your opinion aboutINFO,WARN,DEBUG, and TRACE logs in production?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well there was no specific reason to set level to 'error' in aws env file. I was just testing all the levels with cloudwatch and ended up on error. To answer your question that what should we be doing in production then under normal circumstances production log level should be 'Info'. As far as the different log levels are concerned:
Info: These logs gives useful information about the system in general. This is sort of information which one don't really care about under normal circumstances but very helpful in crisis situation.
WARN: Gives you info about anything that has a potential to cause error OR fatal exception. Usually we have mechanisms for auto-recovery(or system gets recovered on its own sometimes) with problems logged on this level but this keeps us alerted that what can go wrong in future.
Debug/Trace: I would like to address debug and trace in same part. There is always a debate between level of these log levels in hierarchy but in my experience, debug < trace. This is the level which we only turn on in the situation of diagnoses. If we consider debug < trace side, then debug logs fine grained information(but less granular than Trace) about operation being performed while trace level gives us the most fine grained info(even step by step sometimes) about the operation. We only enable it to diagnose the things in production because it has a performance impact on system.
Note: I assumed that we are discussing log TRACE level and not distributed tracing in terms of microservices.