AICTE scraper

Overview

Simple Scrap the Aicte data for the aicte project

End Points

initiatives(get) = http://localhost:1234/api/aicte/initiatives

Getting Started

Clone the repo:

git clone [email protected]:xeoneux/TypeMonPress.git
cd TypeMonPress

Install dependencies:

npm install

Set environment (vars):

cp .env.example .env

Start server:

# Start server
npm start

# Start debug server
npm run debug

Deployment:

# Compile to JS
1. npm run build

# Upload dist/ to your server
2. scp -rp dist/ user@dest:/path

# Install production dependencies only
3. npm install --production

# Use any process manager to start your services
4. pm2 start dist/index.js

In production you need to make sure your server is always up so you should ideally use any of the process manager recommended here.

We recommend PM2 as it has several useful features like it can be configured to auto-start your services if system is rebooted.

Logging

Universal logging library winston is used for logging. It has support for multiple transports. A transport is essentially a storage device for your logs. Each instance of a winston logger can have multiple transports configured at different levels. For example, one may want error logs to be stored in a persistent remote location (like a database), but all logs output to the console or a local file. We just log to the console for simplicity, you can configure more transports as per your requirement.

Contributing

Contributions, questions and comments are all welcome and encouraged. For code contributions submit a pull request with unit test.