From 6a411a23c04d386e1a116285db699fb079b7fbe0 Mon Sep 17 00:00:00 2001 From: Nicolas Vargas Date: Thu, 21 Dec 2023 19:49:48 -0300 Subject: [PATCH] refactor: documentation --- README.md | 338 ++++++++++++++++---------------------------- package.json | 2 +- public/swagger.json | 2 +- 3 files changed, 122 insertions(+), 220 deletions(-) diff --git a/README.md b/README.md index 38e57b42..a6d32ddd 100644 --- a/README.md +++ b/README.md @@ -1,244 +1,146 @@ -# Rsk explorer api +# Rsk Explorer API + +# Requisites +- postgres +- node: v16+ +- access to JSON/RPC interface of a rskj node >= 2.0.1 with this modules enabled: eth, net, web3, txpool, debug and trace. + +# Configuration steps + +## Section 1: Environment setup +- create a database 'explorer_db' +- create sql tables using the script prisma/rsk-explorer-database.sql +- Install pm2: + - npm install -g pm2 + - Enable pm2 log rotation: pm2 install pm2-logrotate + - Enable log-rotation compression: pm2 set pm2-logrotate:compress true + - For more options: + - see [pm2-logrotate](https://pm2.keymetrics.io/docs/usage/log-management/#pm2-logrotate-module) + - see [pm2-logrotate configuration](https://github.com/keymetrics/pm2-logrotate#configure) to set the rotation options +- Configure database credentials, node urls, etc, in src/lib/defaultConfig.js file: -## Description - - Rsk blockchain explorer - -## Components - -### API server - -HTTP/WS server, see the documentation [here](doc/api.md). - -### Blocks services - - Imports blockchain data from rsk node to DB. - -- **blocksRouter:** routes messages between services *REQUIRED* -- **blocksListener:** listens to new blocks from node and announces them as service event. -- **blocksRequester:** requests blocks from node. -- **blocksChecker:** checks database for missing blocks and reorgs, emits missing/bad blocks. -- **txPool:** listens to node transactions pool and stores changes in the DB. -- **blocksBalances:** gets historical addresses balances and stores in the DB. -- **blocksStatus:** checks DB and node status and stores changes in the DB. -- **blocksStats:** gets BC stats and stores in the DB. - -### API user events service - - Allows to update fields on the fly and send async response to clients. - Required by the contract verification module. - -## Requisites - -- mongodb > 4 -- node >= 12.18.2 -- access to JSON/RPC interface of a rskj node >= 2.0.1 - with this modules enabled: eth, net, web3, txpool, debug and trace. - -## Install - -- Install dependencies - -``` shell - npm install - ``` - -## Create log dir - -```shell -sudo mkdir /var/log/rsk-explorer -sudo chown $USER /var/log/rsk-explorer/ -chmod 755 /var/log/rsk-explorer -``` - -Note: You can change the log folder in config.json - -## Configuration file - -(optional) - -``` shell - cp config-example.json config.json - ``` - -see [configuration](#configuration) - -## Start - -### Services - -Services can be started manually one by one, but we recommend to use [PM2](https://github.com/Unitech/pm2). -This repo includes a pm2 ecosystem file that starts all services automatically. - -#### Install PM2 - -``` shell - npm install -g pm2 -``` - -To enable pm2 log rotation - -``` shell - pm2 install pm2-logrotate -``` - -see [pm2-logrotate](https://pm2.keymetrics.io/docs/usage/log-management/#pm2-logrotate-module) -see [pm2-logrotate configuration](https://github.com/keymetrics/pm2-logrotate#configure) to set the rotation options - -e.g: - -```shell - -pm2 set pm2-logrotate:compress true - -``` - -#### Start services - -```shell -pm2 start dist/services/blocks.config.js -``` - -### API - -``` shell - pm2 start dist/api/api.config.js -``` - -## Show PM2 logs in pretty format - -All tasks - -```shell - -:~/rsk-explorer-api$ pm2 log --raw | npx bunyan - -``` - -One task - -```shell - -:~/rsk-explorer-api$ pm2 log blocksListener --raw | npx bunyan - -``` - -## Commands - -Run api in development mode - -``` shell - npm run dev - ``` - -Run blocks service in development mode - -``` shell - npm run blocks - ``` - -Production build to ./dist folder - -``` shell - npm run build - ``` - -## Configuration - - **config.json** - See defaults on: **lib/defaultConfig** - *(config.json overrides this values)* - - Use: - - ```shell - node dist/tools/showConfig.js - ``` - - to check current configuration - -**Configuration Example:** - -``` javascript - "source": { - "node": "localhost", - "port": 4444 +```javascript +{ + source: { + protocol: 'http', + node: 'localhost', + port: 4444, + url: null + }, + sourceRoutes: { // Nod3Router routes, used as default when source is an array of sources + subscribe: 0, // delegates subscriptions to the first node + rsk: 0, // delegates rsk module to the node that handle subscriptions + trace: 1 // delegates trace_ module to the second node }, - "api": { - "address": "localhost", - "port": 3003 + db: { + protocol: 'postgres://', + databaseName: 'explorer_db', + host: 'localhost', + port: 5432, + user: 'postgres', + password: 12345678 }, - "db": { - "server": "localhost", - "port": 27017, - "database": "blockDB" - } + api: { + address: 'localhost', + port: 3003, + lastBlocks: 30, + MIN_LIMIT: 10, + LIMIT: 50, + MAX_LIMIT: 500, + MAX_PAGES: 10, + allowUserEvents: false, + exposeDoc: false, + // All modules are enabled as default + modules: setAllModules(true), + delayedFields, + allowCountQueries: false + }, + blocks: { + blocksQueueSize: 10, + bcTipSize: 120, + batchRequestSize: 20, + debug: false, + ports: [3010], // list of services ports, if the list runs out, the services will try to take the next ports starting from the last + address: '127.0.0.1', + services + }, + forceSaveBcStats: true, + enableTxPoolFromApi: true +} +``` - ``` +### api +- **address** [string] api server bind address +- **port** [number] api server port +- **exposeDoc** [boolean]: serve rsk-openapi-ui on /doc to render swagger.json +- **allowUserEvents** [boolean]: allow contractVerifier + +### To enable contract verifier module: The contractVerifier module requires a connection to a [rsk-contract-verifier](https://github.com/rsksmart/rsk-contract-verifier) instance. The url must be provided on api section: +- set api.allowUserEvents to true +- add the contract verifier url inside api: ```javascript -"api":{ - "contractVerifier": { - "url": "ws://localhost:3008" - } +api:{ + //... other configs, + contractVerifier: { + url: 'ws://localhost:3008' + }, + //... other configs } - ``` -### Source - - Address of rskj node or array of addresses of rskj nodes +## Section 3: Build process +Run commands: +- npm install +- npm run build +- npx prisma generate -e.g.: - -```json -{ - "url":"http://localhost:4444" -} +## Start +- Start block service: npm run start-blocks +- Start api: npm run start-api -``` +Note: an rsk node must be running in port or url specified in defaultConfig.js. Otherwise, blocks service will crash. -e.g: +## Section 4: Logs +To see block service logs: +- npm run blocks-logs-raw (production) +- npm run blocks-logs-pretty (development) -```json -[ - { "url":"http://localhost:4444" }, - { "url":"http://othernode:4444" } -] +To see api logs: +- npm run api-logs-raw (production) +- npm run api-logs-pretty (development) -``` +## Section 5: Tools -### db +Get a block: +- node dist/tools/getBlock.js 4000 (print in console) +- node dist/tools/getBlock.js 4000 --save (print in console and also store in database) -- **server**": "localhost" -- **port**": 27017 -- **database**: "explorerDB" +Get missing segments in database: +- node dist/tools/missingSegments.js -**Optionals:** +## Components -- **user**: < user > -- **password**: < password > +The logic in charge of indexing the blockchain in the database and maintaining data integrity consists of 4 services: +- [blocks-checker-service]: ensures integrity of the most recent 120 blocks in database at the moment of running the service +- [savetip-service]: requests blocks inside the tip size threshold (the top 120 newest, starting from latest block) +- [live-syncer-service]: requests new blocks to the RSK node, starting from latest block +- [static-syncer-service]: Stores immutable blocks (starting from latest - 120). To do so, it checks database gaps between already indexed blocks, and requests the missing ones by iterating the gaps -### blocks - -- **validateCollections** :[Boolean] Validate collections at blocks service start, default false -- **blocksQueueSize**:[Number] blocksRequester queue size -- **bcTipSize**:[Number] Number of confirmations required to check blocks -- **debug**:[Boolean] Enable logging of nod3 requests, default false -- **services**:[Object] {ServiceName:[Boolean]}. All services are enabled as default, to disable a service use: **serviceName:false** +API server: HTTP/WS server +- API Documentation: [docs](doc/api.md) +- Swagger docs: [open api specification](public/swagger.json) -### api +## Development -- **address** [string] api server bind address -- **port** [number] api server port +Run api in development mode: npm run dev +Run blocks in development mode: -- **allowUserEvents** [boolean]: enable/disable userEventsApi -- **exposeDoc** [boolean]: serve rsk-openapi-ui on /doc to render swagger.json +- npm run build +- npm prisma generate +- npm run blocks-start -## Documentation - -- [api documentation](doc/api.md) -- [open api specification](public/swagger.json) +Note Before uploading changes, remember to execute npm run build after upgrading version in package.version, so swagger docs compile the version number too. \ No newline at end of file diff --git a/package.json b/package.json index b40a41dd..820f5bc3 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "rsk-explorer-api", - "version": "2.0.7", + "version": "2.0.8", "description": "", "main": "index.js", "scripts": { diff --git a/public/swagger.json b/public/swagger.json index e8b7ce26..31cce100 100644 --- a/public/swagger.json +++ b/public/swagger.json @@ -1,7 +1,7 @@ { "info": { "title": "rsk-explorer-api", - "version": "2.0.5", + "version": "2.0.8", "description": "explorer API Documentation" }, "swagger": "2.0",