Skip to content

Commit

Permalink
indexsupply.com/shovel/docs: improve getting started section
Browse files Browse the repository at this point in the history
  • Loading branch information
ryandotsmith committed Apr 1, 2024
1 parent bd5f123 commit 8466bd6
Showing 1 changed file with 52 additions and 18 deletions.
70 changes: 52 additions & 18 deletions indexsupply.com/shovel/docs/index.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,73 @@
<title>Shovel Docs</title>

Shovel's main thing is indexing Transactions and Events.
Shovel is program that indexes data from an Ethereum node into a Postgres database. It uses standard JSON RPC APIs provided by all Ethereum nodes (ie Geth,Reth) and hosted nodes (ie Alchemy, Quicknode). It indexes blocks, transactions, and decoded event logs. Shovel uses a declarative JSON config to determine what data should be saved in Postgres.

To do this, you will create a JSON config file defining the events and block data that you want to save along with the Postgres table that will store the saved data. With this file, you can start indexing by running Shovel:

```
./shovel -config config.json
```
The Shovel config contains a database URL, an Ethereum node URL, and an array of Integrations that contain a mapping of Ethereum data to Postgres tables.

<details>
<summary>Here is the smallest possible config file</summary>
<summary>
Here is a basic example of a config that saves ERC20 transfers
</summary>

```
{
"pg_url": "postgres:///shovel",
"eth_sources": [{"name": "m", "chain_id": 1, "url": "https://ethereum-rpc.publicnode.com"}],
"eth_sources": [
{
"name": "mainnet",
"chain_id": 1,
"url": "https://ethereum-rpc.publicnode.com"
}
],
"integrations": [{
"name": "small",
"name": "erc20_transfers",
"enabled": true,
"sources": [{"name": "m"}],
"table": {"name": "small", "columns": []},
"block": [],
"event": {}
"sources": [{"name": "mainnet"}],
"table": {
"name": "erc20_transfers",
"columns": [
{"name": "block_num", "type": "numeric"},
{"name": "tx_hash", "type": "bytea"},
{"name": "from", "type": "bytea"},
{"name": "to", "type": "bytea"},
{"name": "value", "type": "bytea"},
]
},
"block": [
{"name": "block_num", "column": "block_num"},
{"name": "tx_hash", "column": "tx_hash"}
],
"event": {
"name": "Transfer",
"type": "event",
"anonymous": false,
"inputs": [
{"indexed": true, "name": "from", "type": "address", "column": "from"},
{"indexed": true, "name": "to", "type": "address", "column": "to"},
{"indexed": false, "name": "value", "type": "uint256", "column": "value"}
]
}
}]
}
```
_This config file works because there are required columns that are added to the table and to the block object array by default._
</details>

It's likely that you will want to index actual data. To do that, you'll need to fill in the `block` and `event` fields in the config object. See the following sections for instructions on how to do that:
In the example config you will notice that we define a PG table named `erc20_transfers` with 5 columns. Shovel will create this table on startup. We specify 2 fields that we want to save from the block and transaction data and we provide the Transfer event from the ERC20 ABI JSON. The Transfer event ABI snippet has an additional key on the input objects named `column`. The `column` field indicates that we want to save the data from this input and references a column named previously defined in `table`.

We can run this config with:

```
./shovel -config config.json
```

This concludes the basic introduction to Shovel. Please read the rest of this page for a comprehensive understanding of Shovel or browse these common topics:

1. [Event](#event)
2. [Block](#block)
- Index data from multiple chains: [Ethereum Sources](#ethereum-sources)
- Index additional Block/Transaction data: [Block](#block)
- Define and Filter indexed data based on the decoded logs: [Event](#event)
- And some basic [examples](#examples)

You can also browse [Examples](#examples) to find one that does what you need.
Best of luck and feel free to reach out to [[email protected]](mailto:[email protected]) with any questions.

<hr>

Expand Down

0 comments on commit 8466bd6

Please sign in to comment.