Feat/dune

[alexshchur]

• makes the process of fetching transactions potentially eligible for gas refunds rely on Dune instead of Covalent and Subgraph
• adds axios-curlize to make it more obvious what API calls are made during script execution
• adds a draft retry logic (TBC: use default ethers logic) to prevent the script halting too often during development
• adds script that generates dune query (TBC: there should be a better approach, some engine / API / templating lib?)
• adds script for indexing dune transactions pulled from API into a local DB

How to test:

1. generate dune query

yarn gas-refund:generate-dune-query

2. fetch the result of that query into a local file

curl  -X GET \
  'https://api.dune.com/api/v1/query/{queryId}/results' \
  --header 'x-dune-api-key: {API_KEY}' > dune-transactions.json

3. index Dune transactions into the DB

cat dune-transactions.json | yarn gas-refund:index-dune-transactions

4. delete all transactions of the 38 epoch:

delete from "GasRefundTransactions" where epoch >= 38

5. reindex GasRefundTransactions with Dune data:

yarn gas-refund:dev:compute-gas-refund-save-db

6. generate roots for epoch 38:

yarn gas-refund:compute-merkle-tree-save-file

The roots would match the published one

Current state:

• it generates correct roots for the previous distributions, so presumably works correctly
• indexing one epoch takes 300 seconds with Dune (vs > 30 mins previously)
• it took ~8.5K dune credits to fetch the data (on the basic / free plan)

Details on credits calc

data amount for prev epoch is: 24 columns x 300K rows ~= 7.5 mil datapoints
which takes ~7500 credits. The "Plus" plan allows 25,000 credits

[mwamedacen]

minor but I'd be strict on one of the side to prevent double spending across epochs

[alexshchur]

just faced a duplicate, so that is what would happen instead of double spending, will add fix