Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make withdrawal volume metrics meter per day volumes #223

Closed
wants to merge 13 commits into from

Conversation

montekki
Copy link
Contributor

@montekki montekki commented Oct 5, 2023

This PR implement the per-day metering of withdrawal volumes, both requested and already finalized. Basically there are two things that have to be done:

  • On service restarts all requested and finalized withdrawals from DB for this day back to 00:00 UTC have to be queried and added back to the metrics
  • As soon as the day turns, on 00:00 UTC next day the metric counters have to be zeroed.

@montekki montekki marked this pull request as ready for review October 5, 2023 19:32
///
/// # Arguments
/// * `at_date_time`: Look for the block at this date-time
/// * `start_from_block`: Will perform search starting from this block,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's always none

/// * `start_from_block`: Will perform search starting from this block,
/// if `None` is specified then searches from block 1.
/// * `middleware`: The client to perform requests to RPC with.
pub async fn get_block_number_by_timestamp<M: Middleware>(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you looking for the block in api? why not in the database?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't save block timestamps to DB

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably you have to

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll simplify the code and you can forget about the old blocks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not simplify the code, there is no info about each miniblock stored + no subscriptions to new l2 blocks, each l2 block is not processed and so on. The only place where they currently exist is l2_blocks table but that only contains effectively block ranges as we get them out of verify, commit and execute events on l1. So, every time i would get such a blockrange I will have to fetch all l2 block headers for thousands of blocks in the blockrange just to get their timestamps to populate this new row + this wouldn't be precise anyway.

OTOH the current solution makes only ~30 RPC requests on startup

self.tokens.insert(w.event.token, (decimals, address));
let mut guard = self.tokens.write().await;
guard.insert(w.event.token, (decimals, address));
drop(guard);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need to drop it manually? it will be dropped by it self just a few lines further

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and old C habit.

Self {
let tokens = Arc::new(RwLock::new(token_decimals));

tokio::spawn(reset_metrics_at_midnight(tokens.clone(), component_name));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure it's a good idea?
I'd rather nullify it once miniblock timestamp changes the day.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so you may be re-syncing after a restart or something similar and you will receive many miniblocks that change the day in a short period of time, or whatver. you get the idea.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a huge problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two points though:

  1. it will may be zeroing the graph during the day
  2. again, as I've mentioned above, there currently is no subscription to l2 miniblocks, and introducing one would probably be the bigger evil.

@RomanBrodetski
Copy link

I know that this is how we collect and represent this data currently (increment during the day and reset at 00:00UTC), but I think it's awkward - we could track the amount withdrawn as a simple counter and show rate() or increase().

We'd not need get_block_number_by_timestamp - we can show the amount per 10min/hour/day and that should provide a better insight than what we have currently IMO - cc @anton-matterlabs

Even if (for any reason) we want to show the graph like we do now (staircase that resets to zero everyday at 00:00), we can achieve it with something like sum_over_time(irate(my_counter[1d])) * 86400 (The irate function calculates the per-second rate of increase of a counter metric. By summing these per-second rates over time, we can achieve the staircase-like effect).

@montekki
Copy link
Contributor Author

Thanks, @RomanBrodetski , Achieved by querying:

label_keep(label_move(uniswap_price_usd, "id", "token"), "token") * label_set(sum_over_time(irate(label_keep(era_withdrawal_finalizer_meter_withdrawals{token="0x0000000000000000000000000000000000000000"}, "token")[15m])) * 900, "token", "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2")
Screenshot 2023-10-10 at 16 11 21

@montekki montekki closed this Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants