Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transaction.flush() report #1315

Open
3commascapital opened this issue Dec 6, 2024 · 2 comments
Open

Transaction.flush() report #1315

3commascapital opened this issue Dec 6, 2024 · 2 comments

Comments

@3commascapital
Copy link
Contributor

9:09:59 AM INFO  app        Indexed 173 events with 100% complete and 4s remaining
9:10:00 AM INFO  app        Indexed 250 events with 100% complete and 2s remaining
9:10:01 AM INFO  app        Indexed 215 events with 100% complete and 0ms remaining
9:10:01 AM INFO  app        Indexed 18 events with 100% complete and 0ms remaining
9:12:02 AM ERROR indexing   Internal error occurred while flushing cache. Please report this error here: https://github.com/ponder-sh/ponder/issues
9:12:02 AM WARN  user       Failed 'Transaction.flush()' database method 
9:12:02 AM ERROR process    Caught unhandledRejection event
FlushError: canceling statement due to statement timeout
    at /node_modules/@ponder/common/src/queue.ts:184:15
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.flush (/node_modules/@ponder/core/src/indexing-store/historical.ts:754:7)
    at start3 (/node_modules/@ponder/core/src/bin/utils/run.ts:210:5)

9:12:02 AM WARN  process    Received unhandledRejection, starting shutdown sequence
9:12:02 AM ERROR process    Caught unhandledRejection event
FlushError: canceling statement due to statement timeout
    at /node_modules/@ponder/common/src/queue.ts:184:15
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.flush (/node_modules/@ponder/core/src/indexing-store/historical.ts:754:7)
    at start3 (/node_modules/@ponder/core/src/bin/utils/run.ts:210:5)

(node:86573) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 1325)
(Use `node --trace-warnings ...` to show where the warning was created)
9:12:02 AM ERROR indexing   Internal error occurred while flushing cache. Please report this error here: https://github.com/ponder-sh/ponder/issues
9:12:02 AM WARN  user       Failed 'Transaction.flush()' database method 
9:12:07 AM FATAL process    Failed to shutdown within 5 seconds, terminating (exit code 1)
error: script "start" exited with code 1
@typedarray
Copy link
Collaborator

  1. Which version of @ponder/core ?
  2. What database are you using (e.g. local, hosted, etc)? Possible that the database resources are insufficient for the volume of data you're indexing (needs to be scaled vertically).

@crisog
Copy link

crisog commented Dec 12, 2024

  1. Which version of @ponder/core ?
  2. What database are you using (e.g. local, hosted, etc)? Possible that the database resources are insufficient for the volume of data you're indexing (needs to be scaled vertically).

This happened to me too, I was using a max 8GB RAM postgres server on Railway - I've now upgraded to see what happens.

image

The above image is the memory pattern over 7d, the indexer itself restarted multiple times when it couldn't flush.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants