Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ooom dev #9

Open
wants to merge 98 commits into
base: feature/TxDAG-PEVM
Choose a base branch
from
Open

Ooom dev #9

wants to merge 98 commits into from

Conversation

sunny2022da
Copy link
Owner

Description

add a description of your changes here...

Rationale

tell us why we need these changes...

Example

add an example CLI or API response...

Changes

Notable changes:

  • add each change in a bullet point here
  • ...

sunny2022da and others added 30 commits September 25, 2024 16:30
This PR implement the Parallel EVM engine

Co-authored-by: setunapo
Co-authored-by: sunny2022da
Co-authored-by: galaio
Co-authored-by: andyzhang2023
Fix an issue of incorrectly set the origin storage at parallel stateDB's
GetState(). Remove this code because it is already solved by lightCopy

PR: #2
rwset: support collect rwset from statedb;
mvstates: support export DAG;
dag: support travel all execution paths;
dag: refactor versioned TxDAG;
dag: support profile parallel execution path;
protocol: support to transfer TxDAG in NewBLock msg;

PR: #4
* fix several UT with racing issues

* fix incorrect nonce balance codehash issue

case: TestEIP1559 / TestDeleteThenCreate

* Fix ExecutionSpec tests

mainly root caused by balance not updated to dirty correctly.
also fix similar issue with nonce and codehash.

* fix TestBlockChain testcase issue

    TestBlockchain/ValidBlocks/bcStateTests/refundReset.json

Co-authored-by: Sunny <[email protected]>
* dag: add merge execute path method;
pevm: support dispatch with TxDAG;

* dag: add merge execute path method;
pevm: support dispatch with TxDAG;

* dag: clean code;

* statedb: fix some broken uts;

* pevm: support disable slot steal;

---------

Co-authored-by: galaio <[email protected]>
* fix several UT with racing issues

* fix incorrect nonce balance codehash issue

case: TestEIP1559 / TestDeleteThenCreate

* Fix ExecutionSpec tests

mainly root caused by balance not updated to dirty correctly.
also fix similar issue with nonce and codehash.

* fix TestBlockChain testcase issue

    TestBlockchain/ValidBlocks/bcStateTests/refundReset.json

* fix concurrent racing issue of state.accounts.

fix incorrect use of s.accountStorageParallelLock, it is designed
to be used for dirty/pending/original storages, not the accounts and
storages.

Use statedb.AccountMux and statedb.StorageMux for accounts/storages
lock.

* fix issue of DAOTransactions

handle the issue of updateObject of mainDB object touched by DAO transaction.

---------

Co-authored-by: Sunny <[email protected]>
There can be a issue that the object updated by mainDB.GetNonce etc is
obseleted.

The fix use statedb.getStateObjectNoUpdate to avoid touching the
stateObjects of mainDB.

Case: TestBlockchain/ValidBlocks/bcEIP1559/intrinsic.json
…ng; (bnb-chain#10)

* txdag: support txdag transfer in extra;

* txdag: support txdag transfer in extra;

---------

Co-authored-by: galaio <[email protected]>
txdag: record txdag metrics;

txdag: opt txdag flag name;

Co-authored-by: galaio <[email protected]>
…ain#12)

Fix 3 issues:
  - re-execution happens only to new version of baseDB to remove
    redundancy execution. And remove the retry with same baseIndex
    that is conflicted.
  - incorrect merge dirty objects in addrStateChangeInSlot, which
    cause incorrect data.root copied with obseleted stateDB, this
    fix handle the created, stateChanged and deleted object separately.
  - stateObject.GetCommitedState check mainDB of the object delete.

Co-authored-by: Sunny <[email protected]>
* pevm: support delay gas fee calculation;
txdag: check gas fee receiver;
tests: support PEVM+TxDAG UTs;

* txdag: skip some cost time operation;
tests: fix some broken UTs;

---------

Co-authored-by: galaio <[email protected]>
The originStorage will miss some loading in txn execution,do merge
rather than simple copy

This fix also use stateObject specific lock for storage update, rather
than the one in stateDB.

Co-authored-by: Sunny <[email protected]>
txdag: opt rw record flag;

txdag: opt some logic;

Co-authored-by: galaio <[email protected]>
* txdag: opt some logic;

txdag: opt rw set collect logic;

* pevm: opt logs;

* txdag: opt txdag encoding, reduce rlp size;

---------

Co-authored-by: galaio <[email protected]>
* remove finalise

* fix: update maindb txIndex after merge slotDB

otherwise there can be issue that txIndex is load before the change in
mergeSlotDB.

* Fix: avoid update mainDB nonce in executeInSlot

It should use slotDB, otherwise it cause the stateObjects change in
mainDB, which cause racing issue.

* Fix: remove stateDB update during conflict check

stateDB.getState() will update the stateDB's stateObjects, which should
not be updated for the purpose of state read for conflict check.

---------

Co-authored-by: Sunny <[email protected]>
* txdag: add excluded flag;
mvstates: generate txdag with excluded flag;

* pevm: support txdag with excluded tx;

* blockchain: opt txdag file mode;

* pevm: fix dispatch bugs;

* pevm: opt txdag dispatch;

---------

Co-authored-by: galaio <[email protected]>
* pevm: opt slot trigger mechanism;

* txdag: opt execute stat;

* pevm: opt slot trigger mechanism;

* txdag: add txdag more validation logic;

---------

Co-authored-by: galaio <[email protected]>
Make the change into the merged mainDB instead of slotDB to avoid
the concurrency issue

Co-authored-by: Sunny <[email protected]>
* txdag: support new txdep resolve method;
pevm: avoid read txdag file when generating;

* pevm: support read txdag file in const size;

* txdag: reduce mem alloc and async resolve tx dependency;

---------

Co-authored-by: galaio <[email protected]>
)

* pevm: add some parallel tx metrics;

* pevm: opt read large txdag logic;

---------

Co-authored-by: galaio <[email protected]>
This change enable parallel execution when txs count >
parallelTxNum/2 + 2, with lower bound as 4, as the parallel execution
is slower than serial execution with single thread.
This change avoid the rewrite of reads cache in slotDB.
This change fix the issue that prevDestruct is not recorded correctly
and the stateObjectDestruct is not recorded in slotDB's createObject.
The incorrect record causes the GetState get obsoleted state as the
stateObjectDestruct is not correct.
sunny2022da pushed a commit that referenced this pull request Oct 23, 2024
txdag: record txdag metrics;

txdag: opt txdag flag name;

Co-authored-by: galaio <[email protected]>
sunny2022da pushed a commit that referenced this pull request Nov 14, 2024
txdag: record txdag metrics;

txdag: opt txdag flag name;

Co-authored-by: galaio <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants