Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R4R] feat(oracle) #24

Closed
wants to merge 38 commits into from
Closed

Conversation

leonz789
Copy link
Contributor

@leonz789 leonz789 commented Apr 2, 2024

Description

Oracle module is used to collect external prices from sources(offchain/onchain) outside of exocorechain. And the prices access served by oracle module will be used by other modules like operator for voting power calculation.
Supersedes #6 , reconstruct the commits for easy to review.

commits:

  1. proto file definition and some ignite scalffold dependencies
  2. mainly generated files from proto definition
  3. implement keeper for state
  4. add tests for keeper
  5. add cache serves as memory storage for price collection across blocks
  6. add aggregator, this implements the main logic for price aggregation
Screenshot 2024-04-03 at 03 48 43

leonz789 added 30 commits April 3, 2024 00:10
@MaxMustermann2
Copy link
Contributor

Can you please fix the failing workflows?

bwhour

This comment was marked as outdated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The status field uses an integer type to indicate status (1 for open, 2 for closed). Using enumerated types or constant definitions instead of magic numbers makes the code easier to understand and maintain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AggregatorContext contains multiple mappings (e.g. validatorsPower, rounds, aggregators) that may be accessed and modified concurrently in a multi-threaded environment. If this is the case, you need to ensure that access to these mappings is thread-safe and may need to use mutex locking or other concurrency control mechanisms

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concurrency control needs to be considered when multiple goroutines may access or modify the deterministicSource mapping in the calculator. it is a common requirement to handle concurrency in applications like blockchain and price aggregation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there exists a pSource with no prices, accessing pSource.Prices[0] will trigger panic

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use string concatenation (validator + strconv. Itoa(int(pSource.SourceId))) to generate the mapped key, whether this key generation rule may cause key conflicts

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For collection operations (Add operations), assuming that they always succeed without checking the return value can hide problems.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting f, c, and a to nil in the seal method may help with garbage collection, but if there are operations in these objects that require explicit shutdown or cleanup of resources, extra care is needed to ensure that the resources are freed properly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

list4Calculator, list4Aggregator := w.f.filtrate(msg): if the filtrate method encounters any exceptions while processing a message (e.g., the message is formatted incorrectly), the current implementation will not catch or handle those exceptions

Copy link
Contributor

@MaxMustermann2 MaxMustermann2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not fully reviewed.

storeKey storetypes.StoreKey
memKey storetypes.StoreKey
paramstore paramtypes.Subspace
stakingKeeper stakingkeeper.Keeper
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please convert this to the minimum required interface in expected_keepers.go.


keeper keeper.Keeper
accountKeeper types.AccountKeeper
bankKeeper types.BankKeeper
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment here to say that they are only used in the simulation?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im.PSources = append(im.PSources, item.PSources...)
This case is If there are new PSources to be added, it will add the entire PSources array to the existing entry, not just the new SourceId. If if the new PSources array contains some new and some old (already existing in the current entry) SourceIds, then the price sources associated with the old SourceId will be added repeatedly; over time, if new PSources are added frequently, the PSources array in each entry may become very large, affecting data processing and query efficiency。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

functions in this cache file seems no error handling and logging logic.

price.NextRountId = binary.BigEndian.Uint64(iterator.Value())
} else {
var val types.PriceWithTimeAndRound
k.cdc.Unmarshal(iterator.Value(), &val)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k.cdc.Unmarshal has error return which need to be handle. can k.cdc.MustMarshal ?
image

return
}

k.cdc.Unmarshal(b, &price)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k.cdc.Unmarshal has error return which need to be handle. can k.cdc.MustMarshal ?

b := store.Get(types.PricesRoundKey(nextRoundId - 1))
if b != nil {
//should always be true
k.cdc.Unmarshal(b, &price)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k.cdc.Unmarshal has error return which need to be handle. can k.cdc.MustMarshal ?

return
}
}
if len(r.prices) < cap(r.prices) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len(r.prices) < cap(r.prices)
Its a good practice taking takes into account not exceeding the preset capacity.
Is it necessary to take into account other boundary conditions, such as validation of the input parameters ?


// udpate priceAndPower for a specific DSRoundID, if the price exists, increase its power with provided data
// return confirmed=true, when detect power exceeds the threshold
func (r *roundPrices) updatePriceAndPower(pw *priceAndPower, totalPower *big.Int) (updated bool, confirmed bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the updatePriceAndPower method, when new prices and weights are added to roundPrices, if any exceptions are encountered (e.g., if pw.price or pw.power is nil), the code doesn't perform error handling

@bwhour bwhour self-requested a review April 5, 2024 11:08
@leonz789
Copy link
Contributor Author

leonz789 commented Apr 8, 2024

Merging with the original branch introduced confusion due to too many commits. I will reconstruct it to create a cleaner version. Apologies for the mess. (will follow this 6-commit structure for easy review)

@leonz789 leonz789 mentioned this pull request Apr 9, 2024
@leonz789
Copy link
Contributor Author

leonz789 commented Apr 9, 2024

I will close this pr, and #30 will take place for the oracle module feature pr.

@leonz789 leonz789 closed this Apr 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants