Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: 🚑 add missing update_payment_stream calls when deleting files #300

Merged
merged 23 commits into from
Dec 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
2a7bf27
test: :test_tube: add failing tests for payment stream update
TDemeco Dec 20, 2024
1f4e770
Merge branch 'main' into fix/update-payment-stream-stop-storing
TDemeco Dec 20, 2024
9fedede
fix: :test_tube: fix new tests after merge from main
TDemeco Dec 20, 2024
12c8b24
fix: :construction: apply_delta now correctly gets file metadata and …
TDemeco Dec 23, 2024
62bb93d
fix: :bug: add missing payment stream updates to file system
TDemeco Dec 24, 2024
25fd059
chore: :label: run typegen
TDemeco Dec 24, 2024
ff5e7c7
chore: :zap: re-run proofs dealer benchmarks after changes
TDemeco Dec 24, 2024
f6c04a7
fix: :bug: add missing root update for bucket in `pending_file_deleti…
TDemeco Dec 24, 2024
7f0814b
chore: :label: run typegen
TDemeco Dec 24, 2024
45e9eeb
feat: :white_check_mark: add payment stream checks + rebenchmark `pro…
TDemeco Dec 24, 2024
58b7b4e
chore: :label: run typegen
TDemeco Dec 24, 2024
aa89328
fix: :adhesive_bandage: Replace `+` and `-` with safe `saturating` fu…
ffarall Dec 26, 2024
7364a27
fix: :adhesive_bandage: Use `checked_add` instead of `saturating_add`
ffarall Dec 26, 2024
59f1741
fix: :rotating_light: Fix compiler issue caused by change to `checked…
ffarall Dec 26, 2024
f666dfd
fix: :bug: add user account to benchmark proof template and script
TDemeco Dec 26, 2024
aa56cb3
fix: :construction: start amending review
TDemeco Dec 26, 2024
6d30671
fix: :art: finish amending review + run typegen
TDemeco Dec 27, 2024
528b997
fix: :white_check_mark: fix xcm-simulator tests
TDemeco Dec 27, 2024
d3f411f
Merge branch 'main' into fix/update-payment-stream-stop-storing
TDemeco Dec 27, 2024
a8ae718
fix: :white_check_mark: remaining fixes for integration tests
TDemeco Dec 27, 2024
e6539d0
fix: :white_check_mark: only push tx hash once
TDemeco Dec 27, 2024
00b0adf
fix: :rotating_light: remove unused variables from sealBlock
TDemeco Dec 27, 2024
502b0d8
style: :art: change where we print extrinsic errors on sealed blocks
TDemeco Dec 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions api-augment/dist/interfaces/lookup.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api-augment/dist/interfaces/lookup.js.map

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions api-augment/dist/types/interfaces/augment-api-errors.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,10 @@ declare module "@polkadot/api-base/types/errors" {
* Divided by 0
**/
DividedByZero: AugmentedError<ApiType>;
/**
* Failed to fetch the dynamic-rate payment stream.
**/
DynamicRatePaymentStreamNotFound: AugmentedError<ApiType>;
/**
* Failed to verify proof: required to provide a proof of inclusion.
**/
Expand Down
14 changes: 9 additions & 5 deletions api-augment/dist/types/interfaces/augment-api-query.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import type {
PalletCollatorSelectionCandidateInfo,
PalletFileSystemMoveBucketRequestMetadata,
PalletFileSystemPendingFileDeletionRequest,
PalletFileSystemPendingStopStoringRequest,
PalletFileSystemStorageRequestBspsMetadata,
PalletFileSystemStorageRequestMetadata,
PalletMessageQueueBookState,
Expand Down Expand Up @@ -382,11 +383,13 @@ declare module "@polkadot/api-base/types/storage" {
/**
* Pending file deletion requests.
*
* A mapping from a user account id to a list of pending file deletion requests, holding a tuple of the file key and bucket id.
* A mapping from a user Account ID to a list of pending file deletion requests, holding a tuple of the file key, file size and Bucket ID.
**/
pendingFileDeletionRequests: AugmentedQuery<
ApiType,
(arg: AccountId32 | string | Uint8Array) => Observable<Vec<ITuple<[H256, H256]>>>,
(
arg: AccountId32 | string | Uint8Array
) => Observable<Vec<PalletFileSystemPendingFileDeletionRequest>>,
[AccountId32]
> &
QueryableStorageEntry<ApiType, [AccountId32]>;
Expand All @@ -408,17 +411,18 @@ declare module "@polkadot/api-base/types/storage" {
/**
* Pending file stop storing requests.
*
* A double mapping from BSP IDs to a list of file keys pending stop storing requests to the block in which those requests were opened
* and the proven size of the file.
* A double mapping from BSP IDs to a list of file keys pending stop storing requests to the block in which those requests were opened,
* the proven size of the file and the owner of the file.
* The block number is used to avoid BSPs being able to stop storing files immediately which would allow them to avoid challenges
* of missing files. The size is to be able to decrease their used capacity when they confirm to stop storing the file.
* The owner is to be able to update the payment stream between the user and the BSP.
**/
pendingStopStoringRequests: AugmentedQuery<
ApiType,
(
arg1: H256 | string | Uint8Array,
arg2: H256 | string | Uint8Array
) => Observable<Option<ITuple<[u32, u64]>>>,
) => Observable<Option<PalletFileSystemPendingStopStoringRequest>>,
[H256, H256]
> &
QueryableStorageEntry<ApiType, [H256, H256]>;
Expand Down
3 changes: 2 additions & 1 deletion api-augment/dist/types/interfaces/augment-api-tx.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -692,6 +692,7 @@ declare module "@polkadot/api-base/types/submittable" {
(
user: AccountId32 | string | Uint8Array,
fileKey: H256 | string | Uint8Array,
fileSize: u64 | AnyNumber | Uint8Array,
bucketId: H256 | string | Uint8Array,
forestProof:
| SpTrieStorageProofCompactProof
Expand All @@ -701,7 +702,7 @@ declare module "@polkadot/api-base/types/submittable" {
| string
| Uint8Array
) => SubmittableExtrinsic<ApiType>,
[AccountId32, H256, H256, SpTrieStorageProofCompactProof]
[AccountId32, H256, u64, H256, SpTrieStorageProofCompactProof]
>;
requestMoveBucket: AugmentedSubmittable<
(
Expand Down
9 changes: 9 additions & 0 deletions api-augment/dist/types/interfaces/lookup.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3475,6 +3475,7 @@ declare const _default: {
pending_file_deletion_request_submit_proof: {
user: string;
fileKey: string;
fileSize: string;
bucketId: string;
forestProof: string;
};
Expand Down Expand Up @@ -4219,6 +4220,14 @@ declare const _default: {
bucketId: string;
fileSize: string;
};
/**
* Lookup463: pallet_file_system::types::PendingStopStoringRequest<T>
**/
PalletFileSystemPendingStopStoringRequest: {
tickWhenRequested: string;
fileOwner: string;
fileSize: string;
};
/**
* Lookup464: pallet_file_system::types::MoveBucketRequestMetadata<T>
**/
Expand Down
2 changes: 2 additions & 0 deletions api-augment/dist/types/interfaces/registry.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ import type {
PalletFileSystemHoldReason,
PalletFileSystemMoveBucketRequestMetadata,
PalletFileSystemPendingFileDeletionRequest,
PalletFileSystemPendingStopStoringRequest,
PalletFileSystemRejectedStorageRequest,
PalletFileSystemRejectedStorageRequestReason,
PalletFileSystemStorageRequestBspsMetadata,
Expand Down Expand Up @@ -363,6 +364,7 @@ declare module "@polkadot/types/types/registry" {
PalletFileSystemHoldReason: PalletFileSystemHoldReason;
PalletFileSystemMoveBucketRequestMetadata: PalletFileSystemMoveBucketRequestMetadata;
PalletFileSystemPendingFileDeletionRequest: PalletFileSystemPendingFileDeletionRequest;
PalletFileSystemPendingStopStoringRequest: PalletFileSystemPendingStopStoringRequest;
PalletFileSystemRejectedStorageRequest: PalletFileSystemRejectedStorageRequest;
PalletFileSystemRejectedStorageRequestReason: PalletFileSystemRejectedStorageRequestReason;
PalletFileSystemStorageRequestBspsMetadata: PalletFileSystemStorageRequestBspsMetadata;
Expand Down
9 changes: 9 additions & 0 deletions api-augment/dist/types/interfaces/types-lookup.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4249,6 +4249,7 @@ declare module "@polkadot/types/lookup" {
readonly asPendingFileDeletionRequestSubmitProof: {
readonly user: AccountId32;
readonly fileKey: H256;
readonly fileSize: u64;
readonly bucketId: H256;
readonly forestProof: SpTrieStorageProofCompactProof;
} & Struct;
Expand Down Expand Up @@ -5365,6 +5366,12 @@ declare module "@polkadot/types/lookup" {
readonly bucketId: H256;
readonly fileSize: u64;
}
/** @name PalletFileSystemPendingStopStoringRequest (463) */
interface PalletFileSystemPendingStopStoringRequest extends Struct {
readonly tickWhenRequested: u32;
readonly fileOwner: AccountId32;
readonly fileSize: u64;
}
/** @name PalletFileSystemMoveBucketRequestMetadata (464) */
interface PalletFileSystemMoveBucketRequestMetadata extends Struct {
readonly requester: AccountId32;
Expand Down Expand Up @@ -5436,6 +5443,7 @@ declare module "@polkadot/types/lookup" {
readonly isInvalidBucketIdFileKeyPair: boolean;
readonly isInconsistentStateKeyAlreadyExists: boolean;
readonly isFixedRatePaymentStreamNotFound: boolean;
readonly isDynamicRatePaymentStreamNotFound: boolean;
readonly isCannotHoldDeposit: boolean;
readonly isFailedToQueryEarliestFileVolunteerTick: boolean;
readonly isFailedToGetOwnerAccount: boolean;
Expand Down Expand Up @@ -5509,6 +5517,7 @@ declare module "@polkadot/types/lookup" {
| "InvalidBucketIdFileKeyPair"
| "InconsistentStateKeyAlreadyExists"
| "FixedRatePaymentStreamNotFound"
| "DynamicRatePaymentStreamNotFound"
| "CannotHoldDeposit"
| "FailedToQueryEarliestFileVolunteerTick"
| "FailedToGetOwnerAccount"
Expand Down
4 changes: 4 additions & 0 deletions api-augment/src/interfaces/augment-api-errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,10 @@ declare module "@polkadot/api-base/types/errors" {
* Divided by 0
**/
DividedByZero: AugmentedError<ApiType>;
/**
* Failed to fetch the dynamic-rate payment stream.
**/
DynamicRatePaymentStreamNotFound: AugmentedError<ApiType>;
/**
* Failed to verify proof: required to provide a proof of inclusion.
**/
Expand Down
14 changes: 9 additions & 5 deletions api-augment/src/interfaces/augment-api-query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import type {
PalletCollatorSelectionCandidateInfo,
PalletFileSystemMoveBucketRequestMetadata,
PalletFileSystemPendingFileDeletionRequest,
PalletFileSystemPendingStopStoringRequest,
PalletFileSystemStorageRequestBspsMetadata,
PalletFileSystemStorageRequestMetadata,
PalletMessageQueueBookState,
Expand Down Expand Up @@ -391,11 +392,13 @@ declare module "@polkadot/api-base/types/storage" {
/**
* Pending file deletion requests.
*
* A mapping from a user account id to a list of pending file deletion requests, holding a tuple of the file key and bucket id.
* A mapping from a user Account ID to a list of pending file deletion requests, holding a tuple of the file key, file size and Bucket ID.
**/
pendingFileDeletionRequests: AugmentedQuery<
ApiType,
(arg: AccountId32 | string | Uint8Array) => Observable<Vec<ITuple<[H256, H256]>>>,
(
arg: AccountId32 | string | Uint8Array
) => Observable<Vec<PalletFileSystemPendingFileDeletionRequest>>,
[AccountId32]
> &
QueryableStorageEntry<ApiType, [AccountId32]>;
Expand All @@ -417,17 +420,18 @@ declare module "@polkadot/api-base/types/storage" {
/**
* Pending file stop storing requests.
*
* A double mapping from BSP IDs to a list of file keys pending stop storing requests to the block in which those requests were opened
* and the proven size of the file.
* A double mapping from BSP IDs to a list of file keys pending stop storing requests to the block in which those requests were opened,
* the proven size of the file and the owner of the file.
* The block number is used to avoid BSPs being able to stop storing files immediately which would allow them to avoid challenges
* of missing files. The size is to be able to decrease their used capacity when they confirm to stop storing the file.
* The owner is to be able to update the payment stream between the user and the BSP.
**/
pendingStopStoringRequests: AugmentedQuery<
ApiType,
(
arg1: H256 | string | Uint8Array,
arg2: H256 | string | Uint8Array
) => Observable<Option<ITuple<[u32, u64]>>>,
) => Observable<Option<PalletFileSystemPendingStopStoringRequest>>,
[H256, H256]
> &
QueryableStorageEntry<ApiType, [H256, H256]>;
Expand Down
3 changes: 2 additions & 1 deletion api-augment/src/interfaces/augment-api-tx.ts
Original file line number Diff line number Diff line change
Expand Up @@ -605,10 +605,11 @@ declare module "@polkadot/api-base/types/submittable" {
(
user: AccountId32 | string | Uint8Array,
fileKey: H256 | string | Uint8Array,
fileSize: u64 | AnyNumber | Uint8Array,
bucketId: H256 | string | Uint8Array,
forestProof: SpTrieStorageProofCompactProof | { encodedNodes?: any } | string | Uint8Array
) => SubmittableExtrinsic<ApiType>,
[AccountId32, H256, H256, SpTrieStorageProofCompactProof]
[AccountId32, H256, u64, H256, SpTrieStorageProofCompactProof]
>;
requestMoveBucket: AugmentedSubmittable<
(
Expand Down
10 changes: 10 additions & 0 deletions api-augment/src/interfaces/lookup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3555,6 +3555,7 @@ export default {
pending_file_deletion_request_submit_proof: {
user: "AccountId32",
fileKey: "H256",
fileSize: "u64",
bucketId: "H256",
forestProof: "SpTrieStorageProofCompactProof"
},
Expand Down Expand Up @@ -4445,6 +4446,14 @@ export default {
bucketId: "H256",
fileSize: "u64"
},
/**
* Lookup463: pallet_file_system::types::PendingStopStoringRequest<T>
**/
PalletFileSystemPendingStopStoringRequest: {
tickWhenRequested: "u32",
fileOwner: "AccountId32",
fileSize: "u64"
},
/**
* Lookup464: pallet_file_system::types::MoveBucketRequestMetadata<T>
**/
Expand Down Expand Up @@ -4521,6 +4530,7 @@ export default {
"InvalidBucketIdFileKeyPair",
"InconsistentStateKeyAlreadyExists",
"FixedRatePaymentStreamNotFound",
"DynamicRatePaymentStreamNotFound",
"CannotHoldDeposit",
"FailedToQueryEarliestFileVolunteerTick",
"FailedToGetOwnerAccount",
Expand Down
2 changes: 2 additions & 0 deletions api-augment/src/interfaces/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ import type {
PalletFileSystemHoldReason,
PalletFileSystemMoveBucketRequestMetadata,
PalletFileSystemPendingFileDeletionRequest,
PalletFileSystemPendingStopStoringRequest,
PalletFileSystemRejectedStorageRequest,
PalletFileSystemRejectedStorageRequestReason,
PalletFileSystemStorageRequestBspsMetadata,
Expand Down Expand Up @@ -370,6 +371,7 @@ declare module "@polkadot/types/types/registry" {
PalletFileSystemHoldReason: PalletFileSystemHoldReason;
PalletFileSystemMoveBucketRequestMetadata: PalletFileSystemMoveBucketRequestMetadata;
PalletFileSystemPendingFileDeletionRequest: PalletFileSystemPendingFileDeletionRequest;
PalletFileSystemPendingStopStoringRequest: PalletFileSystemPendingStopStoringRequest;
PalletFileSystemRejectedStorageRequest: PalletFileSystemRejectedStorageRequest;
PalletFileSystemRejectedStorageRequestReason: PalletFileSystemRejectedStorageRequestReason;
PalletFileSystemStorageRequestBspsMetadata: PalletFileSystemStorageRequestBspsMetadata;
Expand Down
10 changes: 10 additions & 0 deletions api-augment/src/interfaces/types-lookup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4459,6 +4459,7 @@ declare module "@polkadot/types/lookup" {
readonly asPendingFileDeletionRequestSubmitProof: {
readonly user: AccountId32;
readonly fileKey: H256;
readonly fileSize: u64;
readonly bucketId: H256;
readonly forestProof: SpTrieStorageProofCompactProof;
} & Struct;
Expand Down Expand Up @@ -5630,6 +5631,13 @@ declare module "@polkadot/types/lookup" {
readonly fileSize: u64;
}

/** @name PalletFileSystemPendingStopStoringRequest (463) */
interface PalletFileSystemPendingStopStoringRequest extends Struct {
readonly tickWhenRequested: u32;
readonly fileOwner: AccountId32;
readonly fileSize: u64;
}

/** @name PalletFileSystemMoveBucketRequestMetadata (464) */
interface PalletFileSystemMoveBucketRequestMetadata extends Struct {
readonly requester: AccountId32;
Expand Down Expand Up @@ -5702,6 +5710,7 @@ declare module "@polkadot/types/lookup" {
readonly isInvalidBucketIdFileKeyPair: boolean;
readonly isInconsistentStateKeyAlreadyExists: boolean;
readonly isFixedRatePaymentStreamNotFound: boolean;
readonly isDynamicRatePaymentStreamNotFound: boolean;
readonly isCannotHoldDeposit: boolean;
readonly isFailedToQueryEarliestFileVolunteerTick: boolean;
readonly isFailedToGetOwnerAccount: boolean;
Expand Down Expand Up @@ -5775,6 +5784,7 @@ declare module "@polkadot/types/lookup" {
| "InvalidBucketIdFileKeyPair"
| "InconsistentStateKeyAlreadyExists"
| "FixedRatePaymentStreamNotFound"
| "DynamicRatePaymentStreamNotFound"
| "CannotHoldDeposit"
| "FailedToQueryEarliestFileVolunteerTick"
| "FailedToGetOwnerAccount"
Expand Down
2 changes: 1 addition & 1 deletion api-augment/storagehub.json

Large diffs are not rendered by default.

15 changes: 10 additions & 5 deletions pallets/file-system/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -402,30 +402,31 @@

/// Pending file deletion requests.
///
/// A mapping from a user account id to a list of pending file deletion requests, holding a tuple of the file key and bucket id.
/// A mapping from a user Account ID to a list of pending file deletion requests, holding a tuple of the file key, file size and Bucket ID.
#[pallet::storage]
pub type PendingFileDeletionRequests<T: Config> = StorageMap<
_,
Blake2_128Concat,
T::AccountId,
BoundedVec<(MerkleHash<T>, BucketIdFor<T>), T::MaxUserPendingDeletionRequests>,
BoundedVec<PendingFileDeletionRequest<T>, T::MaxUserPendingDeletionRequests>,
ValueQuery,
>;

/// Pending file stop storing requests.
///
/// A double mapping from BSP IDs to a list of file keys pending stop storing requests to the block in which those requests were opened
/// and the proven size of the file.
/// A double mapping from BSP IDs to a list of file keys pending stop storing requests to the block in which those requests were opened,
/// the proven size of the file and the owner of the file.
/// The block number is used to avoid BSPs being able to stop storing files immediately which would allow them to avoid challenges
/// of missing files. The size is to be able to decrease their used capacity when they confirm to stop storing the file.
/// The owner is to be able to update the payment stream between the user and the BSP.
#[pallet::storage]
pub type PendingStopStoringRequests<T: Config> = StorageDoubleMap<
_,
Blake2_128Concat,
ProviderIdFor<T>,
Blake2_128Concat,
MerkleHash<T>,
(BlockNumberFor<T>, StorageData<T>),
PendingStopStoringRequest<T>,
>;

/// Pending move bucket requests.
Expand Down Expand Up @@ -802,6 +803,8 @@
InconsistentStateKeyAlreadyExists,
/// Failed to fetch the rate for the payment stream.
FixedRatePaymentStreamNotFound,
/// Failed to fetch the dynamic-rate payment stream.
DynamicRatePaymentStreamNotFound,
/// Cannot hold the required deposit from the user
CannotHoldDeposit,
/// Failed to query earliest volunteer tick
Expand Down Expand Up @@ -1277,6 +1280,7 @@
origin: OriginFor<T>,
user: T::AccountId,
file_key: MerkleHash<T>,
file_size: StorageData<T>,
bucket_id: BucketIdFor<T>,
forest_proof: ForestProof<T>,
) -> DispatchResult {
Expand All @@ -1286,6 +1290,7 @@
who.clone(),
user.clone(),
file_key,
file_size,
bucket_id,
forest_proof,
)?;
Expand Down Expand Up @@ -1338,7 +1343,7 @@
}
}

#[pallet::hooks]

Check warning on line 1346 in pallets/file-system/src/lib.rs

View workflow job for this annotation

GitHub Actions / Check lint with clippy

using `map_err` over `inspect_err`
impl<T: Config> Hooks<BlockNumberFor<T>> for Pallet<T>
where
u32: TryFrom<BlockNumberFor<T>>,
Expand Down
Loading
Loading