Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix idb account contains asset, fix wasm gas prices #1603

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/cuddly-monkeys-end.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@penumbra-zone/getters': minor
---

add getAmountFromNote
6 changes: 6 additions & 0 deletions .changeset/itchy-planets-return.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@penumbra-zone/storage': major
'@penumbra-zone/types': major
---

implement accountHasSpendableAsset correctly
5 changes: 5 additions & 0 deletions .changeset/rotten-apes-raise.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@penumbra-zone/wasm': minor
---

do not fail planning upon undefined gas
4 changes: 4 additions & 0 deletions packages/getters/src/note.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import { Note } from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/core/component/shielded_pool/v1/shielded_pool_pb.js';
import { createGetter } from './utils/create-getter.js';

export const getAmountFromNote = createGetter((note?: Note) => note?.value?.amount);
2 changes: 1 addition & 1 deletion packages/services/src/test-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ export interface IndexedDbMock {
getAuctionOutstandingReserves?: Mock;
stakingTokenAssetId?: Mock;
upsertAuction?: Mock;
hasTokenBalance?: Mock;
accountHasSpendableAsset?: Mock;
saveGasPrices?: Mock;
}

Expand Down
32 changes: 16 additions & 16 deletions packages/services/src/view-service/fees.test.ts
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

someone should fix these tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ describe('extractAltFee', () => {
getAuction: vi.fn(),
saveGasPrices: vi.fn(),
getAltGasPrices: vi.fn(),
hasTokenBalance: vi.fn(),
accountHasSpendableAsset: vi.fn(),
};
});

Expand Down Expand Up @@ -66,7 +66,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(inputAssetId)).toBeTruthy();
Expand Down Expand Up @@ -101,7 +101,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(inputAssetId)).toBeTruthy();
Expand Down Expand Up @@ -155,7 +155,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(outputAssetId)).toBeTruthy();
Expand Down Expand Up @@ -191,7 +191,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(inputAssetId)).toBeTruthy();
Expand Down Expand Up @@ -227,7 +227,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(inputAssetId)).toBeTruthy();
Expand Down Expand Up @@ -313,7 +313,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(inputAssetId)).toBeTruthy();
Expand All @@ -339,7 +339,7 @@ describe('extractAltFee', () => {
seqNum: 0n,
});

mockIndexedDb.hasTokenBalance?.mockResolvedValueOnce(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValueOnce(true);

const request = new TransactionPlannerRequest({
dutchAuctionEndActions: [
Expand All @@ -362,7 +362,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const result = await extractAltFee(request, mockIndexedDb as unknown as IndexedDbInterface);
expect(result.equals(inputAssetId)).toBeTruthy();
Expand Down Expand Up @@ -406,9 +406,9 @@ describe('extractAltFee', () => {
mockIndexedDb.saveGasPrices?.mockResolvedValue(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);

// Mock hasTokenBalance twice to control its behavior in the function
mockIndexedDb.hasTokenBalance?.mockResolvedValueOnce(false); // For the specificAssetId check
mockIndexedDb.hasTokenBalance?.mockResolvedValueOnce(true); // For the altGasPrices check
// Mock accountHasSpendableAsset twice to control its behavior in the function
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValueOnce(false); // For the specificAssetId check
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValueOnce(true); // For the altGasPrices check

const request = new TransactionPlannerRequest({
dutchAuctionEndActions: [
Expand Down Expand Up @@ -455,7 +455,7 @@ describe('extractAltFee', () => {

mockIndexedDb.saveGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);
mockIndexedDb.hasTokenBalance?.mockResolvedValue(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValue(true);

const request = new TransactionPlannerRequest({
dutchAuctionWithdrawActions: [
Expand Down Expand Up @@ -508,9 +508,9 @@ describe('extractAltFee', () => {
mockIndexedDb.saveGasPrices?.mockResolvedValue(gasPrices);
mockIndexedDb.getAltGasPrices?.mockResolvedValueOnce(gasPrices);

// Mock hasTokenBalance twice to control its behavior in the function
mockIndexedDb.hasTokenBalance?.mockResolvedValueOnce(false); // For the specificAssetId check
mockIndexedDb.hasTokenBalance?.mockResolvedValueOnce(true); // For the altGasPrices check
// Mock accountHasSpendableAsset twice to control its behavior in the function
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValueOnce(false); // For the specificAssetId check
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValueOnce(true); // For the altGasPrices check

const request = new TransactionPlannerRequest({
dutchAuctionWithdrawActions: [
Expand Down
10 changes: 7 additions & 3 deletions packages/services/src/view-service/fees.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { AssetId } from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/core/a
import { TransactionPlannerRequest } from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/view/v1/view_pb.js';
import { assetIdFromBaseDenom } from '@penumbra-zone/wasm/asset';
import { IndexedDbInterface } from '@penumbra-zone/types/indexed-db';
import { bech32mAssetId } from '@penumbra-zone/bech32m/passet';

// Attempts to extract a fee token, with priority in descending order, from the assets used
// in the actions of the transaction planner request (TPR). If no fee token is found from the
Expand Down Expand Up @@ -117,9 +118,12 @@ export const getAssetFromGasPriceTable = async (
// If a specific asset ID is provided, extracted from the transaction request, check its balance is
// positive and GasPrices for that asset exist.
if (assetId) {
const balance = await indexedDb.hasTokenBalance(request.source, assetId);
const balance = await indexedDb.accountHasSpendableAsset(request.source, assetId);
// This check ensures that the alternative fee token is a valid fee token, for example, TestUSD is not.
const isInGasTable = altGasPrices.find(gp => gp.assetId?.equals(assetId));
TalDerei marked this conversation as resolved.
Show resolved Hide resolved
const isInGasTable = altGasPrices.find(
// TODO: assert this more thoroughly
gp => bech32mAssetId(gp.assetId!) === bech32mAssetId(assetId),
);
if (balance && isInGasTable) {
return assetId;
}
Comment on lines 120 to 129
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part could be improved but i'm going to bed

Copy link
Contributor

@TalDerei TalDerei Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this checks if the assetId passed in exists in the GasPrices table. since we're comparing assetId objects directly, why isn't the equality method sufficient here?

Expand All @@ -129,7 +133,7 @@ export const getAssetFromGasPriceTable = async (
// GasPrices table as a fallback case.
for (const gasPrice of altGasPrices) {
if (gasPrice.assetId) {
const balance = await indexedDb.hasTokenBalance(request.source, gasPrice.assetId);
const balance = await indexedDb.accountHasSpendableAsset(request.source, gasPrice.assetId);
if (balance) {
return gasPrice.assetId;
}
Expand Down
4 changes: 0 additions & 4 deletions packages/services/src/view-service/gas-prices.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import type { Impl } from './index.js';
import { servicesCtx } from '../ctx/prax.js';
import { Code, ConnectError } from '@connectrpc/connect';

/**
* Gas prices are published within the 'CompactBlock' whenever they change. The specific block
Expand All @@ -25,9 +24,6 @@ export const gasPrices: Impl['gasPrices'] = async (_, ctx) => {
const { indexedDb } = await services.getWalletServices();
const gasPrices = await indexedDb.getNativeGasPrices();
const altGasPRices = await indexedDb.getAltGasPrices();
if (!gasPrices) {
throw new ConnectError('Gas prices is not available', Code.NotFound);
}

return {
gasPrices,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ describe('TransactionPlanner request handler', () => {
getAppParams: vi.fn(),
getNativeGasPrices: vi.fn(),
constants: vi.fn(),
hasTokenBalance: vi.fn(),
accountHasSpendableAsset: vi.fn(),
};

mockServices = {
Expand Down Expand Up @@ -104,7 +104,7 @@ describe('TransactionPlanner request handler', () => {
);

mockIndexedDb.stakingTokenAssetId?.mockResolvedValueOnce(true);
mockIndexedDb.hasTokenBalance?.mockResolvedValueOnce(true);
mockIndexedDb.accountHasSpendableAsset?.mockResolvedValueOnce(true);

await transactionPlanner(req, mockCtx);

Expand Down
25 changes: 13 additions & 12 deletions packages/services/src/view-service/transaction-planner/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,29 @@ import { TransactionPlannerRequest } from '@buf/penumbra-zone_penumbra.bufbuild_
import { fvkCtx } from '../../ctx/full-viewing-key.js';
import { extractAltFee } from '../fees.js';
import { assertTransactionSource } from './assert-transaction-source.js';
import { AddressIndex } from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/core/keys/v1/keys_pb.js';
import { toPlainMessage } from '@bufbuild/protobuf';

export const transactionPlanner: Impl['transactionPlanner'] = async (req, ctx) => {
assertValidRequest(req);

const services = await ctx.values.get(servicesCtx)();
const { indexedDb } = await services.getWalletServices();

// Query IndexedDB directly to check for the existence of staking token
const nativeToken = await indexedDb.hasTokenBalance(req.source!, indexedDb.stakingTokenAssetId);
const noNativeTokenAvailable = !(await indexedDb.accountHasSpendableAsset(
toPlainMessage(req.source ?? new AddressIndex()),
indexedDb.stakingTokenAssetId,
));
Comment on lines +19 to +22
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually shouldn't AddressIndex never be undefined, since it will fail the transaction source check. In that case, why can't we use the non-null assertion operator on source?


const planningSwapClaim = req.swapClaims.length > 0;
// Check if we should use the native token or extract an alternate gas fee token.
// Special cased for swap claims as gas fee needs to match the claimFee on the corresponding swap.
const needsAltFeeToken = !nativeToken || req.swapClaims.length > 0;
const gasFeeToken = needsAltFeeToken
? await extractAltFee(req, indexedDb)
: indexedDb.stakingTokenAssetId;
console.log('alt fee reasons', noNativeTokenAvailable, planningSwapClaim);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove log


const gasFeeToken =
noNativeTokenAvailable || planningSwapClaim
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually prefer keeping needsAltFeeToken, so we can tack on more invariants later.

? await extractAltFee(req, indexedDb)
: indexedDb.stakingTokenAssetId;

const fmdParams = await indexedDb.getFmdParams();
if (!fmdParams) {
Expand All @@ -37,12 +44,6 @@ export const transactionPlanner: Impl['transactionPlanner'] = async (req, ctx) =
throw new ConnectError('ChainId not available', Code.FailedPrecondition);
}

// Wasm planner needs the presence of gas prices in the db to work
const gasPrices = await indexedDb.getNativeGasPrices();
if (!gasPrices) {
throw new ConnectError('Gas prices is not available', Code.FailedPrecondition);
}

const idbConstants = indexedDb.constants();
const fvk = await ctx.values.get(fvkCtx)();
const plan = await planTransaction(idbConstants, req, fvk, gasFeeToken);
Expand Down
36 changes: 21 additions & 15 deletions packages/storage/src/indexed-db/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ import {
DutchAuctionDescription,
} from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/core/component/auction/v1/auction_pb.js';
import { ChainRegistryClient } from '@penumbra-labs/registry';
import { PartialMessage } from '@bufbuild/protobuf';
import { getAmountFromRecord } from '@penumbra-zone/getters/spendable-note-record';
import { PartialMessage, PlainMessage, toPlainMessage } from '@bufbuild/protobuf';
import { getAmountFromNote } from '@penumbra-zone/getters/note';
import { isZero } from '@penumbra-zone/types/amount';
import { IDB_VERSION } from './config.js';

Expand Down Expand Up @@ -869,20 +869,26 @@ export class IndexedDb implements IndexedDbInterface {
};
}

async hasTokenBalance(addressIndex: AddressIndex, assetId: AssetId): Promise<boolean> {
const spendableNotes = await this.db.getAllFromIndex(
'SPENDABLE_NOTES',
'assetId',
uint8ArrayToBase64(assetId.inner),
);
async accountHasSpendableAsset(
{ account }: PlainMessage<AddressIndex>,
{ inner }: PlainMessage<AssetId>,
): Promise<boolean> {
// asserts length of assetId.inner
bech32mAssetId({ inner });

return spendableNotes.some(note => {
const spendableNote = SpendableNoteRecord.fromJson(note);
return (
spendableNote.heightSpent === 0n &&
!isZero(getAmountFromRecord(spendableNote)) &&
spendableNote.addressIndex?.equals(addressIndex)
);
const assetNotes = (
await this.db.getAllFromIndex('SPENDABLE_NOTES', 'assetId', uint8ArrayToBase64(inner))
).map(note => SpendableNoteRecord.fromJson(note));

const spendableNotes = assetNotes.filter(({ note, addressIndex, heightSpent }) => {
const noteAddressIndex = toPlainMessage(addressIndex ?? new AddressIndex());
const accountMatch = noteAddressIndex.account === account;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it correct to ignore account randomizer for this comparison?

Copy link
Contributor

@TalDerei TalDerei Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const nonzero = !isZero(getAmountFromNote(note));
const unspent = heightSpent === 0n;
return accountMatch && nonzero && unspent;
});

console.log('accountHasSpendableAsset', account, bech32mAssetId({ inner }), spendableNotes);
return spendableNotes.length > 0;
}
}
7 changes: 5 additions & 2 deletions packages/types/src/indexed-db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ import {
SwapRecord,
TransactionInfo,
} from '@buf/penumbra-zone_penumbra.bufbuild_es/penumbra/view/v1/view_pb.js';
import { PartialMessage } from '@bufbuild/protobuf';
import { PartialMessage, PlainMessage } from '@bufbuild/protobuf';
import type { Jsonified } from './jsonified.js';

export interface IdbUpdate<DBTypes extends PenumbraDb, StoreName extends StoreNames<DBTypes>> {
Expand Down Expand Up @@ -150,7 +150,10 @@ export interface IndexedDbInterface {
auctionId: AuctionId,
): Promise<{ input: Value; output: Value } | undefined>;

hasTokenBalance(addressIndex: AddressIndex, assetId: AssetId): Promise<boolean>;
accountHasSpendableAsset(
addressIndex: PlainMessage<AddressIndex>,
assetId: PlainMessage<AssetId>,
): Promise<boolean>;
}

export interface PenumbraDb extends DBSchema {
Expand Down
2 changes: 1 addition & 1 deletion packages/wasm/crate/src/planner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ pub async fn plan_transaction_inner<Db: Database>(
let gas_prices = storage
.get_gas_prices_by_asset_id(&fee_asset_id)
.await?
.ok_or_else(|| anyhow!("GasPrices not available"))?;
.unwrap_or_default();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment below addressing #1602

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a mistake, we shouldn't use default gas prices
And use only those gas prices that have been published in compact blocks

Copy link
Contributor

@TalDerei TalDerei Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this sentiment. For example, this could theoretically allow paying fees with unsupported gas tokens, like TestUSD. However, I don't think it would ever error in the planner because we have checks in the frontend at in getAssetFromGasPriceTable, which would then trigger this error.


let fee_tier = match request.fee_mode {
None => FeeTier::default(),
Expand Down
Loading