Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split and cleanup client #9

Merged
merged 44 commits into from
Sep 3, 2024
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
42c3806
splitting up client
f321x Jul 16, 2024
2a3b524
merge
f321x Jul 21, 2024
b6fc274
separate client logic, cleanup
f321x Jul 21, 2024
3b9d16c
remaining changes
f321x Jul 21, 2024
71efb91
add comments
f321x Jul 21, 2024
29aabb1
put trade mode in metadata
f321x Jul 21, 2024
e181c89
minor improvements
f321x Jul 21, 2024
9d6ad91
Create ecash wallet at the parse input phase, the trade parner needs …
rodant Jul 25, 2024
c9bb1cf
Fix npubs by usong to_bech32.
rodant Jul 26, 2024
b7288cf
rewrite in a more idiomatic form.
rodant Jul 31, 2024
64923f7
Move EscrowClient struct to its module.
rodant Aug 2, 2024
236346c
Inline init_ttrade in main.
rodant Aug 2, 2024
6b10695
Extract wallet creation from RawCliInput
rodant Aug 5, 2024
7e28154
Start introducing the escrow client as state machine for better testa…
rodant Aug 6, 2024
0cf6574
Remove dependency from cli in escrow client.
rodant Aug 6, 2024
7cef1c7
Some clean up.
rodant Aug 6, 2024
d32dc48
Move trade mode to the escrow client.
rodant Aug 6, 2024
3ffe54f
Split escrow metadata in the contract and escrow registration parts.
rodant Aug 9, 2024
7e1b21a
Use PublicKey type in trade contract.
rodant Aug 10, 2024
71af4c2
Replace tuple though the registration message struct.
rodant Aug 13, 2024
af5121c
Remove utils modules.
rodant Aug 13, 2024
2a88dbc
Remove PubkeyMessage and rearrange models.
rodant Aug 14, 2024
7ee5b40
Fix remaining warnings.
rodant Aug 14, 2024
a27ea9a
bump up nostr_sdk version and revert to nip04 direct messages, nip17 …
rodant Aug 15, 2024
85ce4d0
working version with nip17 private direct messages.
rodant Aug 16, 2024
a9714b7
Make func sync after review comment.
rodant Aug 18, 2024
13082e7
Introduce timeout and improvements in receive_escrow_message
rodant Aug 20, 2024
542a2be
Better having a get public key method than a get npub.
rodant Aug 20, 2024
f001e9c
Remove dependency to cli in ClientNostrClient.
rodant Aug 20, 2024
5014664
Hide nostr client in ClientNostrInstance.
rodant Aug 20, 2024
9ae1ba4
Improve the coordinator loop for running forever.
rodant Aug 21, 2024
0d7a2a2
Improvement for error situations.
rodant Aug 21, 2024
645158c
Bumb up nostr_sdk version.
rodant Aug 21, 2024
5e20036
Handle some error cases.
rodant Aug 22, 2024
6a0e061
Avoid unneeded async func.
rodant Aug 22, 2024
5aaa656
Remove nostr instance and clean up escrow client.
rodant Aug 22, 2024
3bcfde4
Fix race condition in receiving registration and introduce escrow cli…
rodant Aug 26, 2024
d038b12
remove unused imports.
rodant Aug 26, 2024
8d79e1c
Deposit enough funds and proceed to a delivery.
rodant Aug 30, 2024
09396a7
Fix typos.
rodant Sep 1, 2024
f4b75cb
Update coordinator/src/escrow_coordinator/mod.rs
rodant Sep 1, 2024
e2e4382
Initialize the notifications receiver at creation and pull events on …
rodant Sep 2, 2024
e1d295e
Define nostr client as field of the escrow client and pass all its fi…
rodant Sep 2, 2024
94d2424
Handle receice errors from a subscription.
rodant Sep 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix race condition in receiving registration and introduce escrow cli…
…ent states.
  • Loading branch information
rodant committed Aug 26, 2024
commit 3bcfde413584eba8201329fdf199921afe664f4e
113 changes: 64 additions & 49 deletions client/src/escrow_client/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use std::sync::Arc;

use crate::common::model::EscrowRegistration;
use cdk::nuts::Token;

@@ -9,27 +11,21 @@ pub enum TradeMode {
Seller,
}

pub struct EscrowClient {
nostr_client: NostrClient, // can either be a Nostr Client or Nostr note signer (without networking)
pub struct InitEscrowClient {
ecash_wallet: ClientEcashWallet,
escrow_registration: Option<EscrowRegistration>,
escrow_contract: TradeContract,
trade_mode: TradeMode,
}

// todo: model EscrowClient as an state machine (stm). This will improve testability too.
impl EscrowClient {
// creates the inital state: the coordinator data isn't present.
/// Initial Escrow Client state.
impl InitEscrowClient {
pub fn new(
nostr_client: NostrClient,
ecash_wallet: ClientEcashWallet,
escrow_contract: TradeContract,
trade_mode: TradeMode,
) -> Self {
Self {
nostr_client,
ecash_wallet,
escrow_registration: None,
escrow_contract,
trade_mode,
}
@@ -40,60 +36,75 @@ impl EscrowClient {
/// After this the coordinator data is set, state trade registered.
///
/// After this state the trade contract is effectfull as well, possible coordinator fees must be payed.
pub async fn register_trade(&mut self) -> anyhow::Result<()> {
pub async fn register_trade(
&self,
nostr_client: Arc<NostrClient>,
) -> anyhow::Result<RegisteredEscrowClient> {
let my_pubkey = nostr_client.public_key();
let nostr_client_ref = nostr_client.clone();
let reg_msg_fut =
Copy link
Owner Author

@f321x f321x Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why we need to spawn a new task to listen to the escrow msg and then await it afterwards. Couldn't we just send the contract message to the coordinator and then start listening to answers (receive_escrow_message) of the coordinator with a limit higher than 0 (e.g. 1) so we receive "old" messages? I guess there is a slight time benefit if we're already subscribed when expecting the coordinator msg but is it worth this additional complexity?

        let message_filter = Filter::new()
            .kind(Kind::GiftWrap)
            .pubkey(self.keys.public_key())
            .limit(1);

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the maintainer of rust-nostr limit(0) is the way to go in case of GiftWrapped event and thus also for NIP-17. See the thread rust-nostr/nostr#173 (comment). Those events have a random created_at field which can be within 2 days in the past. If the client isn't listening before sending the contract, in some situations it misses the answer of the coordinator. I have observed this several times.

Copy link
Owner Author

@f321x f321x Sep 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you observed the client missing the events when subscribing after the event was published, did you test with Timestamp::now() or limit(0)? Just for my general understanding, I see now why Timestamp::now() doesn't work due to the random created_at, but in my understanding limit(1) should still work as it just tells the relay to return the last stored event. To make it reliable even in case there are other dms stored to the pubkey that could be returned due to the random created_at we could combine a timestamp - 2days + limit(higher number like 10), then use the newest event by the created_at in the wrapped rumor kind 14 event which should be correct and not random.

Like this for example:

    pub async fn receive_escrow_message(&self, timeout_secs: u64) -> anyhow::Result<String> {
        let message_filter = Filter::new()
            .kind(Kind::GiftWrap)
            .pubkey(self.keys.public_key())
            .since(Timestamp::now() - Duration::from_secs(172900))  // 2 days + 100 sec
            .limit(20);  // some limit

        let subscription_id = self.client.subscribe(vec![message_filter], None).await?.val;

        let mut notifications = self.client.notifications();

        let loop_future = async {
            loop {
                if let Ok(notification) = notifications.recv().await {
                    if let RelayPoolNotification::Event { event, .. } = notification {
                        let rumor = self.client.unwrap_gift_wrap(&event).await?.rumor;
// check if its a plausible timestamp
                        if rumor.kind == Kind::PrivateDirectMessage && rumor.created_at > Time::now() - Duration::from_secs(120) {
                            break Ok(rumor.content) as anyhow::Result<String>;
                        }
                    }
                }
            }
        };

    pub async fn register_trade(
        &self,
        nostr_client: Arc<NostrClient>,
    ) -> anyhow::Result<RegisteredEscrowClient> {
        let coordinator_pk = &self.escrow_contract.npubkey_coordinator;
        let contract_message = serde_json::to_string(&self.escrow_contract)?;
        dbg!("sending contract to coordinator...");
        nostr_client
            .client
            .send_private_msg(*coordinator_pk, &contract_message, None)
            .await?;

        let registration_message = nostr_client.receive_escrow_message(20).await?;
        let escrow_registration = serde_json::from_str(&registration_message)?;
        Ok(RegisteredEscrowClient {
            prev_state: self,
            escrow_registration,
        })
    }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've observed this issue for a long time, when we had the since(now) filter and also after switching to limit(0). It happened when the buyer sends first the contract and the seller as second one. In this case the coordinator fires almost always the registration before the buyer starts listening for events. You can see this for example on master and in earlier stages of this branch. The fix to this by listening before sending the contract to the coordinator happened way later than the switch to nip-17. Now we still have a similar issue in a later step in case of the seller, it can start listening for the token after the buyer sent the token, I have also observed this. I think the approach with a higher limit isn't reliable enough and we need a different design. The nostr client must start listening from the very beginning and buffer relevant messages, then the escrow client can ask any time for messages with a new version of receive_escrow_message and pick the one it is looking for. It is an idea similar to an actor system where the nostr client acts as a mailbox for the escrow client. Note that in this approach we wouldn't limit the filter, limit(0) would keep listening for new events.

Copy link
Owner Author

@f321x f321x Sep 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could just split receiving in two parts. One function that subscribes to the filter and returns the subscription. And another function that pulls events from the subscription (.recv()) channel. So nostr-sdk is the buffer which seems more elegant than spawning tasks.

On the other side, Nostr clients like Amethyst also seem to be able to pull NIP17 events afterwards, i guess they just subscribe to NIP17 events, and then sort them by the created_at in the rumor.

I think we also should consider that traders are most probably not always online at the same time, and go offline between the single steps when waiting for the other party. So we may need to be able to continue at a specific step loading the state from somewhere. Opened issue #14

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your idea of trying to use the nostr-sdk as event buffer, I'll try it out.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think Amethyst does a kind of logic as you mentioned.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I works as you proposed above, subscribing first at start up and then consuming on demand and relying only on the nostr-sdk. Now it is simpler, but I still must clean up a little bit. The escrow client can have a field for the nostr client again :)

Thanks for your comments and ideas, really helpful!

I found also a new problem, some times the escrow client expects a registration but it gets a token message and panics. I could extract this in a new issue, agree? If it doesn't happen (starting first the buyer), it runs successfully so far.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool👍
Yeah let's open an issue for this so we get this PR merged and can work on smaller issues in parallel.

tokio::spawn(
async move { nostr_client_ref.receive_escrow_message(my_pubkey, 10).await },
);

let coordinator_pk = &self.escrow_contract.npubkey_coordinator;
let message = serde_json::to_string(&self.escrow_contract)?;
let contract_message = serde_json::to_string(&self.escrow_contract)?;
dbg!("sending contract to coordinator...");
self.nostr_client
nostr_client
.client
.send_private_msg(*coordinator_pk, &message, None)
.send_private_msg(*coordinator_pk, &contract_message, None)
.await?;

let my_pubkey = self.nostr_client.public_key();
let message = self
.nostr_client
.receive_escrow_message(my_pubkey, 10)
.await?;
self.escrow_registration = Some(serde_json::from_str(&message)?);
Ok(())
let registration_message = reg_msg_fut.await??;
let escrow_registration = serde_json::from_str(&registration_message)?;
f321x marked this conversation as resolved.
Show resolved Hide resolved
Ok(RegisteredEscrowClient {
prev_state: self,
escrow_registration,
})
}
}

pub struct RegisteredEscrowClient<'a> {
prev_state: &'a InitEscrowClient,
escrow_registration: EscrowRegistration,
}

impl<'a> RegisteredEscrowClient<'a> {
/// Depending on the trade mode sends or receives the trade token.
///
/// After this the state is token sent or received.
pub async fn exchange_trade_token(&self) -> std::result::Result<(), anyhow::Error> {
match self.trade_mode {
pub async fn exchange_trade_token(
&self,
nostr_client: &NostrClient,
) -> anyhow::Result<TokenExchangedEscrowClient> {
match self.prev_state.trade_mode {
TradeMode::Buyer => {
// todo: store the sent token in this instance
self.send_trade_token().await?;
Ok(())
// todo: store the sent token in next instance
self.send_trade_token(nostr_client).await?;
Ok(TokenExchangedEscrowClient { _prev_state: self })
}
TradeMode::Seller => {
// todo: store the received token in this instance
self.receive_and_validate_trade_token().await?;
Ok(())
// todo: store the received token in next instance
self.receive_and_validate_trade_token(nostr_client).await?;
Ok(TokenExchangedEscrowClient { _prev_state: self })
}
}
}

/// State change for the buyer. The state after that is token sent.
///
/// Returns the sent trade token by this [`EscrowClient`].
async fn send_trade_token(&self) -> anyhow::Result<String> {
let escrow_contract = &self.escrow_contract;
let escrow_registration = self
.escrow_registration
.as_ref()
.ok_or(anyhow!("Escrow registration not set, wrong state"))?;

async fn send_trade_token(&self, nostr_client: &NostrClient) -> anyhow::Result<String> {
let escrow_contract = &self.prev_state.escrow_contract;
let escrow_token = self
.prev_state
.ecash_wallet
.create_escrow_token(escrow_contract, escrow_registration)
.create_escrow_token(escrow_contract, &self.escrow_registration)
.await?;

debug!("Sending token to the seller: {}", escrow_token.as_str());
debug!("Sending token to the seller: {}", escrow_token);

self.nostr_client
nostr_client
.client
.send_private_msg(escrow_contract.npubkey_seller, &escrow_token, None)
.await?;
@@ -104,21 +115,25 @@ impl EscrowClient {
/// State change for a seller. The state after this is token received.
///
/// Returns the received trade token by this [`EscrowClient`].
async fn receive_and_validate_trade_token(&self) -> anyhow::Result<Token> {
let escrow_contract = &self.escrow_contract;
let client_registration = self
.escrow_registration
.as_ref()
.ok_or(anyhow!("Escrow registration not set, wrong state"))?;
let wallet = &self.ecash_wallet;

let message = self
.nostr_client
async fn receive_and_validate_trade_token(
&self,
nostr_client: &NostrClient,
) -> anyhow::Result<Token> {
let escrow_contract = &self.prev_state.escrow_contract;
let wallet = &self.prev_state.ecash_wallet;

let message = nostr_client
.receive_escrow_message(escrow_contract.npubkey_buyer, 10)
.await?;
wallet.validate_escrow_token(&message, escrow_contract, client_registration)
wallet.validate_escrow_token(&message, escrow_contract, &self.escrow_registration)
}
}

pub struct TokenExchangedEscrowClient<'a> {
_prev_state: &'a RegisteredEscrowClient<'a>,
}

impl<'a> TokenExchangedEscrowClient<'a> {
/// Depending on the trade mode deliver product/service or sign the token after receiving the service.
///
/// The state after this operation is duties fulfilled.
20 changes: 11 additions & 9 deletions client/src/main.rs
Original file line number Diff line number Diff line change
@@ -3,8 +3,10 @@ mod ecash;
mod escrow_client;

use std::env;
use std::sync::Arc;

use anyhow::anyhow;
use async_utility::futures_util::{FutureExt, TryFutureExt};
use cashu_escrow_common as common;
use cli::trade_contract::FromClientCliInput;
use cli::ClientCliInput;
@@ -33,13 +35,13 @@ async fn main() -> anyhow::Result<()> {
let escrow_contract =
TradeContract::from_client_cli_input(&cli_input, escrow_wallet.trade_pubkey.clone())?;
let nostr_client = NostrClient::new(cli_input.trader_nostr_keys).await?;
let mut escrow_client =
EscrowClient::new(nostr_client, escrow_wallet, escrow_contract, cli_input.mode);

escrow_client.register_trade().await?;
debug!("Common trade registration completed");

escrow_client.exchange_trade_token().await?;

escrow_client.do_your_trade_duties().await
let nostr_client_arc = Arc::new(nostr_client);
InitEscrowClient::new(escrow_wallet, escrow_contract, cli_input.mode)
.register_trade(nostr_client_arc.clone())
.await?
.exchange_trade_token(&nostr_client_arc)
.await?
.do_your_trade_duties()
.await?;
Ok(())
}