Notes from Boltz integration #689

spacebear21 · 2025-05-07T21:39:48Z

spacebear21
May 7, 2025
Maintainer

I've hit some roadblocks in the Boltz <> Payjoin integration.

The first major one is that I'm no longer able to run the Boltz environment locally after a machine restart...

The second set of issues relates to implementing session persistence for Boltz using the 0.23 Persister design and fitting it into Boltz's backend architecture. I know #675 already addresses some (maybe all?) of these points but Dan suggested I document my developer experience somewhere, and maybe this can help inform design decisions for 0.24 persistence

The current Persister trait design creates several constraints.

pub trait Persister<V: Value> {
    type Token: From<V>;
    type Error: std::error::Error + Send + Sync + 'static;
    fn save(&mut self, value: V) -> Result<Self::Token, Self::Error>;
    fn load(&self, token: Self::Token) -> Result<V, Self::Error>;
}

/// Reference implementation (from payjoin-cli)
pub(crate) struct ReceiverPersister(Arc<Database>);
impl ReceiverPersister {
    pub fn new(db: Arc<Database>) -> Self { Self(db) }
}

impl Persister<Receiver> for ReceiverPersister {
    type Token = ReceiverToken;
    type Error = crate::db::error::Error;
    fn save(&mut self, value: Receiver) -> std::result::Result<ReceiverToken, Self::Error> {
        let recv_tree = self.0 .0.open_tree("recv_sessions")?;
        let key = value.key();
        let value = serde_json::to_vec(&value).map_err(Error::Serialize)?;
        recv_tree.insert(key.clone(), value.as_slice())?;
        recv_tree.flush()?;
        Ok(key)
    }
    fn load(&self, key: ReceiverToken) -> std::result::Result<Receiver, Self::Error> {
        let recv_tree = self.0 .0.open_tree("recv_sessions")?;
        let value = recv_tree.get(key.as_ref())?.ok_or(Error::NotFound(key.to_string()))?;
        serde_json::from_slice(&value).map_err(Error::Deserialize)
    }
}

Since save(&mut self, value: Receiver) only takes a Receiver value, any external data that needs to be saved alongside the receiver session must live on the struct itself. For example, in Boltz we want to save a swap ID for a given Receiver session. This requires introducing an intermediate Manager for individual payjoin sessions, which exposes external data such as swap_id on itself (note that the reference implementation doesn't do this, and instead provides one generic Persister for all sessions).

/// Manager for all payjoin sessions
pub struct PayjoinManager {
    ohttp_keys: OhttpKeys,
    btc: Currency,
    payjoin_repo: Arc<PayjoinSessionHelperDatabase>,
}

impl PayjoinManager {
    async fn receive_payjoin<'a>(
        &self,
        address: Address,
        //swap: Swap,
        amount: Option<Amount>,
        label: Option<String>,
    ) -> Result<payjoin::PjUri<'a>> {
        let new_receiver =
            NewReceiver::new(address.clone(), DIRECTORY, self.ohttp_keys.clone(), None)?;
        let mut persister = PayjoinSessionManager {
            db: self.payjoin_repo.clone(),
            swap_id: "SWAPID".to_string(),
        };
        let token = new_receiver
            .persist(&mut persister)
            .map_err(|e| anyhow!("Failed to persist receiver: {}", e))?;

        let receiver = Receiver::load(token, &persister).expect("Failed to create receiver");

        let pj_uri = pj_uri_with_extras(&receiver, amount, label)?;
        tokio::spawn(spawn_payjoin_receiver(receiver, address, self.btc.clone()));
        info!("Request Payjoin by sharing this Payjoin Uri: \n{}", pj_uri);
        Ok(pj_uri)
    }
}

/// Manager for a given payjoin session
struct PayjoinSessionManager {
    db: Arc<PayjoinSessionHelperDatabase>,
    swap_id: String,
}

impl Persister<Receiver> for PayjoinSessionManager {
    type Token = ReceiverToken;
    type Error = SessionError; // hack
    fn save(&mut self, value: Receiver) -> std::result::Result<ReceiverToken, Self::Error> {
        let token = value.key();
        let payjoin_session = PayjoinSession {
            id: token.to_string(),
            json: serde_json::to_string(&value).expect("should be valid json"),
            swapId: self.swap_id.clone(),
        };
        self.db.insert(&payjoin_session).unwrap();
        Ok(token)
    }
    fn load(&self, key: ReceiverToken) -> std::result::Result<Receiver, Self::Error> {
        let payjoin_session = self.db.get_by_id(&key.to_string()).unwrap();
        let receiver = serde_json::from_str(&payjoin_session.json).unwrap();
        Ok(receiver)
    }
}

/// This is the actual database representation of a payjoin session (they use PostgreSQL with diesel as ORM)
#[derive(
    Queryable, Selectable, Insertable, AsChangeset, Associations, PartialEq, Default, Clone, Debug,
)]
#[diesel(belongs_to(Swap, foreign_key = swapId))]
#[diesel(table_name = crate::db::schema::payjoinSessions)]
#[allow(non_snake_case)]
pub struct PayjoinSession {
    pub id: String,
    pub swapId: String,
    pub json: String,
}

While this approach works, it feels a bit like a compromise to satisfy Persister's constraints. It's tempting to bypass the payjoin Persister by using the NoopPersister and to simply initiate PayjoinSession and make the DB insert call in receive_payjoin:

    async fn receive_payjoin<'a>(
        &self,
        address: Address,
        //swap: Swap,
        amount: Option<Amount>,
        label: Option<String>,
    ) -> Result<payjoin::PjUri<'a>> {
        let new_receiver =
            NewReceiver::new(address.clone(), DIRECTORY, self.ohttp_keys.clone(), None)?;
        let token = new_receiver
            .persist(&mut NoopPersister)
            .map_err(|e| anyhow!("Failed to persist receiver: {}", e))?;
        let receiver = Receiver::load(token, &mut NoopPersister).expect("Failed to create receiver");

        let payjoin_session = PayjoinSession {
            id: token.to_string(),
            json: serde_json::to_string(&receiver).expect("should be valid json"),
            swapId: self.swap_id.clone(),
        };
        self.db.insert(&payjoin_session).unwrap();

        let pj_uri = pj_uri_with_extras(&receiver, amount, label)?;
        tokio::spawn(spawn_payjoin_receiver(receiver, address, self.btc.clone()));
        info!("Request Payjoin by sharing this Payjoin Uri: \n{}", pj_uri);
        Ok(pj_uri)
    }

Also, this design doesn't lend itself to storing specific attributes, e.g. an expires_after field that could be used to cleanup expired sessions with SQL UPDATE, because the session context on Receiver is private. It generally seems to assume simple k-v storage where implementers are expected to serialize the Receiver as a whole and store it as a single value.
Although load is presumably the "right" way to construct a Receiver, there are common cases where it's undesirable to do so. For example, bulk resuming all active sessions from the database by calling load on every session individually would be inefficient, and it would be better to load all Receiver's in a single SELECT.
Finally, some other thoughts not related to persistence directly but rather to our reference implementation and the purpose it serves. I think it should provide much better error handling examples, since hard panics are unacceptable in most contexts (e.g. running an exchange). It improved in this this area with recoverable error handling, but doesn't provide examples of a) what to do after responding with a recoverable error b) when should a receiver give up on a payjoin and broadcast the fallback tx c) what should the sender do with a recoverable error...

DanGould · 2025-05-07T21:51:13Z

DanGould
May 7, 2025
Maintainer

Finally, some other thoughts not related to persistence directly but rather to our reference implementation and the purpose it serves. I think it should provide much better error handling examples, since hard panics are unacceptable in most contexts (e.g. running an exchange). It improved in this this area with recoverable error handling, but doesn't provide examples of a) what to do after responding with a recoverable error b) when should a receiver give up on a payjoin and broadcast the fallback tx c) what should the sender do with a recoverable error...

I believe @thebrandonlucas is going to be champion of this payjoin-cli recoverable error handling, but I don't think this concrete explanation of the issue has been brought up yet. Glad you chose to persist it too.

0 replies

arminsabouri · 2025-05-08T16:52:11Z

arminsabouri
May 8, 2025
Collaborator

Thanks for writing this up! Couple comments

Since save(&mut self, value: Receiver) only takes a Receiver value:

In the Session Event Log (SEL) design, things work a bit differently. The persister trait still only accepts specific values—namely, session events. However, for external metadata that is application-specific, we recommend storing that in the database representation of the session.

For example, in pj-cli, we have SessionWrappers that include a session ID (for Boltz, this might be a swap_id), a closed boolean indicating whether the session is still pending, and a list of session events. This parent SessionWrapper is where we intend to store metadata like created_at or swap_id.

To query session-specific details, we’ve created a SessionHistory. Under the hood, SessionHistory consumes the persister and replays all session events. After replaying, you can query details like expired_at or pj_uri.

In a SQL setup, I imagine this would take the form of a PayjoinReceiverSessions table with a primary key that’s either auto-incrementing or application-specific, and a PayjoinReceiverSessionEvents table with a foreign key linking to PayjoinReceiverSessions. Alongside the FK, PayjoinReceiverSessionEvents should include a BLOB column for serialized data and another column to help sort events when they are loaded for replay.

Also, this design doesn't lend itself to storing specific attributes, e.g., an expires_after field that could be used to clean up expired sessions with SQL UPDATE, because the session context on Receiver is private.

This is a good point. I think we can extract that value and store it in PayjoinReceiverSessions as we process the session. Alternatively, a less efficient way to obtain this value is to replay the logs, which gives you access to SessionHistory, where you can call the expired_at method. The first approach is more efficient (in terms of serialization/deserialization), but the latter is guaranteed to work with all future releases of rust-payjoin and requires less developer intervention.

Recoverable error handling:

This is something I’m also unclear on and hoping to clarify with SEL. After skimming the list of receiver replayable errors, many seem like terminal errors where the only viable action is to broadcast the fallback transaction. For example, ReplayableError::InputsSeen or ReplayableError::MissingPayment—in these cases, there appears to be no meaningful recovery.

The sender could repost a new original proposal that might resolve the issue, but as you mentioned, we don’t implement this in our reference, and I’m not sure if reposting is even a good idea. Regarding the fallback transaction: in SEL, we want to design it so that (1) we log this as a session event and (2) mark the session as closed.

0 replies

DanGould · 2025-05-08T18:30:47Z

DanGould
May 8, 2025
Maintainer

Notes from call this afternoon May 8 on this topic

do we need an expires_after field

No. Load sessions in until they're marked closed in db using the .close() function, then ignore those. Any additional data added to DB, not from -in-memory session is an optimization.

@nothingmuch:

1 major change
original 0.23 persister was for all sessions and payjoin library chose session ID as a hash of URI
In the new version persisters are per-session where the application associates a session id that
the application itself implements.

@spacebear21:

have you thought about loading 100 sessions and loading one shot?

@nothingmuch: the application would still be responsible for grouping these together. Main
consideration is that now closing of sessions is explicit.

@spacebear21:

if u have active session and go offline, for example by shutting off the receiver, would you still have
to get sessions and bring them into memory in order to get the status?

@nothingmuch

You'd recover the session: replay the typestate for "active" sessions based on logs. recovery would realize
expiration has elapsed. We could expose expiration as a high level part of a persister trait but this
seems like a micro optimization. Replaying the log is sufficient.

the first time a session is "active" but expired, the close method gets called when expiration is detected.
The best practice for the close method is to leave a db session closed IN THE DATABASE so it doesn't have to be loaded again

At the time of close, you could read view details into history and save it to DB. BUT that's an optimization over just reading the Receiver into memory and using that for data.

@Spacebear

"In a SQL setup..." paragraph in above comment w/ additional BLOB works as a solution.

The new design addresses pretty much all issues I brought up in gist. In order to move forward, should we
wait for 0.24? do persistence manually until session event log ships? It seems like if I go with 0.23 I'll
have to make changes for session event log to be compatible with 0.24.

This is a good opportunity before 0.24 to check it into the Boltz x Payjoin PR as a sort of integration
test / proof of concept to verify that the design works.

@0xBEEFCAF3

the receiver persistence is now sketched out. bindings tests pass. e2e tests pass
now the pattern is being applied to the sender.

I wonder when bindings migration would happen

@DanGould

let's address this after we get the session event log in. One thing at a time. little's law applies.

@spacebear21

For cake and bullbitcoin we should do the 0.23 impl wire proto upgrades w/ NoOpPersister
Put Boltz aside to focus on getting Cake, BullBitcoin on the same page and Cake published.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Payjoin Dev Kit

Notes from Boltz integration #689

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 3 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Payjoin Dev Kit

Notes from Boltz integration #689

Uh oh!

Uh oh!

spacebear21 May 7, 2025 Maintainer

Replies: 3 comments

Uh oh!

DanGould May 7, 2025 Maintainer

Uh oh!

arminsabouri May 8, 2025 Collaborator

Uh oh!

DanGould May 8, 2025 Maintainer

Notes from call this afternoon May 8 on this topic

spacebear21
May 7, 2025
Maintainer

DanGould
May 7, 2025
Maintainer

arminsabouri
May 8, 2025
Collaborator

DanGould
May 8, 2025
Maintainer