Add support for SN running multiple lightning nodes

[huumn]

Because most lightning implementations have poor support for clustering, afaict the best way to achieve an HA lightning service is to run multiple lightning nodes (and we can share routes/liquidity via a large private channel between them).

To achieve this we'll want to:

1. map each of our LN node's pubkeys to an autheticated grpc object/session
2. any function that takes a lightning invoice, and performs actions/queries on the invoice using the GRPC connection, will connect to the correct LN node using the invoice's pubkey
3. for each LN node, we'll need to run invoice and payment subscriptions

[ekzyis]

How do we decide from which node we create an invoice? Just randomly with code in our application code? Or outsourced to nginx gRPC load balancing?

Load balance a cluster of gRPC services, using Round Robin, Least Connections, or other methods to distribute calls across the cluster. You can then scale your gRPC‑based service when you need additional capacity.

we can share routes/liquidity via a large private channel between them

But if one node goes down, we lose all routes through the channels only this node had, right? So we would also need to decide which nodes of us have open channels to which other external nodes afaict

[huumn]

How do we decide from which node we create an invoice?

We'll use our primary node by default. This other node is just a backup during time of maintenance or other issues.

But if one node goes down, we lose all routes through the channels only this node had, right?

Yes.

So we would also need to decide which nodes of us have open channels to which other external nodes afaict

What?

[ekzyis]

We'll use our primary node by default. This other node is just a backup during time of maintenance or other issues.

Ah, I see

What?

I just wanted to say that we either have to open the same channels on both nodes or one node will have degraded connectivity/liquidity when the other goes down. But I wasn't considering using a backup to be highly available, only load balancing to achieve HA (equal nodes that are equally used). In the former case, it makes sense that the backup node only has "essential channels".