Feature Request: Investigate Use of Existing Impl Pathfinding

[carlaKC]

Describe desired feature

When running with a fully simulated network, it would be amazing of SimLN could use the same pathfinding as LND/CLN/Eclair. LND would be the highest priority, because it's the majority of the network.

Possible Approaches

Extern C

• Create C libraries that implement each pathfinding algorithm
  • CLN: easy, compile library in C
  • LND: use c-shared to build C shared library
  • Eclair: not possible, so far as I know
• Use extern to call C code for each pathfinding

This would require some (likely painful) investigation to figure out if the c-shared approach would work for LND. There are all sorts of caveats about C-compatible types that I haven't looked into.

IPC

• Create "stub" servers that call directly to the implementations routing code:
  • Listen on socket, accepting parameters expressed in json
  • Run pathfinding
  • Return result
• SimLN then spins up each of these small servers using Command
• Implement the Pathfinder trait in Add pathfinder trait #273 which is responsible for shooting the request into the socket

Notes:

• I don't have strong opinions on how to handle IPC (never done this myself)
• We could also define a common RPC for these mini-servers, but that's a lot of boilerplate
• Main priority is that this should be ✨ seamless ✨ for the end user, SimLN should handle all the ugly stuff.

Use case for feature

A common use case for SimLN's random payment generation is to create "background noise" payments in a network, and then take and measure some action within that busy network.

Would you like to contribute code for this feature?

I don't have capacity to take this on right now, but would be happy to participate in design discussions and review.

[carlaKC]

IPC seems like a better approach to me because:

• We can support all LN impls
• It doesn't involve painful binding voodoo

I mostly wrote up extern c because it ✨ looks cool ✨, but that's not a very strong motivation.

[carlaKC]

Had a few more thoughts about how we could achieve this using the IPC option (with some specifics for LND).
Going to skim over the IPC part, because that's just plugging two streams of data together.

Key design goals we should keep in mind:

• We want separate payment history tracking per node, because a nodes sharing a liquidity view of the network isn't realistic
• We ideally want to share read only things like the network graph, so that we don't have to duplicate it in memory

---

In the LND context, this would probably look something like one big: map[pubkey]ChannelRouter:

• Each node has their own MissionControl, which is responsible for tracking payment history/results.
• The nodes share the same implementation of the Graph interface, so that we don't have to duplicate it.

This is mostly exposed publicly in LND afaik:

• New ChanneRouter
• Config
• FindRoute
• Report payment outcomes

What seems like it'll be the most painful is the persistence piece - we might need to stub out some things to keep everything in memory.

Overall flow would be:

• Startup:
  • Start SimLN with graph
  • Start LND with graph and list of "LND algorithm using" nodes
• LND stub:
  • Creates one Graph impl from the SimLN graph
  • Creates one ChannelRouter per node
• Find route:
  • SimLN calls pathfinding trait
  • Trait impl sends IPC query to LND stub (including source node)
  • LND stub looks up source node in map
  • LND stub calls FindRoute on ChannelRouter
  • LND stub sends IPC query back to SimLN trait impl
  • Trait impl provides the route to SimLN
    (same for reporting payment success/failure)

[f3r10]

I have created a POC with the suggestions. https://github.com/f3r10/sim-ln-plumbing. Still need to keep the connection with sim-ln.