Does Cap'n Web support Websocket Hibernation

[natanparmigiano-paganinitech]

Hello!

First of all, amazing work, we are considering switching out tRPC for Cap'n Web for our next projects!

But an important question was raised internally:

When using Websockets in CF Workers, does Cap'n Web handle websocket hibernation by default?

Is there any example on how to implement hibernation (with DO) together with Cap'n Web?

[kentonv]

At this time, it does not. But this is definitely something we want to fix!

[kentonv]

To add a bit of detail here, I don't think the solution is for Cap'n Web itself to try to support hibernation, but rather for us to extend the way hibernation works in the Workers Runtime, so that RPC stubs are hibernatable. What you'd do then is use Cap'n Web between a browser and a stateless worker, which in turn uses Workers built-in RPC from there to the DO. Workers RPC would then need to support hibernating RpcTargets and storing stubs through hibernation.

[Sillyvan]

Just looked this up and wondered if i could use Cap'n'Web to basically get a typed WS between the client and my DO with hibernation.

Also saw this #110

Do i understand correctly that:
You can pass a DO as your "RPC Target"
But no matter what you would not get ws hibernation?

[kentonv]

Not sure I exactly understand the question.

However: Holding a DO stub does not prevent hibernation. So if, in your stateless routing worker, you return the DO stub itself to the client over Cap'n Web, that in itself shouldn't prevent hibernation. It's only while calls are active that it won't be able to hibernate. Or if you hold stubs pointing at functions or RpcTarget objects in the DO's isolate.

[SamJB123]

So after:

• falling in love w/ capnweb,
• then hitting the no-hibernation problem,
• moving away to using direct-websocket rpc, running into performance/reliability issues with direct-websocket rpc in a distributed system, getting lovesick, and coming back..

I've had a crack at this problem: https://github.com/SamJB123/capnweb-experiments/tree/hibernation! Test gui here: https://capnweb-hibernation-test-worker.sam-e96.workers.dev/.

It (poetically) works by using magic map. It does so to capture the stuff needed to reconstruct targets (negative exports) and stubs (negative imports) after hibernation.

It needed one protocol change: on stub calls (positive imports/exports), the caller (importer) needs to pass their import ID to the callee (exporter), who uses it as the source of truth. Seems safe to do so for positive imports/exports (different story with negatives).

Sessions use the hibernating socket api, with the data needed to restore from hibernation stored via serializeAttachment. So yeah the experiment monopolizes that atm. Real implementation needs to store stuff in a way that lets downstream devs use serializeAttachment normally.

It's definitely got junk code and issues I haven't found yet, but after 18 hours and having it now actually work, had to share in case it helps @kentonv / others.

One known limitation: proxy passthrough. Proxied workers rpc stubs instantiated by another durable object block hibernation still. But:

• proxy capabilities work great if they're instantiated by the durable object the client is directly connected to (i.e., use a 'user hub' DO as an agent/entry point for the user, by spinning up capabilities that let the user call methods of other DOs over RPC). I plan to use this pattern: every user connects to a DO instance unique to their user ID, which then brokers connections to a wider web. This complicates bidirectional callback to the client. But a user-hub pattern means other DOs can basically call a client using their personal DO instance address.
• root durable object stubs aren't a problem either. I haven't tested workerentrypoint stubs, but given DO stubs work, I'd expect them to play nice as well.

[kentonv]

Neat!

It needed one protocol change: on stub calls (positive imports/exports), the caller (importer) needs to pass their import ID to the callee (exporter), who uses it as the source of truth. Seems safe to do so for positive imports/exports (different story with negatives).

Hmm, why wouldn't you store the current ID in the attachment along with the other session state?

In any case, this seems like a cool hack, though a bit different from what I intended as the long-term approach. The solution I'm hoping to build is:

• You would terminate the Cap'n Web session from the browser in a Worker, not a DO. Then you do RPC to a DO on from there. So the DO perceives everything as regular Workers RPC, doesn't know anything about Cap'n Web. This part works great today -- but once you've passed through a stub to or from the DO, it can block the DO from hibernating.
• A DO would be able to create and return RPC stubs that are marked in such a way that the system knows how to recreate them after hibernation, so their existence doesn't block hibernation.
• A DO would also be able to store outbound RPC stubs into a space that survives hibernation, e.g. to maintain a list of current subscriber callbacks.

This would all compose nicely to allow hibernation with RPC in general, and Cap'n Web in particular.

[SamJB123]

Hmm, why wouldn't you store the current ID in the attachment along with the other session

Would certainly work too - was 50/50 split on this. I experimented with this after thinking about a future scenario with long-lived sessions where resilience starts to become a louder concern and both sides of the ledger might be able to hibernate (or client equivalent), not just one. Realised you could either let the positive IDs essentially restart after one party hibernates or accept that in long-lived, high-frequency sessions you could get to v highly incremented IDs. Then noticed that the system worked, even for promise pipelines etc, so long as the exporter and importer IDs matched (sequential not intrinsically needed) and that exporters provided their ID back anyway - so figured "why not move the ID-passing one step sooner to make things explicit".

In any case, this seems like a cool hack, though a bit different from what I intended as the long-term approach. The solution I'm hoping to build is:

• You would terminate the Cap'n Web session from the browser in a Worker, not a DO. Then you do RPC to a DO on from there. So the DO perceives everything as regular Workers RPC, doesn't know anything about Cap'n Web. This part works great today -- but once you've passed through a stub to or from the DO, it can block the DO from hibernating.
• A DO would be able to create and return RPC stubs that are marked in such a way that the system knows how to recreate them after hibernation, so their existence doesn't block hibernation.
• A DO would also be able to store outbound RPC stubs into a space that survives hibernation, e.g. to maintain a list of current subscriber callbacks.

This would all compose nicely to allow hibernation with RPC in general, and Cap'n Web in particular.

Yeah I love this. After reading your earlier post touching on hibernating rpctargets, it is exactly what I'm working to build my actual architecture around right now (as opposed to direct websockets to every DO instance). The 'user hub' DO model I mentioned is a stopgap - minting per-client capabilities for methods on other DOs that survive hibernation. Designing it to be easily swapped to an ephemeral worker when other DOs can mint those capabilities themselves and still hibernate.

[SamJB123]

Update - after the exchange here I started thinking about how hibernating websockets could better align with the long-term vision for capnweb / hibernating RpcTargets (I want to build architecture now that's ready for it in the future!).

Have found something that seems to be working great - where clients connect to a worker (not a DO directly!), and the hack I shared above is used not at the client boundary, but instead between the worker and the durable object! Check it out.

I've made it a template available here for those who want to experiment too: https://github.com/SamJB123/tanstack-start-cloudflare-websockets. Pushed the experimental fork to npm to make that easy for people.

That template also carries a pattern for plugging tanstack start's server functions into capnweb's websocket connection as well, since the app is running on the same worker the user is connected to. Usually they require HTTP fetches. The result is simpler server functions. For example for things like authentication, you could now verify them once when opening the websocket connection, and automatically have that context on every server function (including route preloading etc) after that.