Dealing with unreliable connections

[ignatz]

(context: TrailBase maintainer here)

I was recently thinking a bit more about providing stronger guarantees in the light of "event loss". The TrailBase integration is pretty naive, uses SSE and if it gets auto-reconnected state would likely diverge.

To my understanding electric also uses SSE (https://electric-sql.com/docs/api/http#server-sent-events-sse), which is generally pretty opaque when it comes to managing connection state. After taking a quick look at its current implementation, it's not clear to me if there's maybe some magic happening under the hood (e.g. invalidate and re-build state from scratch if sequence numbers get out of sync 🤷‍♀️). Does electric's integration offer any guarantees?

I was mostly wondering what the official stance is for:

• Are any guarantees pushed down to the transport?
• Even if the transport is reliable but gets fully disconnected, what's the best-practice to re-sync/fix-up collections on the developer's side?

[ignatz]

Friendly ping. Still curious to learn what electric and others do when SSE events get lost.

Meanwhile, I've added WebSocket support to TrailBase, which at least let's clients explicitly handle disconnects. Now I'm wondering if there's maybe a way to reset TanStack/DBs internal state machine to "initializaing" (similar to markReady just the other way round :) ) - Thanks

[ignatz]

Friendly ping. @KyleAMathews sorry for the personal nudge, would love to learn how electric thinks about and deals with SSE package loss 🙏

[KyleAMathews]

SSE modes sends regular updates about the latest offset so if the client disconnects for whatever reason, the client knows where they read to and can reconnect at that point.

[KyleAMathews]

yeah this is handled internally in the electric client.

[KyleAMathews]

There's a truncate event that collections can send that can be used for starting over.

[ignatz]

If i had a transport where disconnects are detected but no "truncate", could i reset TanStack/db to refetch as if it was just instantiated or is the expectation that all integrations should implement a catch-up internally?

[KyleAMathews]

yeah DB has no opinions about how sync happens. truncate is just for when you need to tell db to delete all data in a collection and restart.

[ignatz]

Oh is see now, truncate isn't internal to electric but exactly what i was asking for, i.e. a way to reset DB when an interruption was detected. This would allow TrailBase to ensure consistency by either adding a sequence number to SSE events or switching to WebSocket. Awesome - much appreciated