39 enable remote access for embeddedno std environments#124
Draft
lxsaah wants to merge 30 commits into
Draft
Conversation
…er engine - Introduced `session/mod.rs` with frozen trait signatures for connector-session contracts. - Added `server.rs` implementing the reactive server engine for handling connections. - Created tests in `session_engine.rs` to validate RPC, streaming subscriptions, and fire-and-forget writes using an in-memory transport. - Implemented a channel-backed `Connection`, `Listener`, and `Dialer` for testing purposes. - Developed a line-oriented `EnvelopeCodec` for encoding and decoding messages. - Established an echo dispatch to handle RPC calls and subscriptions, ensuring role-neutral communication.
…ession Phase 3 server-port prep (issue #39). The AimX wire reshape dissolved every seam except one: record.drain needs lazy per-connection cursors, but Dispatch is a shared Arc<D> with &self — nowhere for mutable per-connection state. Split the dispatch role: - Dispatch (Send + Sync, one Arc per server): authenticate + open() factory. - Session (Send, one Box per accepted connection): call/subscribe/write on &mut self. run_session owns the Box<dyn Session> and threads &mut into it. subscribe is defaulted to NotFound (its 'static stream is side-neutral). Additive + object-safe (mirrors the Phase-2 encode_inbound/decode_outbound precedent); recorded in 037. Engine round-trip + AimX client tests still green; contracts still cross-compile to thumbv7em. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… AimxDispatch) Phase 3 server half (issue #39). Replaces the test-local UdsListener + TestDispatch with production server code: - UdsListener: Listener over tokio UnixListener (the accepting transport half, deferred from the client port; reuses the role-neutral UdsConnection). - AimxDispatch<R> (shared) + AimxSession<R> (per-connection): method bodies ported from remote/handler.rs onto the Session seam. record.drain's lazy per-record cursors live in AimxSession. subscribe reuses stream_record_updates; write/record.set reuse set_record_from_json (single-writer-per-key intact). SecurityPolicy enforced per-call (ReadOnly denies; writable_records membership). v2 client param shapes: record.get {name}, record.set {name,value}, write {value}. - build_aimx_server: supervisor.rs's socket setup (remove-stale/bind/chmod) then the spawn-free serve() engine; max_connections/max_subs -> SessionLimits. The aimx_session exit test now stands up a real AimDb and drives the production server end-to-end (hello/get/set/subscribe/write) over a real UDS socket. Legacy handler.rs/supervisor.rs remain untouched (main green); retired in a later stage. record.query is a sync fn returning the 'static handler future (an async fn(&self) would force AimxSession: Sync, which the drain_readers box is not). Welcome's writable_records is derived from the policy directly so build_aimx_server is self-contained standalone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ntConnector) Phase 3 server port (issue #39), client refinements: - Subscribe-ack (run_client): a server subscribe-failure Reply carries an id the client never registered as a pending call. Recognize it and drop the matching event sink so a rejected subscribe ends the stream (None) instead of hanging forever. Contract: success is acknowledged implicitly by events flowing; the server replies only on failure. Covered by a new session_engine test. - pump_client(db, scheme, handle): mirrors records both directions over a running run_client engine — outbound routes stream local updates via ClientHandle::write; inbound routes subscribe and produce into local records through the Router (arbiter path; single-writer-per-key intact). Uses the type-erased consumer/serializer + producer/deserializer machinery (db.runtime_any() supplies the ctx for context-aware (de)serializers). - AimxClientConnector: a ConnectorBuilder for the `aimx://` scheme so records can .link_to/.link_from an AimX peer; on build it dials via run_client and returns the pump futures + engine future for the runner (spawn-free). The registerable wrapper around pump_client, mirroring build_aimx_server on the server side. New aimdb-client test mirrors a record client->server and server->client through the real connector path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the typed wrappers tools/aimdb-cli + tools/aimdb-mcp need before they can migrate off the legacy AimxClient: drain_record(+_with_limit) -> DrainResponse, query, graph_nodes/edges/topo_order, reset_stage_profiling, reset_buffer_metrics (all over the cheap-clone ClientHandle). DrainResponse now lives in engine.rs so it survives connection.rs's removal in the next stage. Additive only — no caller switches yet, so main stays green. Covered by the aimx_session production-server test (graph + drain). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Updated dependencies in Cargo.toml to include futures crate. - Replaced instances of AimxClient with AimxConnection in graph, record, and watch commands in aimdb-cli. - Modified live output formatting to accommodate changes in event structure. - Refactored connection handling in aimdb-mcp to use AimxConnection instead of AimxClient. - Ensured all relevant functions and tests are updated to reflect the new connection structure.
- Simplified `WebSocketConnectorImpl` by removing the `raw_payload` field and related logic, delegating payload framing to the `WsCodec`. - Introduced `WsDispatch` and `WsSession` for handling WebSocket connections, separating concerns for dispatching messages and managing session state. - Added `WsServerConnection` to manage WebSocket connections on the server side, including multi-topic subscription handling. - Retired the previous session management logic in favor of a more modular approach using `run_session`. - Implemented a new `SnapshotProvider` trait for late-join functionality, allowing clients to receive the current state of topics upon subscription. - Enhanced the transport layer with `WsDialer` and `WsClientConnection` for client-side WebSocket handling. - Updated server state management to include shared dispatch and client manager. - Added tests for multi-topic subscription handling and ensured compatibility with existing message protocols.
…et client and codec
…o adapter integration
… and client integration
… up documentation in server.rs
- Updated the client and server engines to use `async-channel` instead of `tokio` channels, enabling a no_std compatible design. - Changed duration fields in `ClientConfig` from `Duration` to `u64` milliseconds for better compatibility with no_std. - Introduced a `TimeOps` trait for runtime-specific time operations, allowing the engines to remain agnostic of the underlying runtime. - Modified the `run_client` and `run_session` functions to accept a `TimeOps` parameter, facilitating reconnect backoff and keepalive functionality. - Added a minimal `TestClock` implementation for testing purposes, ensuring the engine can be driven without a full runtime. - Created a new smoke test for the Embassy adapter to validate the runtime-neutral client engine. - Updated WebSocket connector to utilize the new `TimeOps` interface for reconnect and keepalive configurations.
- Introduced `aimdb-uds-connector` crate for Unix-domain socket transport. - Replaced legacy remote access implementation with `SessionClientConnector` and `SessionServerConnector`. - Updated `AimDbBuilder` to use `UdsClient` and `UdsServer` for remote access configuration. - Enhanced `AimxCodec` to support `no_std + alloc` features. - Refined server dispatch and connection handling to improve modularity and maintainability. - Updated integration tests to utilize the new UDS connector.
- Added `from_query` method to `ConnectorConfig` for building configurations from URL query parameters, allowing for dynamic setting of `timeout_ms` and passing other options verbatim. - Updated `Cargo.toml` to include `aimdb-core/connector-session` in the `std` feature for improved functionality. - Refactored `MqttConnectorImpl` to utilize `pump_sink` and `pump_source` for handling inbound and outbound data, streamlining the connection and subscription process. - Introduced `MqttSink` and `MqttEventLoopSource` to encapsulate publishing and event loop handling, respectively, improving code organization and clarity. - Removed deprecated methods and unnecessary complexity in the MQTT connector implementation. - Deleted outdated design document on frozen connector-session contracts and added a new design document outlining the architecture for remote access via connectors.
…connector and session engine integration
- Updated comments in `pump.rs` to enhance clarity and remove redundant references to documentation sections. - Simplified and clarified comments in `server.rs`, focusing on the reactive server engine and its components. - Improved documentation in `join.rs` to provide clearer references to functions and their usage. - Clarified comments in `typed_record.rs` regarding the producer service function. - Enhanced documentation in `lib.rs` of the UDS connector to better explain its purpose and functionality. - Streamlined comments in `transport.rs` of the UDS connector, emphasizing the role of the connection and listener. - Improved clarity in authentication comments in `auth.rs` of the WebSocket connector. - Refined comments in `codec.rs` of the WebSocket connector to clarify the per-connection codec's functionality. - Updated comments in `connector.rs` of the WebSocket connector to clarify the routing of inbound writes. - Enhanced documentation in `dispatch.rs` of the WebSocket connector to clarify the dispatch and session handling. - Improved clarity in `lib.rs` of the WebSocket connector regarding the shared codec and transport modules. - Streamlined comments in `transport.rs` of the WebSocket connector to clarify the purpose of transport adapters.
25 tasks
…ncy across modules
…improve documentation
…caching for outbound messages
… message processing
- Add ClientManager for handling subscriptions and broadcasting messages to clients. - Introduce WsBusSink for routing serialized record updates to the ClientManager. - Create WsDispatch for managing WebSocket sessions and handling authentication. - Implement HTTP server with WebSocket upgrade and health check endpoints. - Add StreamableRegistry for managing schema names of streamable types. - Define reusable session handler traits for query handling and snapshot provision. - Implement tests for ClientManager and StreamableRegistry functionalities.
…Socket client example formatting
…ation for point-in-time reads
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Issue #39 asks for AimX remote access on embedded (
no_std/Embassy). Rather than bolt on a parallelRemoteTransporttrait, this PR pursues convergence: it builds one shared, runtime-neutral session substrate in the connector layer and ports every existing remote stack onto it. The four hand-rolled networking stacks — AimX server + client and WebSocket server + client, each re-implementing bind/accept/connect, sessions, framing, and RPC — collapse onto two engines (one reactive server, one proactive client), after which embedded transports fall out for free.Design doc:
docs/design/remote-access-via-connectors.md. Phases 0–6 land here; Phase 7 (on-target validation) is tracked separately.What's new
aimdb-core::session— the shared substrate (featureconnector-session,no_std + alloc, runtime-neutral):Connection/Listener/Dialer), codec (EnvelopeCodec), dispatch (Dispatch+ per-connectionSession), over a role-neutralInbound/Outboundmessage set withPayload = Arc<[u8]>.serve(accept loop) +run_session(per-connection biased select: RPC + streaming subscriptions + writes), spawn-free, honoringSessionLimits.run_clientreturns a cheap-cloneClientHandle(call/subscribe/write) + the engine future; reply demux by id, reconnect backoff, idle keepalive, bounded offline queue. Its only runtime dependency is the adapter'sTimeOpsclock.pump_sink/pump_sourceextract the boilerplate every data-plane connector hand-rolled.SessionClientConnector/SessionServerConnectorwrap the engines onto theConnectorBuilderspine, so a transport crate contributes only itsDialer/Listener/Connectiontriple under a configurable scheme.session::aimx— the AimX-v2 NDJSON protocol:AimxCodec(no_std + alloc) +AimxDispatch(std-only). The hand-rolledremote/handler.rs(1,759 lines) andremote/supervisor.rsare deleted; their behavior is now the engine + dispatch.aimdb-uds-connector(new crate) — the UDS transport (UdsConnection/UdsDialer/UdsListener) withUdsServer/UdsClientsugar. UDS socket setup relocated out of core.Ports onto the substrate:
serve/run_session+run_client(newcodec/transport/dispatchmodules; the 704-lineclient/connector.rsloop deleted;client_manager/sessionslimmed to a fan-out bus). Wire-identical, gated by a round-trip test.pump_sink/pump_sourcewith per-route config from the link URL query.aimdb-clientAimxClient→AimxConnection, rebuilt onrun_clientover the UDS dialer; CLI/MCP updated to match.AimDbBuilder::with_remote_access(config)removed. Register a connector instead:.with_connector(aimdb_uds_connector::UdsServer::from_config(config)).aimdb_client::AimxClient→AimxConnection(engine-based);subscribe()now returns a stream (no server-allocated subscription id).Testing
session_engine.rs): RPC + streaming + write round-trip across the channel-backed substrate, failed-subscribe stream termination, and per-connection subscription-cap reaping.pump_clienttests; Embassysession_smokeproving the engines run on the EmbassyTimeOpsclock; tokio drain tests migrated toUdsServer.cargo docbuilds clean (-D warnings) foraimdb-coreand the three connector crates.Tracked follow-ups (open issues)
#120no_std AimX server port ·#121TCP connector ·#122serial connector ·#123transport-agnostic host client +--connect <url>resolver ·#41dropped-event tracking ·#13perf validation (the Phase-7 gate).