feat: experimental MCP analytics (PostHog::MCP) for the Ruby mcp gem - #269
feat: experimental MCP analytics (PostHog::MCP) for the Ruby mcp gem#269rafaeelaudibert wants to merge 9 commits into
Conversation
Wrap an MCP::Server so every tool call, handshake, listing, prompt, resource read, and failure is captured as a $mcp_* event with the same wire contract as @posthog/mcp and posthog.mcp. Ships inside posthog-ruby behind `require 'posthog/mcp'`; the mcp gem is a peer dependency. - PostHog::MCP.instrument(server, client = nil, **options); the client falls back to the posthog-rails PostHog.client facade - Prepends MCP::Server#handle_request to strip injected arguments, time the call, and record results/errors without ambient state; Fiber[] storage only for the Streamable HTTP transport hop - Context/intent injection, conversation ids, get_more_tools, stateless Mcp-Session-Id tokens, llm_model capture, identify, before_send, event_properties, sanitization, PII redaction, truncation - PostHog::MCP::Client for custom dispatchers and PostHog::MCP::RackMiddleware - Private per-event _lib/_lib_version override in Client#capture so MCP events report $lib posthog-ruby-mcp without relabeling the host client - Experimental: warns on require and on instrument Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Docs PR: PostHog/posthog.com#20022 |
posthog-ruby Compliance ReportDate: 2026-09-09 19:07:14 UTC ✅ All Tests Passed!46/46 tests passed Capture Tests✅ 29/29 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
|
Prompt To Fix All With AI### Issue 1
lib/posthog/mcp/truncation.rb:24
**Payload limit drops events**
The MCP truncation budget allows events up to 102,400 bytes, but the core client rejects serialized messages over 32,768 bytes. Tool responses or parameters between those limits pass MCP truncation and are then omitted from the batch, causing valid tool-call analytics to be lost. The final payload, including any `before_send` changes, needs to fit the core message budget.
### Issue 2
lib/posthog/mcp/identity.rb:16-27
**Identity cache is unsynchronized**
`IdentityCache#get` performs an unlocked delete followed by reinsertion, while concurrent requests share this cache and both identification and event capture mutate it. Interleaved requests can temporarily lose an entry, evict the wrong entry, or attach an identity updated by another request. Protect these compound operations with the existing per-server mutex and cover concurrent identification in a test.
### Issue 3
lib/posthog/mcp/rack_middleware.rb:46-49
**Failed initialization mints sessions**
The middleware mints a session token before calling the application and adds it to every response, including rejected or failed initialization responses. A client that retains this header can replay a session created for an initialization that never succeeded. This also differs from the automatic transport, which mints only after successful dispatch. Attach the token only after confirming initialization succeeded.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "feat: add experimental PostHog::MCP anal..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| MAX_DEPTH = 10 | ||
| MAX_BREADTH = 100 | ||
| MAX_STRING_LENGTH = 32_768 | ||
| MAX_EVENT_BYTES = 102_400 |
There was a problem hiding this comment.
The MCP truncation budget allows events up to 102,400 bytes, but the core client rejects serialized messages over 32,768 bytes. Tool responses or parameters between those limits pass MCP truncation and are then omitted from the batch, causing valid tool-call analytics to be lost. The final payload, including any before_send changes, needs to fit the core message budget.
Knowledge Base Used: Event capture SDK
Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/posthog/mcp/truncation.rb
Line: 24
Comment:
**Payload limit drops events**
The MCP truncation budget allows events up to 102,400 bytes, but the core client rejects serialized messages over 32,768 bytes. Tool responses or parameters between those limits pass MCP truncation and are then omitted from the batch, causing valid tool-call analytics to be lost. The final payload, including any `before_send` changes, needs to fit the core message budget.
**Knowledge Base Used:** [Event capture SDK](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/posthog-ruby/-/docs/event-capture-sdk.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
There was a problem hiding this comment.
Fixed in b72f5e0. Valid: the core client drops messages over Defaults::Message::MAX_BYTES (32KB). Budget is now that minus 2KB headroom, and the size pass trims the largest strings before reducing depth. Regression spec asserts every queued message stays under the limit.
| def get(session_id) | ||
| identity = @cache.delete(session_id) | ||
| return nil if identity.nil? | ||
|
|
||
| @cache[session_id] = identity | ||
| end | ||
|
|
||
| def set(session_id, identity) | ||
| @cache.delete(session_id) | ||
| @cache.shift if @cache.length >= @max_size && !@cache.key?(session_id) | ||
| @cache[session_id] = identity | ||
| end |
There was a problem hiding this comment.
IdentityCache#get performs an unlocked delete followed by reinsertion, while concurrent requests share this cache and both identification and event capture mutate it. Interleaved requests can temporarily lose an entry, evict the wrong entry, or attach an identity updated by another request. Protect these compound operations with the existing per-server mutex and cover concurrent identification in a test.
Knowledge Base Used: Event capture SDK
Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/posthog/mcp/identity.rb
Line: 16-27
Comment:
**Identity cache is unsynchronized**
`IdentityCache#get` performs an unlocked delete followed by reinsertion, while concurrent requests share this cache and both identification and event capture mutate it. Interleaved requests can temporarily lose an entry, evict the wrong entry, or attach an identity updated by another request. Protect these compound operations with the existing per-server mutex and cover concurrent identification in a test.
**Knowledge Base Used:** [Event capture SDK](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/posthog-ruby/-/docs/event-capture-sdk.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Fixed in b72f5e0. IdentityCache now has its own mutex and merge! does the read-merge-write atomically; added a 16-thread concurrent identify spec.
| status, headers, body = @app.call(env) | ||
| if token && headers.respond_to?(:key?) && headers.keys.none? { |k| k.to_s.casecmp?(MCP_SESSION_HEADER) } | ||
| headers[MCP_SESSION_HEADER] = token | ||
| end |
There was a problem hiding this comment.
The middleware mints a session token before calling the application and adds it to every response, including rejected or failed initialization responses. A client that retains this header can replay a session created for an initialization that never succeeded. This also differs from the automatic transport, which mints only after successful dispatch. Attach the token only after confirming initialization succeeded.
Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/posthog/mcp/rack_middleware.rb
Line: 46-49
Comment:
**Failed initialization mints sessions**
The middleware mints a session token before calling the application and adds it to every response, including rejected or failed initialization responses. A client that retains this header can replay a session created for an initialization that never succeeded. This also differs from the automatic transport, which mints only after successful dispatch. Attach the token only after confirming initialization succeeded.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Fixed in b72f5e0. The middleware attaches the token only on a 2xx response and clears env['posthog_mcp.session'] otherwise; spec added.
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewThis pull request adds experimental PostHog analytics for applications using the Ruby MCP gem, including request-scoped event capture and session attribution. One issue has been addressed, but a request-scope propagation problem remains on Ruby 3.0 and 3.1. During concurrent requests, child fiber or thread work can fall back to shared session data, allowing an attacker to cause another user’s event properties to be attributed to the attacker’s PostHog session and identity. The impact is limited to analytics attribution, but the remaining race should be fixed before relying on this integration in concurrent production workloads. Open issues (1)
Fixed/addressed: 1 · PR risk: 5/10 |
- Budget truncation under the core client's 32KB per-message limit (which drops larger messages at batch time) and trim the largest strings before reducing depth so big tool responses keep their shape - Make IdentityCache thread-safe and merge identities atomically - RackMiddleware attaches the minted Mcp-Session-Id only on a successful initialize response Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
| @@ -0,0 +1,116 @@ | |||
| # PostHog MCP analytics for Ruby | |||
|
|
|||
| > **Experimental.** `PostHog::MCP` is new and its API, options, and the captured `$mcp_*` event schema may change in a minor release. A one-line warning is logged when you require it. Please report issues at https://github.com/PostHog/posthog-ruby/issues. | |||
There was a problem hiding this comment.
lets remove all the code snippets and event/prop names from this file to https://posthog.com/docs/mcp-analytics so its a single source of truth
There was a problem hiding this comment.
While this is experimental I thought it made sense to keep this here as source of truth. Eventually once the MCP Analytics starts owning this then they can probably own moving this. Are you ok with that?
|
@PostHog/mcp-analytics is there any demand for this (i've not seen any request for it)? its one more package to maintain so take that into consideration |
marandaneto
left a comment
marandaneto
left a comment
There was a problem hiding this comment.
Advisory review of the complete PR patch at b72f5e0. These seven findings were reproduced with focused regression tests in disposable worktrees. The existing MCP and core-client suite passed (250 examples). Validation used Ruby 4.0.6, MCP 1.5.0, and available dependency versions rather than the complete locked bundle; temporary controls were not retained as production fixes.
| return if data.nil? | ||
|
|
||
| Instrumentation.capture_event(data, { | ||
| 'session_id' => data.session_id, |
There was a problem hiding this comment.
blocking: Scope custom events to the active request. capture reads server-global data.session_id, but the active request's session is prepared only after its tool returns. After Alice and then Bob initialize, a custom event emitted inside Alice's next tool call is attributed to Bob—even without concurrency. Retain request-local session and identity instead of using the last completed request's state. Reproduction: reproduced — the focused review_instrumentation_regression_spec.rb RSpec test received custom-event distinct ID bob while the corresponding automatic tool event correctly used alice.
There was a problem hiding this comment.
Fixed by e8f93cb.
Confirmed, and it did not need concurrency to trigger: dispatch_tool_call only resolved the session in prepare_request after the handler returned (the conversation anchor is only known then), so during Alice's tool body data.session_id still pointed at whoever finished last. Two layers in the fix:
- Prime before the tool body.
Instrumentation#prime_sessionresolves the transport/token session ontodata.session_idbeforeyield. Emits nothing; the post-callprepare_requeststill owns identify, lazy initialize and the conversation anchor. - Request-local read. The primed session is pinned to the in-flight
RequestScope(the fiber-storage scope the transport already opens per HTTP request), andAnalytics#capturereads it from there first, falling back todata.session_idonly when there is no scope (stdio, single client). That covers overlapping requests on a threaded server too, which priming alone does not.
Specs in http_transport_spec.rb cover the sequential case you reproduced and an overlapping one (Alice's tool body lets Bob's whole request finish before it captures).
Heads-up for @posthog/mcp (Node). It has half of this. captureToolCall runs prepareToolCallEvent (which resolves the session and writes data.sessionId) before execute(downstreamRequest), so the sequential Alice/Bob case is attributed correctly there. What it is missing is the second layer: captureCustomEvent in packages/mcp/src/index.ts reads trackingData.sessionId, one slot per server, and tool handlers are async. If Alice's handler awaits anything while Bob's request comes in and re-primes data.sessionId, Alice's analytics.capture(...) after the await is attributed to Bob. There is no AsyncLocalStorage (or equivalent request-local store) in the package today. The equivalent fix is to open an AsyncLocalStorage store per request in the request-handler patch, stash the primed sessionId in it, and have captureCustomEvent read from the store before falling back to trackingData.sessionId. Worth an issue on posthog-js.
| if @options.report_missing && name == missing_name && !real_tool?(name) | ||
| result = Tools.result | ||
| safely { record_missing_capability(name, original_arguments, request) } | ||
| return result |
There was a problem hiding this comment.
blocking: Preserve MCP's lifecycle for virtual tool calls. This early return skips the underlying handler lambda, including its protocol-envelope validation and cancellation cleanup. MCP has already registered the stateful request's cancellation object, so every completed get_more_tools call retains an in-flight entry indefinitely; malformed modern-protocol requests also incorrectly succeed. The virtual response needs to run within the real request lifecycle. Reproduction: reproduced — focused MCP 1.5.0 transport tests retained three cancellation entries after three completed virtual calls and returned HTTP 200 for a malformed virtual call whose ordinary-tool control returned HTTP 400/-32602.
There was a problem hiding this comment.
Fixed by 6ccd422.
instrument now registers get_more_tools on the server with define_tool (skipped when the application already owns the name, which then behaves as an ordinary tool). The gem dispatches it like any other tool, so argument validation, envelope checks and the in-flight cancellation cleanup all run; the instrumentation only swaps the recorded event for $mcp_missing_capability. The manual tools/list append is gone since the gem lists it itself. Specs: stateful HTTP spec asserts no in-flight entries remain after three virtual calls, and a call without context now gets the gem's Missing required arguments error response while still recording the missing capability.
| return sanitized unless sanitized.is_a?(Hash) | ||
|
|
||
| result = sanitized.dup | ||
| if result['content'].is_a?(Array) |
There was a problem hiding this comment.
blocking: Redact binary prompt and resource response shapes too. Only top-level tool-result content arrays reach sanitize_content_block. Automatically captured prompt results instead use messages[].content, and resource reads use contents[].blob; these receive only generic string sanitization, so binary payloads below the 10KB heuristic threshold reach $mcp_response unchanged despite the documented binary-content redaction. Reproduction: reproduced — the focused review_payloads_spec.rb RSpec test found the original blob c2Vuc2l0aXZl in both $mcp_prompt_get and $mcp_resource_read payloads returned by the actual sink.
| end | ||
|
|
||
| def decode_percent(value) | ||
| URI.decode_www_form_component(value) |
There was a problem hiding this comment.
blocking: Preserve literal plus signs when decoding base64 data URLs. URI.decode_www_form_component converts literal + into spaces, but + is a valid standard-base64 character in a data URL. Valid binary URLs then fail detection and can be transmitted unchanged. Use percent-decoding that preserves literal plus signs. Reproduction: reproduced — the focused review_payloads_spec.rb RSpec test passed data:application/octet-stream;base64, followed by '++//' * 3000 through the sink and received the complete URL in $mcp_parameters instead of the binary placeholder.
There was a problem hiding this comment.
Fixed by 6ca3657.
decode_percent uses URI.decode_uri_component (percent-only) with URI::DEFAULT_PARSER.unescape as the fallback on Ruby 3.0/3.1. Spec covers your '++//' * 3000 payload and a percent-encoded plus.
| ].freeze | ||
|
|
||
| NORMALIZED_FIELDS = %w[parameters response identify_actor_data error].freeze | ||
|
|
There was a problem hiding this comment.
blocking: Include custom properties in bounded normalization. User-controlled properties from custom events, event_properties, and capture_tool_call are excluded from normalization and progressive depth reduction. Large numeric arrays cannot be reduced by largest-string trimming, so oversized events survive this pipeline and are then discarded by the core batch despite the message-size guarantee. Reproduction: reproduced — the focused review_payloads_spec.rb RSpec test captured properties: { rows: (1..10_000).to_a }, produced a 49,288-byte message, and observed PostHog::MessageBatch retain zero messages instead of one.
There was a problem hiding this comment.
Fixed by bbcc662.
properties is now part of NORMALIZED_FIELDS, so custom event properties, event_properties and capture_tool_call properties get breadth/depth normalization and take part in progressive reduction. Spec reproduces the 10k-row array and asserts the event lands under MAX_EVENT_BYTES.
| # Deep-copies a value with string keys so the pipeline can rely on one shape. | ||
| def stringify_keys(value) | ||
| case value | ||
| when Hash then value.to_h { |k, v| [k.to_s, stringify_keys(v)] } |
There was a problem hiding this comment.
blocking: Detect cycles before recursively copying telemetry. Ruby custom properties can contain a self-reference, but this traversal runs before the cycle-aware normalizer and recurses indefinitely. The resulting SystemStackError is not caught by the sink's StandardError rescue, so analytics crashes through into host code. This concerns Ruby custom properties, not oversized or cyclic JSON request bodies. Reproduction: reproduced — the focused review_payloads_spec.rb RSpec test supplied properties['self'] = properties and observed SystemStackError escaping the public capture_tool_call method before core-client dispatch.
There was a problem hiding this comment.
Fixed by 3615dea.
stringify_keys tracks visited containers by identity and emits the same [Circular ~] marker the normalizer uses (shared, non-cyclic references are still copied). Client#capture_tool_call with a self-referential properties hash no longer raises; covered in client_spec and sanitization_spec.
| raw_context = args.is_a?(Hash) ? (args[:context] || args['context']) : nil | ||
| intent = raw_context.is_a?(String) && !raw_context.strip.empty? ? raw_context.strip : nil | ||
| PreparedToolCall.new( | ||
| args: strip_context(args), |
There was a problem hiding this comment.
blocking: Preserve tool-owned context arguments in custom dispatchers. prepare_tool_list leaves a tool's existing context schema intact, but prepare_tool_call unconditionally removes that argument. A tool requiring its own context consequently receives incomplete arguments. Track which arguments the integration actually injected and strip only those. Reproduction: reproduced — the focused review_custom_dispatch_spec.rb RSpec test prepared a tool with its own required context field and expected { context: 'application data' } after call preparation but received {}; an ownership-aware temporary control passed.
There was a problem hiding this comment.
Fixed by 5f135d3.
prepare_tool_call takes an optional input_schema: (the tool's raw inputSchema, the same Hash handed to prepare_tool_list). When the schema declares context, the argument is left in args; otherwise it is the one we injected and it is stripped as before. Intent extraction is unchanged either way. Custom dispatchers are stateless across pods, so asking for the schema at call time is the only way to decide ownership reliably; remembering what prepare_tool_list injected would only be right on the pod that served the listing. Without the keyword the behaviour is unchanged. README updated, spec matches your reproduction.
|
left a few comments but @PostHog/mcp-analytics should know more about the specifics |
`Analytics#capture` read the server-wide `data.session_id`, which the tool call only settled *after* its handler returned. A custom event emitted from inside a tool body therefore carried whichever session the previous request left behind, and with it that caller's identity. Two layers: - prime the session before the tool body runs (session id only; the conversation anchor is still resolved after the call) - pin the primed session to the in-flight `RequestScope`, and have `capture` read it from there first, so overlapping requests on a threaded server cannot cross-attribute either Specs cover both the sequential and the overlapping Alice/Bob case. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… in comments Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
| # tool body, so a custom event captured inside the tool is attributed to this | ||
| # request even while another request on the same server is in flight. | ||
| def current | ||
| FIBER_STORAGE ? Fiber[KEY] : Thread.current[KEY] |
There was a problem hiding this comment.
Low: Child work loses the request scope on Ruby 3.0 and 3.1
Thread.current[] is fiber-local and is not inherited by a tool-created fiber or thread. In that child, Analytics#capture falls back to the shared data.session_id, so an attacker racing another request can cause one user's custom event properties to be recorded under the attacker's session and PostHog identity. Propagate an immutable request context explicitly to child work, or fail closed instead of using the server-wide session when an HTTP capture has no request scope.
@marandaneto I didn't hear any demand from them, but I have my side projects in Rails and I want to instrument them with MCP Analytics, so I built this for myself. I 100% believe we should not merge this if the team is not interested in maintaining it, because I don't plan on maintaining all this myself. I can get the same by doing a point implementation just for my server, too. |
i mean all the work is done so i think we should merge and release it but i'd like to bring this up because i heard the same from the AIO team, they are too thin to bring support for more SDKs (or just maintain it) |
`URI.decode_www_form_component` is form decoding and turns `+` into a space, but `+` is a standard base64 character. Valid binary data URLs therefore failed detection and were captured verbatim. Use percent-only decoding instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Custom event properties, `event_properties` and `capture_tool_call` properties skipped normalization and progressive depth reduction. A large numeric array cannot be shrunk by string trimming, so such events stayed oversized and the core client dropped them at batch time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`stringify_keys` runs before the cycle-aware normalizer and recursed forever on self-referential custom properties. The resulting `SystemStackError` is not a `StandardError`, so it escaped the sink into host code. Track visited containers and emit the same `[Circular ~]` marker the normalizer uses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The virtual tool used to short-circuit the gem's dispatch lambda, which skipped envelope validation and leaked the in-flight cancellation entry the gem registers before handing us the lambda. `instrument` now defines the tool on the server (unless the application already owns the name), the gem dispatches it like any other tool, and the instrumentation only swaps the recorded event for `$mcp_missing_capability`. The manual `tools/list` append goes away since the gem lists the tool itself. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`prepare_tool_list` leaves a `context` field the tool declares itself intact, but `prepare_tool_call` stripped the argument unconditionally, so such a tool received incomplete arguments. `prepare_tool_call` now takes an optional `input_schema:` and strips `context` only when the schema does not declare it. Without the schema the behaviour is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
i'll review this for the new spec soon, but as @marandaneto pointed, we on the MCP Analytics team do not have the capability to support this SDK for now. i think we can merge this though - work is done as Manoel pointed, but maybe on the docs, we say it's not an official solution and we don't provide support for it yet. |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
Replaced this long report with nine concise inline comments and a short summary.
| tool = tools.is_a?(Hash) ? tools[name] : nil | ||
| schema = tool.respond_to?(:input_schema) ? tool.input_schema&.to_h : nil | ||
| owned = [] | ||
| owned << 'context' if @options.context_enabled? && !SchemaMutation.declares_param?(schema, 'context') |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P1] Preserve composed/reference schemas
Before tools/list, a tool whose allOf schema requires application-owned context loses that argument and fails validation; the identical call succeeds after listing. Use one conservative injection/ownership guard, including $ref: injecting required context beside a reference to a closed object also makes discovery unsatisfiable. The standalone prepare_tool_call has the same composed-ownership problem.
| return sanitized unless sanitized.is_a?(Hash) | ||
|
|
||
| result = sanitized.dup | ||
| if result['content'].is_a?(Array) |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P1] Redact binary prompt/resource responses
Only top-level content blocks are redacted. Real prompts/get and resources/read calls still capture short binary payloads from messages[].content.data and contents[].blob in $mcp_response. Extend shape-aware redaction to those response forms while preserving client-facing results. This reproduces the existing unresolved finding.
| event['response'] = result | ||
| if tool_result_error?(result) | ||
| event['is_error'] = true | ||
| event['error'] = Exceptions.capture_exception(Sanitization.stringify_keys(result)) |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Extract errors before adding conversation metadata
With enable_conversation_id, identical failures become same failure {"conversation_id":"<fresh UUID>"} in $mcp_error_message, fragmenting error grouping. Extract errors from the original result before appending the conversation handle. Current TS already does this.
| # full {#prepare_request} still runs after the call, because the conversation | ||
| # anchor is only known once the tool has returned. Emits nothing. | ||
| def prime_session | ||
| session_id, = Session.resolve(@data, mcp_session_id(@token), token: @token) |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Resolve identity and echoed conversation before capture
On a first modern request with a valid echoed conversation ID, analytics.capture inside the tool uses an anonymous, temporary session; the automatic tool event uses the identified person and conversation session. Resolve and pin both identity and the already-known conversation anchor before invoking the tool.
| return tool unless tool.is_a?(Hash) | ||
|
|
||
| name = fetch(tool, :name) | ||
| return tool if virtual_tool?(name) |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Complete model capture for missing capabilities
With capture_model: true, get_more_tools neither advertises llm_model nor records $mcp_llm_model when supplied. Its recorder and the standalone client's preparation/missing-capability helpers need the model support already present in current Python and TS.
| status, headers, body = @app.call(env) | ||
| # Only a successful initialize gets the token, so a client cannot replay a | ||
| # session minted for a handshake the server rejected. | ||
| if token && success?(status) && headers.respond_to?(:key?) && |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Check initialization success beyond HTTP status
An HTTP 200 response containing a JSON-RPC error still gets a new mcp-session-id. The legacy transport contract associates that header with InitializeResult. Require a successful initialization result or an explicit success signal from the dispatcher before attaching the token.
| 'session_id' => session_id, | ||
| 'resource_name' => request_resource_name(request), | ||
| 'event_type' => EventType::IDENTIFY, | ||
| 'parameters' => { 'request' => request, 'extra' => captured_extra(extra) }, |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Remove raw intent from identify parameters
With identify enabled, context: "Look up jane@example.com" is redacted on $mcp_tool_call but survives verbatim in $identify under $mcp_parameters.request.params.arguments.context. Use the captured-parameter builder here too. Python/TS share this privacy gap; parity alone would preserve it.
| instance_method PostHog::MCP::Client#capture_missing_capability(context: ..., parameters: ..., protocol_version: ..., distinct_id: ..., session_id: ..., client_user_agent: ..., vendor_client: ..., set_properties: ..., groups: ..., properties: ..., timestamp: ...) | ||
| instance_method PostHog::MCP::Client#capture_tool_call(tool_name, intent: ..., intent_source: ..., parameters: ..., response: ..., duration_ms: ..., is_error: ..., error: ..., error_type: ..., category: ..., tool_description: ..., protocol_version: ..., distinct_id: ..., session_id: ..., client_user_agent: ..., vendor_client: ..., set_properties: ..., groups: ..., properties: ..., timestamp: ..., llm_model: ..., llm_model_source: ...) | ||
| instance_method PostHog::MCP::Client#capture_tools_list(tool_names: ..., parameters: ..., response: ..., duration_ms: ..., is_error: ..., error: ..., error_type: ..., protocol_version: ..., distinct_id: ..., session_id: ..., client_user_agent: ..., vendor_client: ..., set_properties: ..., groups: ..., properties: ..., timestamp: ...) | ||
| instance_method PostHog::MCP::Client#prepare_tool_call(name, args = ...) |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Refresh the public API snapshot
prepare_tool_call now accepts input_schema:, but this snapshot omits it. bundle exec rake public_api:check fails locally and in CI. Regenerate the snapshot after settling the API changes.
| properties_key = key_for(schema, :properties) | ||
| schema[properties_key] = {} unless schema[properties_key].is_a?(Hash) | ||
| additional_key = key_for(schema, :additionalProperties) | ||
| schema.delete(additional_key) if schema[additional_key] == false |
There was a problem hiding this comment.
Note
🤖 Automated comment by QA Swarm — not written by a human
[P2] Preserve additionalProperties: false
For a strict tool declaring x, {x: "yes", context: "why", typo: 123} passes the advertised schema but dispatch rejects /typo. The injected property is already allowed through properties; keep additionalProperties: false so discovery matches validation. Python preserves it; TS shares this bug.
3 participants
💡 Motivation and Context
PostHog ships MCP analytics for Node (
@posthog/mcp) and Python (posthog.mcp), but not Ruby. This addsPostHog::MCP, an experimental integration that wraps a server built on the official Rubymcpgem so every tool call, handshake, listing, prompt, resource read, and failure lands in PostHog as a$mcp_*event with the same wire contract as the other two SDKs.Setup is one line, and Rails apps that already call
PostHog.initcan omit the client:What's included:
PostHog::MCP.instrumentwith feature parity: injectedcontextintent argument, conversation ids (prompt-back +structuredContentmirror),get_more_tools,llm_modelcapture, statelessMcp-Session-Idtokens, transport identity headers,identify/$identify,before_send,event_properties, sanitization, PII redaction of intent, and 100KB truncation. Ruby also emits the prompt/resource events the other SDKs reserve.PostHog::MCP::Clientfor custom dispatchers (nomcpgem needed) andPostHog::MCP::RackMiddlewarefor custom Rack stacks._lib/_lib_versionoverride inClient#capture. MCP events report$lib: posthog-ruby-mcp; unlike Node/Python this does not relabel the host client, so a Rails app's other events keepposthog-rails.mcpadded to the Gemfile test group only (peer dependency, gemspec unchanged); public API snapshot now tracksPostHog::MCP.Design notes for reviewers:
around_requesthook lacks the request, session, and params, so the integration prependsMCP::Server#handle_requestand wraps the dispatch lambda it returns. That lambda'sparamsis the same Hash the tool receives, which is what lets us strip injected arguments before thetool.call(**args)splat (an unknown keyword would otherwise become an opaque-32603). NoThread.current; the only ambient state isFiber[]storage for the Streamable HTTP transport → server hop, because the gem re-parses the body in between.require 'posthog/mcp'and oninstrument(stderr outside Rails, never stdout, since stdio servers own it). README, YARD, and the changeset say so too.💚 How did you test it?
bundle exec rspec: 812 examples, 0 failures (115 new examples underspec/posthog/mcp, including end-to-end specs against a realMCP::ServeroverServer#handleand overStreamableHTTPTransport#callin stateless and stateful modes, a concurrency spec, and the frozen cross-SDK vectors for FNV-1a session ids and theMcp-Session-Idtoken copied from the Python/JS suites).bundle exec rubocop: clean.bundle exec rake public_api:check: passes.examples/mcp_server.rbdriven over stdio with realinitialize/tools/list/tools/callJSON-RPC lines; captured events dumped to stderr.📝 Checklist
If releasing new changes
pnpm changesetto generate a changeset file (.changeset/quiet-ruby-mcp-analytics.md, minor)🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Written with Claude Code (Claude Fable 5.1) from a plan the DRI reviewed and approved. The agent explored the JS and Python MCP packages and the Ruby
mcpgem source, and the DRI made the calls on scope (full parity), wiring (explicitinstrumentonly, no Railtie auto-wiring), and$libattribution (per-event override rather than relabeling the client). The DRI also asked for the experimental marking and for avoidingThread.current; the hook design was reworked from anaround_request+ thread-local draft to the singlehandle_requestprepend described above. Cross-SDK divergences were resolved deliberately: Python's empty-tools/listerror text and code-point FNV iteration, JS's[Array]truncation marker and requested-version era check for token minting.🤖 Generated with Claude Code