Standardize act, observe, and extract result shapes - #2460
Conversation
|
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
2 issues found and verified against the latest diff
Confidence score: 3/5
- In
packages/server/services/cacheService.ts, the cache result-shape change appears to be a breaking Stagehand REST API contract update without the required integration coverage, which raises real regression risk at the client/server boundary and could break cache consumers at runtime — add an integration test underpackages/server/testthat exercises the new shape end to end. - In
packages/server/services/actService.ts, cache persistence behavior after the shape migration is not covered, so miss/hit flows could silently stop writing or returning cached action data/metadata and degrade correctness or performance — add focused cache-enabled miss and hit tests that assert the expected action payload structure.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/server/services/cacheService.ts">
<violation number="1" location="packages/server/services/cacheService.ts:215">
P2: Custom agent: **Any breaking changes to Stagehand REST API client / server implementation must be covered by an integration test under packages/server/test**
Breaking change to cache result shape without integration test coverage for the new `metadata.cacheStatus` path
The `withCache` utility now writes `cacheStatus` into `result.metadata.cacheStatus` instead of `result.cacheStatus`. This changed code path is exercised by act, observe, and extract, yet none of the server integration tests verify that a cache hit or miss correctly populates `metadata.cacheStatus`. Add coverage in `packages/server/tests/` that exercises `withCache` (or the services that use it) and asserts the `metadata.cacheStatus` value for both HIT and MISS scenarios.</violation>
</file>
<file name="packages/server/services/actService.ts">
<violation number="1" location="packages/server/services/actService.ts:96">
P3: Cache persistence after the result-shape migration lacks coverage, so a future data/metadata regression can silently stop act cache writes or hits. Add focused cache-enabled miss and hit tests asserting the action array is persisted/replayed and `metadata.cacheStatus` is set.
(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED]</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
94ba308 to
06d0e91
Compare
There was a problem hiding this comment.
All reported issues were addressed across 15 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…sult-shapes # Conflicts: # rules/ast-grep/sdk-parity.test.ts
Summary
act(),observe(), andextract()the same{ data, metadata }protocol and SDK result shapeStagehandResultMetadataschema containing optionalcacheStatusandactionIdresultfield with JSON-validateddataand removex-stainless-anymetadata.cacheStatusWhy
The protocol already carried cache status and action ID for act, observe, and extract, but the SDKs returned only each method's inner payload. That made metadata inaccessible and left three inconsistent public contracts.
All three methods now return:
act().dataisActResultData,observe().dataisAction[], andextract().datais refined by the caller's Zod or Pydantic schema.User impact
This is a V4 API-shape change:
actResult.successbecomesactResult.data.successobserveResult[0]becomesobserveResult.data[0]extractResult.dataresult.metadata.cacheStatusin TypeScript andresult.metadata.cache_statusin PythonCache field names, hit/miss values, key calculation, thresholds, reads, and writes are unchanged.
Summary by cubic
Standardized act(), observe(), and extract() to return a consistent { data, metadata } shape across the protocol and all SDKs (TypeScript, Python, Go). Metadata now carries cache status and action ID; docs, wire casing, server caching, examples, and tests are updated.
New Features
ActResult,ObserveResult,ExtractResult).StagehandResultMetadata(actionId?,cacheStatus?) andCacheStatusin protocol/types and all SDKs.extract()returns JSON-validated data; the protocol preservesextract().datakeys on the wire, while casing structuredact/observedata.Migration
act().success→act().data.successobserve()[0]→observe().data[0]extract()result moved under.datawith typeExtractResult<Schema>; read cache via.metadata.cacheStatusact()now returnsActResult;result.success→result.data.successobserve()returnsObserveResult;actions[0]→actions.data[0]extract()returnsExtractResult[Model]; read model via.dataand cache via.metadata.cache_statusAct()now returnsActResult;result.Success→result.Data.SuccessObserve()returnsObserveResult;actions[0]→result.Data[0]Extract()returnsExtractResult; decode fromresult.Dataand read cache viaresult.Metadata.CacheStatusWritten for commit b96d99b. Summary will update on new commits.