Skip to content

Scope app-data/clientId unique indexes by thingtime; 400 instead of 500 on null search conditions - #90

Merged
lopugit merged 3 commits into
developfrom
claude/fix-appdata-index-search-null-guard
Aug 17, 2026
Merged

Scope app-data/clientId unique indexes by thingtime; 400 instead of 500 on null search conditions#90
lopugit merged 3 commits into
developfrom
claude/fix-appdata-index-search-null-guard

Conversation

@lopugit

@lopugit lopugit commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

This is the single combined PR for the two confirmed PR #69 review findings. It supersedes #83 and #86.

  • Scope the app crystal.clientId unique index to thingtime: 'app'.
  • Scope the (ownerId, crystal.appId, crystal.key) unique index to thingtime: 'app-data'.
  • Migrate both deployed legacy index names through createIndexReplacing.
  • Retry legacy-index drops with bounded backoff when a concurrent index build temporarily rejects the drop.
  • Return a typed 400 for non-object search-condition entries instead of throwing a 500 for conditions: [null].
  • Keep the three API regressions from the two focused PRs:
    • things-search-null-condition
    • things-data-app-shaped-create
    • things-data-app-shaped-duplicate

Why

Free-form data Things may legitimately carry clientId, appId, and key inside crystal. Exists-only partial filters made those ordinary data Things participate in unrelated app-platform uniqueness constraints. That could make two ordinary data Things collide, or let one data Thing wedge an app-data write into repeated duplicate-key failures.

The search parser separately dereferenced null.field, turning malformed client input into an unhandled 500.

The new partial filters use the same thingtime discriminators already stamped and queried by the app and app-data write paths, so genuine app/app-data uniqueness remains enforced while unrelated data Things no longer collide.

Consolidation

Verification

  • Targeted ESLint for the three touched Remix source/test files: pass.
  • Mongo collection unit suite: 16/16 pass.
  • Full Remix unit suite: pass.
  • Live headless API suite for auth + things: 51/51 pass, including all three restored regression cases.
  • Typecheck ratchet remains non-blocking and reports only unrelated existing diagnostics; no touched file appears in the new-error list.
  • Graphify query resolves the new retry helper, search guard, and API-test nodes from the refreshed graph.

🤖 Consolidated with Codex

@vercel

vercel Bot commented Jul 19, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
thingtime Ready Ready Preview Aug 17, 2026 9:39am
thingtime (develop) Ready Ready Preview Aug 17, 2026 9:39am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Conflict Resolver: this PR conflicts with main in real source files (beyond graphify-out/), and no AI credential is configured in this repo, so automatic resolution was skipped.

To enable AI conflict resolution, add the ANTHROPIC_API_KEY or CLAUDE_CODE_OAUTH_TOKEN Actions secret (easiest: run /install-github-app from the claude CLI). Until then, please resolve the conflicts manually.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Merging main into this branch changes .github/workflows/, which the default token cannot push. Merge the base branch manually (git merge main), or add a CONFLICT_RESOLVER_PAT secret (repo + workflow scope) so this workflow can handle it. Workflow run

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Merging main into this branch changes .github/workflows/, which the default token cannot push. Merge the base branch manually (git merge main), or add a CONFLICT_RESOLVER_PAT secret (repo + workflow scope) so this workflow can handle it. Workflow run

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Merging main into this branch changes .github/workflows/, which the default token cannot push. Merge the base branch manually (git merge main), or add a CONFLICT_RESOLVER_PAT secret (repo + workflow scope) so this workflow can handle it. Workflow run

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Could not auto-resolve conflicts with main — manual resolution needed. See the workflow run.

Conflicted files (as recorded by the merge step):

  • remix/app/api/utils/mongodb/collections.ts

@github-actions

Copy link
Copy Markdown
Contributor

🤝 Merged main into claude/fix-appdata-index-search-null-guard — conflicts auto-resolved by the resolve-pr-conflicts workflow.

Conflicted files:

  • remix/app/api/utils/mongodb/collections.ts

Please review the merge commit before relying on it.

@lopugit
lopugit changed the base branch from main to develop August 6, 2026 06:52
@github-actions github-actions Bot added the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 8, 2026
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

🧬 Rebased claude/fix-appdata-index-search-null-guard onto develop with the AI PR/stack rebase workflow.

  • Previous head: 7fddf51cf5184892aac83e1ee6bcbfff7f2f620d
  • Rewritten head: e019a185f9133e97741607402fc2dd74c23b294a
  • graphify: refreshed and committed (semantic mode: none)
  • Direct stacked children dispatched: 0

The branch was published once with an exact force-with-lease. Please review the rewritten commits before relying on them.

@github-actions github-actions Bot removed the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 8, 2026
@lopugit
lopugit force-pushed the claude/fix-appdata-index-search-null-guard branch from 7fddf51 to e019a18 Compare August 8, 2026 01:43
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🤖 Auto-resolve running — the conflict resolver started working on this PR at 14:27 UTC.

Expected to finish around 14:35 UTC (resolutions typically land in 3-8 minutes; the job times out at 30). On success a merge commit resolving the conflicts is pushed to this branch and a result comment follows — no manual action is needed meanwhile.

Live status

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🤝 Merged develop into claude/fix-appdata-index-search-null-guard — conflicts auto-resolved by the resolve-pr-conflicts workflow.

Conflicted files:

  • graphify-out/GRAPH_REPORT.md
  • graphify-out/graph.json
  • graphify-out/manifest.json
  • remix/app/api/utils/mongodb/collections.ts

graphify-out/ was reset wholesale to the develop side (repo rule: one side, never mixed — the graph merge driver is unavailable in CI).
Then re-ran graphify on the merged code WITH LLM semantic extraction (graphify extract, claude-cli backend) and committed the result — content new to this merge is semantically indexed; unchanged content came from the tracked cache.

Please review the merge commit before relying on it.

@github-actions github-actions Bot added the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 9, 2026
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🤖 Auto-rebase running — the stack rebase started working on this PR at 09:27 UTC.

Expected to finish around 09:42 UTC (most rebases still land in 5-15 minutes; the hosted-runner ceiling is 360). On success this branch is force-pushed onto its new base and a result comment follows — no manual action is needed meanwhile.

Live status

@lopugit
lopugit force-pushed the claude/fix-appdata-index-search-null-guard branch from 2e52d5a to 221402a Compare August 9, 2026 08:46
@github-actions github-actions Bot removed the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 9, 2026
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🧬 Rebased claude/fix-appdata-index-search-null-guard onto develop with the AI PR/stack rebase workflow.

  • Previous head: 2e52d5a5a4cb9b29a99c287d9e412dfe0e62f1d5
  • Rewritten head: 221402adfa02dcc3c1999026d73eb20eacb4aafc
  • graphify: refreshed and committed (semantic mode: none)
  • Direct stacked children dispatched: 0

The branch was published once with an exact force-with-lease. Please review the rewritten commits before relying on them.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🤝 Merged develop into claude/fix-appdata-index-search-null-guard — conflicts auto-resolved by the resolve-pr-conflicts workflow.

No AI resolution was needed by merge time; the branch was updated with a plain merge commit.

graphify-out/ was reset wholesale to the develop side (repo rule: one side, never mixed — the graph merge driver is unavailable in CI).
Then re-ran graphify on the merged code WITH LLM semantic extraction (graphify extract, claude-cli backend) and committed the result — content new to this merge is semantically indexed; unchanged content came from the tracked cache.

Please review the merge commit before relying on it.

@github-actions github-actions Bot added the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 9, 2026
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🧬 Rebased claude/fix-appdata-index-search-null-guard onto develop with the AI PR/stack rebase workflow.

  • Previous head: d4b47d2f02737a8b71917bdd10ca39dd4918edff
  • Rewritten head: 60ca2a808797d107359e0a4822536f63a3c38b76
  • graphify: refreshed and committed (semantic mode: none)
  • Direct stacked children dispatched: 0

The branch was published once with an exact force-with-lease. Please review the rewritten commits before relying on them.

@github-actions github-actions Bot removed the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 9, 2026
@lopugit
lopugit force-pushed the claude/fix-appdata-index-search-null-guard branch from d4b47d2 to 60ca2a8 Compare August 9, 2026 11:48
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

⚠️ The resolver stopped in a later safety, verification, or publication step. No residual conflicted paths were confirmed; see the workflow run.

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

🧹 Develop S3 preview removed

The PR-specific alias and every workflow-created develop deployment were removed when this PR closed.

The ordinary generated Vercel Preview remains available on the shared development runtime.

@github-actions
github-actions Bot temporarily deployed to develop-pr-90 August 17, 2026 08:52 Destroyed
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

⚠️ The resolver stopped in a later safety, verification, or publication step. No residual conflicted paths were confirmed; see the workflow run.

@github-actions github-actions Bot added the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 17, 2026
lopugit and others added 3 commits August 17, 2026 09:31
PR #69 merged before these landed; both are fixed forward against main.

1. app-data / clientId unique indexes matched free-form data things
   (collections.ts). The (ownerId, crystal.appId, crystal.key) index filtered on
   crystal.appId+crystal.key existing but NOT on thingtime, while setAppData
   filters thingtime:'app-data'. So a plain thingtime:['data'] thing using
   'appId'/'key' as ordinary data keys (sanitizeDataCrystal allows both) squats
   the unique slot: app writes miss it, insertOne 11000s all 3 attempts, and the
   caller gets a permanent 503. The likelier hit is user-facing — two ordinary
   data things by one owner sharing those values 409 each other via the
   duplicate-key mapping in things.ts. The crystal.clientId index had the same
   exists-only shape.

   Both are now scoped by thingtime, the same fix already applied to the
   reaction index (targetId: {$type:'string'}) for this exact collision class.
   Narrowing clientId keeps its structural-uniqueness property intact because
   every lookup in apps/apps.ts already filters thingtime:'app' — a data thing
   could never answer one.

   Deployed DBs already carry these indexes under auto-generated names, so both
   move to createIndexReplacing with explicit names + the legacy names, reusing
   the existing 85/86 conflict path.

   Verified against a real mongod: reproduced the old 11000 collisions, ran the
   in-place upgrade, confirmed the legacy names are gone, that app-data and data
   things now coexist, that genuine duplicates (same user/app/key, and duplicate
   clientId) are still blocked, that a different owner is still allowed, and
   that a re-run is a no-op with no drop. ensureIndexes() then migrated the
   local dev DB in place from the old indexes.

2. conditions:[null] returned 500 instead of 400 (search.ts). isGroup(null) is
   false, so buildCondition(null) dereferenced null.field; the API catch-all
   rethrows non-Response errors. Every other malformed shape already 400s, so
   this was a single missed guard. Rate limiting runs first, so it was never a
   DoS lever — a wrong status code and error-log noise.

   Verified by driving the real searchThings(): null / undefined / nested null
   in a group all threw TypeError before and all return 400 now; string, number,
   array and {} were and remain 400; a valid condition still queries normally.

tsc --noEmit unchanged vs main (136 -> 136; zero new type errors).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lopugit
lopugit force-pushed the claude/fix-appdata-index-search-null-guard branch from 7095d8f to 7ced028 Compare August 17, 2026 09:34
@github-actions github-actions Bot removed the ai-rebase-in-progress AI stack rebase currently owns this PR branch label Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🧬 Rebased claude/fix-appdata-index-search-null-guard onto develop with the AI PR/stack rebase workflow.

  • Previous head: 7095d8f870f45ac9f1931d67053b44498bca3a4a
  • Rewritten head: 7ced028ff2b32d88b2d485eca012bdee24e3e0f8
  • graphify: refreshed and committed (semantic mode: none)
  • Direct stacked children dispatched: 0

The branch was published once with an exact force-with-lease. Please review the rewritten commits before relying on them.

@github-actions
github-actions Bot temporarily deployed to develop-pr-90 August 17, 2026 09:35 Destroyed
@lopugit
lopugit merged commit 6a026ec into develop Aug 17, 2026
24 checks passed
@lopugit

lopugit commented Aug 17, 2026

Copy link
Copy Markdown
Owner Author

🤖 Promotion conflict resolution was queued automatically for promote/pr-90-fix-appdata-index-search-null-guard--to-main at exact base main (fd970e9ab4bb729e2117f92e9afc8e9782154a61).

The trusted worker will reconstruct, verify, publish, and attest the review branch; no manual branch update is needed.

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

✅ The trusted promotion worker opened #296 for promote/pr-90-fix-appdata-index-search-null-guard--to-main.

Review the exact resolved paths and immutable snapshot in the promotion PR comment. Workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 Verified automatic promotion resolution for promote/pr-90-fix-appdata-index-search-null-guard--to-main at 572434d4437b0d0c5fa96a0f34fc400d875b7314.

This marker is inert until the live remote branch equals the attested head. Workflow run.

github-actions Bot added a commit that referenced this pull request Aug 17, 2026
Thingtime-Promotion-Source-PR: 90

Thingtime-Promotion-Base-Ref: main

Thingtime-Promotion-Base-SHA: fd970e9

Thingtime-Promotion-Branch: promote/pr-90-fix-appdata-index-search-null-guard--to-main

Thingtime-Promotion-Source-Tip-SHA: 6a026ec

Thingtime-Promotion-Source-Start-SHA: 08699b9

Thingtime-Promotion-Source-End-SHA: 6a026ec

Thingtime-Promotion-Source-Lineage: verified

Thingtime-Promotion-Plan-Hash: 52e75f7eab1671a81c0c8c21e1a6cc0348a9bc60ca7d5a7d29eae8c32b28ee16

Thingtime-Promotion-Patch-ID: 432937c84e53ae84298e7dd9a94a51dbe0cfefa4

Resolved by the promotion worker: https://github.com/lopugit/thingtime/actions/runs/32030650552
lopugit added a commit that referenced this pull request Aug 18, 2026
… by thingtime; 400 instead of 500 on null search conditions (#90)
lopugit added a commit that referenced this pull request Aug 18, 2026
… by thingtime; 400 instead of 500 on null search conditions (#90)
github-actions Bot added a commit that referenced this pull request Aug 18, 2026
… by thingtime; 400 instead of 500 on null search conditions (#90)
github-actions Bot added a commit that referenced this pull request Aug 19, 2026
… by thingtime; 400 instead of 500 on null search conditions (#90)
lopugit added a commit that referenced this pull request Aug 20, 2026
…search-null-guard--to-main

[Promote] Scope app-data/clientId unique indexes by thingtime; 400 instead of 500 on null search conditions (#90)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant