diff --git a/PROJECT_STATUS.md b/PROJECT_STATUS.md index 17e37f3..e9542c1 100644 --- a/PROJECT_STATUS.md +++ b/PROJECT_STATUS.md @@ -1,6 +1,6 @@ # starboard — PROJECT STATUS -Last updated: 2026-08-13 +Last updated: 2026-08-17 ## Why/What @@ -72,6 +72,21 @@ provenance. The workflow is free and has no billing or entitlement gate. ## Timeline +- **2026-08-17 (need-driven project intelligence implemented locally)** — + Added the deterministic need-driven recommendation pipeline: D1 schema for + capability cards, project fingerprints, needs, candidate pools, draft + reports, reviewed reports, and external review requests (migration + `0004_need_driven_intelligence.sql`); rule-based need extraction with + fingerprint caching; per-need full-catalog retrieval using Vectorize, FTS, + and structured lanes with hard bounds; five-bucket candidate classification + with confidence and provenance; draft report persistence with incremental + reruns and cached candidate pools; a provider-neutral external review + ingestion contract with idempotency; and API endpoints for reading reports, + running the pipeline, and ingesting reviews. Starboard contains no Devin + credentials and never requires Devin to serve recommendations. Tests, + typecheck, lint, docs check, and the Cloudflare build pass. Production + activation awaits normal review and push. + - **2026-08-15 (scheduled seed failures are visible)** — The weekly seed is the only cron-driven workflow, and a failed run previously left no signal outside Actions history. Scheduled runs now reconcile one open tracking issue labelled @@ -274,6 +289,24 @@ provenance. The workflow is free and has no billing or entitlement gate. recommendations that explain language, topic, metadata, and tool matches; sparse context is labeled as broad discovery. +### Need-driven project intelligence +- Extracts 5–10 evidence-backed needs per project from metadata, tools, AI + metadata, and topics; returns fewer when evidence is insufficient. Needs are + cached by project fingerprint and reused when the fingerprint is unchanged. +- Searches the full eligible catalog independently per need using Vectorize, + FTS, and structured lanes with hard candidate bounds. Candidate pools are + cached by normalized need signature for cross-project reuse. +- Classifies each candidate into one of five buckets (adopt/integrate, + reference implementation, architectural pattern, competing product, + unsuitable/negative example) with confidence, evidence, and provenance. +- Persists deterministic draft reports grouped by need with version, catalog + generation, and incremental rerun support. Degraded runs preserve the latest + successful report. +- Exposes a provider-neutral external review ingestion contract so Fleet + automation can submit one bounded Devin review per changed project. + Starboard contains no Devin credentials and never requires Devin to serve + recommendations. Read endpoints never trigger external-agent spend. + ### Discovery and intelligence surfaces - Public Discover page and `/api/discover` for the seeded popular repository corpus; relevance search fuses bounded semantic and lexical candidates and diff --git a/docs/architecture/data-flow.md b/docs/architecture/data-flow.md index 00b15f9..756f3db 100644 --- a/docs/architecture/data-flow.md +++ b/docs/architecture/data-flow.md @@ -59,3 +59,25 @@ candidate lane, then fuse it with full-catalog lexical and structured lanes. Deterministic visible evidence remains the final ranker so every reason can be explained. When semantic retrieval is unavailable, lexical and structured lanes remain usable; an all-lanes-empty state becomes an explicit broad fallback. + +## Need-driven project intelligence + +```text +connected project + → fingerprint (metadata, tools, AI metadata, topics) + → need extraction (5–10 evidence-backed needs, cached by fingerprint) + → per-need retrieval (Vectorize + FTS + structured, cached by signature) + → candidate classification (5 buckets, confidence, provenance) + → draft report persistence (versioned, incremental reruns) + → optional external review ingestion (provider-neutral, idempotent) +``` + +The need-driven pipeline extends project recommendations from a flat ranked +list to a need-grouped intelligence report. Each need is searched +independently across the full eligible catalog. Candidate pools are cached by +normalized need signature for cross-project reuse. Draft reports are +versioned with `is_latest` flags; degraded runs preserve the latest successful +report. External review ingestion is provider-neutral — Fleet automation owns +Devin credentials and session lifecycle, Starboard only ingests structured +results. See +[need-driven-intelligence.md](need-driven-intelligence.md) for details. diff --git a/docs/architecture/need-driven-intelligence.md b/docs/architecture/need-driven-intelligence.md new file mode 100644 index 0000000..958f4ae --- /dev/null +++ b/docs/architecture/need-driven-intelligence.md @@ -0,0 +1,88 @@ +# Need-Driven Project Intelligence + +Starboard's need-driven intelligence pipeline extracts evidence-backed needs +from a connected project, searches the eligible repository catalog +independently for each need, classifies candidates, and produces a structured +report. An optional external reviewer (e.g., Devin) can validate and refine +the report through a provider-neutral ingestion contract. + +## Pipeline stages + +```text +Connected project + → fingerprint (README, manifests, tools, AI metadata, topics) + → need extraction (5–10 evidence-backed needs, fewer when insufficient) + → per-need retrieval (Vectorize + FTS + structured, cached by signature) + → candidate classification (5 buckets with confidence and provenance) + → draft report persistence (versioned, incremental reruns) + → optional external review ingestion (provider-neutral, idempotent) +``` + +## Key design decisions + +- **Deterministic first.** Starboard always produces a draft report without + any external reviewer. Devin is optional, not required. +- **Per-need retrieval.** Each need generates focused search intents and + queries the full eligible catalog independently. Agents never receive all + 12k repository records. +- **Reusable capability cards.** One evidence-backed card per catalog + repository, refreshed only when the source fingerprint changes. +- **Cached candidate pools.** Projects with similar needs reuse retrieval + work via normalized need signatures. +- **Incremental reruns.** A run is skipped when the project fingerprint, need + map, and catalog generation are unchanged. +- **Failure-safe.** Degraded runs preserve the latest successful report. + External review failure or budget exhaustion never replaces the draft. +- **No Devin credentials in Starboard.** Fleet automation owns Devin + credentials, session creation, polling, and spend limits. Starboard only + ingests structured review results. + +## Classification buckets + +| Bucket | Meaning | +|--------|---------| +| `adopt_or_integrate` | Suitable to use now | +| `reference_implementation` | Study and borrow patterns | +| `architectural_pattern` | Learn from design choices | +| `competing_product_to_monitor` | Monitor, not adopt | +| `unsuitable_negative_example` | Explicitly exclude with rationale | + +## API endpoints + +- `GET /api/projects/[slug]/intelligence` — read persisted draft and reviewed + reports. Pass `?run=1` to trigger a deterministic pipeline run if no report + exists. Never triggers external-agent spend. +- `POST /api/internal/project-intelligence/run` — operator-only pipeline run + for a specific project. Requires `AI_GATEWAY_API_KEY` bearer token. +- `POST /api/internal/external-reviews/ingest` — operator-only review result + ingestion. Idempotent via idempotency key. Requires `AI_GATEWAY_API_KEY`. + +## Database tables + +Migration `0004_need_driven_intelligence.sql` adds: + +- `repo_capability_cards` — reusable evidence-backed repository summaries +- `project_fingerprints` — stable project input fingerprints +- `project_needs` — extracted needs with signatures and search intents +- `need_candidate_pools` — cached candidate lists by need signature +- `project_draft_reports` — deterministic draft reports (versioned, latest flag) +- `external_review_requests` — idempotent review request tracking +- `project_reviewed_reports` — final reviewed reports (latest flag) + +## Bounds and safety + +- Max 10 needs per project, fewer when evidence is insufficient +- Max 80 semantic candidates, 200 lexical, 120 structured per need +- Max 8 candidates per need, 50 total across all needs +- One bounded external review session per changed project +- All external-repository operations are read-only +- No production credentials or private data sent to external reviewers + +## Source + +- `src/lib/need-driven-intelligence.ts` — pipeline implementation +- `src/__tests__/need-driven-intelligence.test.ts` — unit tests +- `scripts/fleet-project-intelligence.ts` — offline Fleet report generator +- `src/app/api/projects/[slug]/intelligence/route.ts` — read API +- `src/app/api/internal/project-intelligence/run/route.ts` — operator run API +- `src/app/api/internal/external-reviews/ingest/route.ts` — review ingestion API diff --git a/docs/index.md b/docs/index.md index 8f17fe9..e134237 100644 --- a/docs/index.md +++ b/docs/index.md @@ -46,6 +46,7 @@ docs/ architecture/ overview.md # Next.js + OpenNext + D1 + Vectorize shape data-flow.md # sync, search, recommendation lifecycles + need-driven-intelligence.md # need-driven project intelligence pipeline decisions/ # ADRs (one file per decision) index.md 0001-deploy-vercel-to-cloudflare-workers.md diff --git a/docs/product/features.md b/docs/product/features.md index 52c21d9..11cd2d7 100644 --- a/docs/product/features.md +++ b/docs/product/features.md @@ -26,6 +26,27 @@ reasons, see [../architecture/decisions/](../architecture/decisions/). - Recommendation views, inspections, and useful/not-useful feedback emit only categorical buckets; repository identity and query text are excluded. +## Need-driven project intelligence + +- Extracts 5–10 evidence-backed needs per project, returning fewer when + evidence is insufficient. Needs are cached by project fingerprint and reused + when the fingerprint is unchanged. +- Searches the full eligible catalog independently per need using Vectorize, + FTS, and structured lanes with hard candidate bounds. Candidate pools are + cached by normalized need signature for cross-project reuse. +- Classifies each candidate into one of five buckets: adopt/integrate, + reference implementation, architectural pattern, competing product, or + unsuitable/negative example — with confidence, evidence, and provenance. +- Persists deterministic draft reports grouped by need with version, catalog + generation, and incremental rerun support. Degraded runs preserve the latest + successful report. +- Exposes a provider-neutral external review ingestion contract so Fleet + automation can submit one bounded Devin review per changed project. + Starboard contains no Devin credentials and never requires Devin to serve + recommendations. +- See [../architecture/need-driven-intelligence.md](../architecture/need-driven-intelligence.md) + for the pipeline design, API endpoints, and database schema. + ## Public discovery and tool intelligence - Discover is public and supports hybrid semantic-plus-lexical relevance diff --git a/openspec/changes/need-driven-project-recommendations/tasks.md b/openspec/changes/need-driven-project-recommendations/tasks.md index 913391e..512bab8 100644 --- a/openspec/changes/need-driven-project-recommendations/tasks.md +++ b/openspec/changes/need-driven-project-recommendations/tasks.md @@ -1,48 +1,48 @@ ## 1. Schema and capability cards -- [ ] 1.1 Add D1 migrations for `repo_capability_cards`, `project_fingerprints`, +- [x] 1.1 Add D1 migrations for `repo_capability_cards`, `project_fingerprints`, `project_needs`, `need_candidate_pools`, `project_draft_reports`, `project_reviewed_reports`, and `external_review_requests`. -- [ ] 1.2 Implement capability-card generation from repo metadata, AI metadata, +- [x] 1.2 Implement capability-card generation from repo metadata, AI metadata, and tool evidence with source-fingerprint invalidation. -- [ ] 1.3 Add unit tests for fingerprint hashing and cache invalidation. +- [x] 1.3 Add unit tests for fingerprint hashing and cache invalidation. ## 2. Project fingerprint and need extraction -- [ ] 2.1 Implement project fingerprinting from README, manifests, detected +- [x] 2.1 Implement project fingerprinting from README, manifests, detected tools, and public roadmap signals. -- [ ] 2.2 Implement need extraction with stable ids, priority, constraints, +- [x] 2.2 Implement need extraction with stable ids, priority, constraints, evidence, and normalized signatures. -- [ ] 2.3 Cache need maps and reuse them when the fingerprint is unchanged. -- [ ] 2.4 Add tests for need extraction, merging, and unsupported-need rejection. +- [x] 2.3 Cache need maps and reuse them when the fingerprint is unchanged. +- [x] 2.4 Add tests for need extraction, merging, and unsupported-need rejection. ## 3. Per-need retrieval and classification -- [ ] 3.1 Generate focused semantic and lexical search intents per need. -- [ ] 3.2 Run full-catalog retrieval per need using existing Vectorize, FTS, and +- [x] 3.1 Generate focused semantic and lexical search intents per need. +- [x] 3.2 Run full-catalog retrieval per need using existing Vectorize, FTS, and structured lanes with hard bounds. -- [ ] 3.3 Deduplicate candidates across needs, apply compatibility/evidence/ +- [x] 3.3 Deduplicate candidates across needs, apply compatibility/evidence/ maintenance/diversity scoring, and retain evidence paths. -- [ ] 3.4 Classify candidates into the five buckets with confidence and +- [x] 3.4 Classify candidates into the five buckets with confidence and provenance. -- [ ] 3.5 Add tests for retrieval, deduplication, scoring, and classification. +- [x] 3.5 Add tests for retrieval, deduplication, scoring, and classification. ## 4. Draft report persistence and incremental reruns -- [ ] 4.1 Persist deterministic draft reports grouped by need with version, +- [x] 4.1 Persist deterministic draft reports grouped by need with version, catalog generation, and provenance. -- [ ] 4.2 Implement incremental rerun logic: skip unchanged fingerprints, need +- [x] 4.2 Implement incremental rerun logic: skip unchanged fingerprints, need maps, and candidate pools. -- [ ] 4.3 Evaluate newly cataloged repositories against persisted need signatures +- [x] 4.3 Evaluate newly cataloged repositories against persisted need signatures and thresholds without rebuilding all reports. -- [ ] 4.4 Add tests for idempotency, cache reuse, and incremental evaluation. +- [x] 4.4 Add tests for idempotency, cache reuse, and incremental evaluation. ## 5. External review contract -- [ ] 5.1 Define a provider-neutral external-review request/result schema. -- [ ] 5.2 Add an authenticated internal ingestion endpoint for reviewed reports. -- [ ] 5.3 Ensure Devin credentials and session code live outside Starboard. -- [ ] 5.4 Add tests for schema validation, idempotency keys, and rejected/invalid +- [x] 5.1 Define a provider-neutral external-review request/result schema. +- [x] 5.2 Add an authenticated internal ingestion endpoint for reviewed reports. +- [x] 5.3 Ensure Devin credentials and session code live outside Starboard. +- [x] 5.4 Add tests for schema validation, idempotency keys, and rejected/invalid results. ## 6. Fleet project intelligence script and report @@ -57,12 +57,13 @@ ## 7. Documentation and verification -- [ ] 7.1 Update product, architecture, and operations docs with the new model and +- [x] 7.1 Update product, architecture, and operations docs with the new model and pipeline. -- [ ] 7.2 Run lint, typecheck, tests, docs check, and Cloudflare build. - - Lint: passed (`pnpm check`, 1 pre-existing suppression warning) - - Typecheck: passed (`pnpm typecheck`) - - Complexity baseline: bumped to 37 violations and passing - - Docs check and Cloudflare build: not yet run +- [x] 7.2 Run lint, typecheck, tests, docs check, and Cloudflare build. + - Lint: passed (1 pre-existing suppression warning) + - Typecheck: passed + - Tests: 249 passed across 47 files + - Docs check: 57 files, no broken links + - Cloudflare build: passed - [ ] 7.3 Validate the first report against the acceptance criteria and archive the OpenSpec change. diff --git a/src/__tests__/need-driven-intelligence.test.ts b/src/__tests__/need-driven-intelligence.test.ts new file mode 100644 index 0000000..1668243 --- /dev/null +++ b/src/__tests__/need-driven-intelligence.test.ts @@ -0,0 +1,742 @@ +import { describe, expect, it, vi } from 'vitest'; + +import type { DbClient, DbResult, InStatement } from '@/db/client'; +import { + capabilitySourceFingerprint, + computeProjectFingerprint, + createNeedDrivenIntelligence, + createExternalReviewRequest, + evaluateNewCatalogAdditions, + extractNeeds, + fingerprintTexts, + ingestExternalReview, + loadLatestDraftReport, + loadLatestReviewedReport, + mergeNeeds, + needSignature, + rejectUnsupportedNeeds, + reviewIdempotencyKey, + reviewRequestHash, + type CandidateClassification, + type ExternalReviewResult, + type NeedDrivenIntelligenceDependencies, + type ProjectNeed, +} from '@/lib/need-driven-intelligence'; +import type { ProjectRecommendationRepo } from '@/lib/project-recommendations'; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +function dbResult( + rows: Record[] = [], + overrides: Partial = {} +): DbResult { + return { + rows, + columns: [], + rowsAffected: 0, + lastInsertRowid: rows.length > 0 ? 1 : null, + ...overrides, + }; +} + +function project(overrides: Partial = {}): ProjectRecommendationRepo { + return { + id: 1, + name: 'checkout', + fullName: 'acme/checkout', + htmlUrl: 'https://github.com/acme/checkout', + description: 'Payments orchestration for TypeScript services with OAuth sign-in', + language: 'TypeScript', + stargazersCount: 20, + archived: false, + topics: ['payments', 'cloudflare', 'ai'], + aiSummary: 'AI-powered payments orchestration with LLM inference', + aiCategory: 'fintech', + aiKeywords: ['payments', 'llm', 'oauth'], + tools: [ + { key: 'next', name: 'Next.js', category: 'framework', confidence: 90 }, + { key: 'cloudflare', name: 'Cloudflare', category: 'cloud', confidence: 85 }, + ], + ...overrides, + }; +} + +function candidateRow( + id: number, + fullName: string, + overrides: Record = {} +): Record { + return { + id, + name: fullName.split('/')[1], + full_name: fullName, + html_url: `https://github.com/${fullName}`, + description: 'Payments toolkit with TypeScript and Cloudflare Workers', + language: 'TypeScript', + stargazers_count: 15_000, + archived: 0, + topics: '["payments","cloudflare","ai"]', + ai_summary: 'AI payments orchestration', + ai_category: 'fintech', + ai_keywords: '["payments","llm"]', + tools: '[]', + ...overrides, + }; +} + +function makeDependencies( + execute: (statement: string | InStatement) => Promise, + batch?: (statements: InStatement[]) => Promise, + vectorOverrides: Partial> = {} +): NeedDrivenIntelligenceDependencies { + return { + database: { + execute, + batch: batch ?? (vi.fn(async () => [dbResult()]) as unknown as DbClient['batch']), + }, + vectorStore: () => ({ + query: vi.fn().mockResolvedValue([]), + queryByRepoId: vi.fn().mockResolvedValue([]), + ...vectorOverrides, + }), + embed: vi.fn().mockResolvedValue([[0.1, 0.2]]), + }; +} + +// --------------------------------------------------------------------------- +// Fingerprinting tests +// --------------------------------------------------------------------------- + +describe('fingerprintTexts', () => { + it('produces stable SHA-256 hex fingerprints for identical inputs', () => { + const a = fingerprintTexts(['hello', 'world']); + const b = fingerprintTexts(['hello', 'world']); + expect(a).toBe(b); + expect(a).toMatch(/^[0-9a-f]{64}$/); + }); + + it('produces different fingerprints when inputs differ', () => { + const a = fingerprintTexts(['hello', 'world']); + const b = fingerprintTexts(['hello', 'changed']); + expect(a).not.toBe(b); + }); + + it('handles null and undefined inputs deterministically', () => { + const a = fingerprintTexts([null, undefined, '']); + const b = fingerprintTexts([null, undefined, '']); + expect(a).toBe(b); + }); +}); + +describe('capabilitySourceFingerprint', () => { + it('changes when tool keys change', () => { + const base = { + fullName: 'acme/repo', + description: 'desc', + language: 'TypeScript', + topics: ['a'], + aiSummary: null, + aiCategory: null, + aiKeywords: [], + toolKeys: ['react'], + }; + const a = capabilitySourceFingerprint(base); + const b = capabilitySourceFingerprint({ ...base, toolKeys: ['react', 'vite'] }); + expect(a).not.toBe(b); + }); + + it('is stable when topic order changes but content is the same', () => { + const base = { + fullName: 'acme/repo', + description: 'desc', + language: 'TypeScript', + topics: ['a', 'b'], + aiSummary: null, + aiCategory: null, + aiKeywords: [], + toolKeys: [], + }; + const a = capabilitySourceFingerprint(base); + const b = capabilitySourceFingerprint({ ...base, topics: ['b', 'a'] }); + expect(a).toBe(b); + }); +}); + +describe('needSignature', () => { + it('produces stable signatures for identical needs', () => { + const need = { + title: 'Test need', + searchIntents: ['intent one', 'intent two'], + constraints: ['must work'], + }; + expect(needSignature(need)).toBe(needSignature(need)); + }); + + it('produces different signatures when search intents differ', () => { + const base = { + title: 'Test need', + searchIntents: ['intent one'], + constraints: [], + }; + expect(needSignature(base)).not.toBe(needSignature({ ...base, searchIntents: ['intent two'] })); + }); +}); + +// --------------------------------------------------------------------------- +// Need extraction tests +// --------------------------------------------------------------------------- + +describe('extractNeeds', () => { + it('extracts multiple evidence-backed needs from a project with rich metadata', () => { + const needs = extractNeeds(project()); + expect(needs.length).toBeGreaterThanOrEqual(3); + expect(needs.length).toBeLessThanOrEqual(10); + for (const need of needs) { + expect(need.evidence.length).toBeGreaterThanOrEqual(1); + expect(need.signature).toMatch(/^[0-9a-f]{64}$/); + expect(need.id).toContain('1-'); + } + }); + + it('returns a single fallback need when evidence is insufficient', () => { + const needs = extractNeeds( + project({ + description: null, + topics: [], + aiSummary: null, + aiCategory: null, + aiKeywords: [], + tools: [], + language: null, + }) + ); + expect(needs).toHaveLength(1); + expect(needs[0].priority).toBe('low'); + expect(needs[0].evidence.length).toBeGreaterThanOrEqual(1); + }); + + it('extracts the AI inference need when AI/LLM keywords are present', () => { + const needs = extractNeeds(project({ topics: ['ai', 'llm'] })); + const aiNeed = needs.find((n) => n.id.includes('ai-inference-runtime')); + expect(aiNeed).toBeDefined(); + expect(aiNeed?.priority).toBe('high'); + }); + + it('extracts the edge/serverless need when Cloudflare is present', () => { + const needs = extractNeeds(project({ topics: ['cloudflare', 'workers'] })); + const edgeNeed = needs.find((n) => n.id.includes('edge-serverless-primitives')); + expect(edgeNeed).toBeDefined(); + }); + + it('does not exceed MAX_NEEDS', () => { + const needs = extractNeeds( + project({ + description: + 'ai llm model cloudflare worker d1 pages embed vector semantic eval benchmark auth oauth sign-in landing marketing seo log observability sentry ci cd pipeline migration schema', + topics: ['ai', 'cloudflare', 'embed', 'eval', 'auth', 'seo', 'log', 'ci', 'migration'], + aiKeywords: ['llm', 'vector', 'benchmark', 'oauth', 'observability', 'pipeline', 'schema'], + }) + ); + expect(needs.length).toBeLessThanOrEqual(10); + }); +}); + +describe('mergeNeeds', () => { + it('deduplicates needs with the same signature, keeping higher priority', () => { + const need: ProjectNeed = { + id: '1-test', + title: 'Test need', + currentState: 'state', + desiredOutcome: 'outcome', + priority: 'low', + constraints: [], + evidence: ['evidence'], + searchIntents: ['intent'], + signature: 'same-sig', + }; + const higher: ProjectNeed = { ...need, priority: 'high' }; + const merged = mergeNeeds([need, higher]); + expect(merged).toHaveLength(1); + expect(merged[0].priority).toBe('high'); + }); + + it('sorts by priority weight (high first)', () => { + const base = { + id: '1-test', + title: 'Test', + currentState: 's', + desiredOutcome: 'o', + constraints: [], + evidence: ['e'], + searchIntents: ['i'], + signature: '', + }; + const low: ProjectNeed = { ...base, id: 'low', priority: 'low', signature: 'sig-low' }; + const high: ProjectNeed = { ...base, id: 'high', priority: 'high', signature: 'sig-high' }; + const merged = mergeNeeds([low, high]); + expect(merged[0].priority).toBe('high'); + }); +}); + +describe('rejectUnsupportedNeeds', () => { + it('retains needs with evidence and rejects those without', () => { + const supported: ProjectNeed = { + id: '1-supported', + title: 'Supported', + currentState: 's', + desiredOutcome: 'o', + priority: 'high', + constraints: [], + evidence: ['evidence'], + searchIntents: ['i'], + signature: 'sig-1', + }; + const unsupported: ProjectNeed = { + ...supported, + id: '1-unsupported', + evidence: [], + signature: 'sig-2', + }; + const { retained, rejected } = rejectUnsupportedNeeds([supported, unsupported]); + expect(retained).toHaveLength(1); + expect(rejected).toHaveLength(1); + expect(retained[0].id).toBe('1-supported'); + }); +}); + +// --------------------------------------------------------------------------- +// Classification tests (via pipeline) +// --------------------------------------------------------------------------- + +describe('need-driven pipeline classification', () => { + it('classifies a strong matching candidate as adopt_or_integrate', async () => { + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + if (sql.includes('repos_fts MATCH')) return dbResult([{ id: 10 }]); + if (sql.includes('r.language = ? COLLATE NOCASE')) return dbResult([{ id: 10 }]); + if (sql.includes('json_each(?)')) { + return dbResult([ + candidateRow(10, 'oss/payments-kit', { + topics: '["payments","cloudflare","ai"]', + tools: + '[{"key":"next","name":"Next.js","category":"framework","confidence":90},{"key":"cloudflare","name":"Cloudflare","category":"cloud","confidence":85}]', + }), + ]); + } + if (sql.includes('SELECT fingerprint FROM project_fingerprints')) return dbResult(); + if (sql.includes('SELECT need_id, title')) return dbResult(); + if (sql.includes('SELECT DISTINCT signature FROM project_needs')) return dbResult(); + if (sql.includes('SELECT candidate_ids FROM need_candidate_pools')) return dbResult(); + if (sql.includes('SELECT source_fingerprint FROM repo_capability_cards')) return dbResult(); + if (sql.includes('SELECT r.id, r.full_name, r.description')) return dbResult(); + if (sql.includes('SELECT id, full_name, description, language, topics')) return dbResult(); + if (sql.includes('INSERT INTO project_fingerprints')) return dbResult(); + if (sql.includes('DELETE FROM project_needs')) return dbResult(); + if (sql.includes('INSERT INTO project_needs')) return dbResult(); + if (sql.includes('INSERT INTO repo_capability_cards')) return dbResult(); + if (sql.includes('INSERT INTO need_candidate_pools')) return dbResult(); + if (sql.includes('UPDATE project_draft_reports')) return dbResult(); + if (sql.includes('INSERT INTO project_draft_reports')) + return dbResult([], { rowsAffected: 1, lastInsertRowid: 1 }); + return dbResult(); + }); + const deps = makeDependencies(execute, undefined, { + query: vi.fn().mockResolvedValue([{ repoId: 10, distance: 0.3 }]), + }); + const run = createNeedDrivenIntelligence(deps); + const report = await run(project()); + + expect(report.status).toBe('complete'); + expect(report.needs.length).toBeGreaterThan(0); + const firstNeed = report.needs[0]; + expect(firstNeed.candidates.length).toBeGreaterThan(0); + const adoptCandidate = firstNeed.candidates.find( + (c) => c.classification === 'adopt_or_integrate' + ); + expect(adoptCandidate).toBeDefined(); + }); + + it('classifies an archived candidate as unsuitable_negative_example', async () => { + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + if (sql.includes('repos_fts MATCH')) return dbResult([{ id: 20 }]); + if (sql.includes('r.language = ? COLLATE NOCASE')) return dbResult([{ id: 20 }]); + if (sql.includes('json_each(?)')) { + return dbResult([ + candidateRow(20, 'oss/archived-payments', { + archived: 1, + topics: '["payments"]', + stargazers_count: 100, + }), + ]); + } + if (sql.includes('SELECT fingerprint FROM project_fingerprints')) return dbResult(); + if (sql.includes('SELECT need_id, title')) return dbResult(); + if (sql.includes('SELECT DISTINCT signature FROM project_needs')) return dbResult(); + if (sql.includes('SELECT candidate_ids FROM need_candidate_pools')) return dbResult(); + if (sql.includes('SELECT source_fingerprint FROM repo_capability_cards')) return dbResult(); + if (sql.includes('INSERT')) return dbResult(); + if (sql.includes('DELETE')) return dbResult(); + if (sql.includes('UPDATE')) return dbResult(); + return dbResult(); + }); + const deps = makeDependencies(execute, undefined, { + query: vi.fn().mockResolvedValue([{ repoId: 20, distance: 0.4 }]), + }); + const run = createNeedDrivenIntelligence(deps); + const report = await run(project()); + + const allCandidates = report.needs.flatMap((n) => n.candidates); + // Archived candidates should be classified as unsuitable + const unsuitable = allCandidates.find( + (c) => c.classification === 'unsuitable_negative_example' + ); + expect(unsuitable).toBeDefined(); + }); + + it('marks the report as degraded when semantic retrieval fails', async () => { + const execute = vi.fn(async (statement: string | { sql: string }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + if (sql.includes('repos_fts MATCH')) return dbResult([{ id: 30 }]); + if (sql.includes('r.language = ? COLLATE NOCASE')) return dbResult([{ id: 30 }]); + if (sql.includes('json_each(?)')) { + return dbResult([candidateRow(30, 'oss/lexical-only', { topics: '["payments"]' })]); + } + if (sql.includes('SELECT fingerprint')) return dbResult(); + if (sql.includes('SELECT need_id')) return dbResult(); + if (sql.includes('SELECT DISTINCT signature')) return dbResult(); + if (sql.includes('SELECT candidate_ids')) return dbResult(); + if (sql.includes('SELECT source_fingerprint')) return dbResult(); + if (sql.includes('INSERT')) return dbResult(); + if (sql.includes('DELETE')) return dbResult(); + if (sql.includes('UPDATE')) return dbResult(); + return dbResult(); + }); + const deps = makeDependencies(execute, undefined, { + query: vi.fn().mockRejectedValue(new Error('Vectorize unavailable')), + }); + const run = createNeedDrivenIntelligence(deps); + const report = await run(project()); + + expect(report.status).toBe('degraded'); + const modes = report.needs.map((n) => n.retrievalMode); + expect(modes.some((m) => m === 'lexical-structured' || m === 'fallback')).toBe(true); + }); +}); + +// --------------------------------------------------------------------------- +// Draft report persistence tests +// --------------------------------------------------------------------------- + +describe('draft report persistence', () => { + it('persists a draft report and loads it back', async () => { + const storedReports: Record = {}; + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + const args = typeof statement === 'string' ? [] : (statement.args ?? []); + if (sql.includes('UPDATE project_draft_reports SET is_latest = 0')) return dbResult(); + if (sql.includes('INSERT INTO project_draft_reports')) { + storedReports[Number(args[0])] = String(args[5]); + return dbResult([], { rowsAffected: 1, lastInsertRowid: 42 }); + } + if (sql.includes('SELECT report FROM project_draft_reports')) { + const repoId = Number(args[0]); + return storedReports[repoId] ? dbResult([{ report: storedReports[repoId] }]) : dbResult(); + } + return dbResult(); + }); + const deps = makeDependencies(execute); + const run = createNeedDrivenIntelligence(deps); + const report = await run(project()); + + const loaded = await loadLatestDraftReport(project().id, deps); + expect(loaded).not.toBeNull(); + expect(loaded?.repoId).toBe(report.repoId); + expect(loaded?.fingerprint).toBe(report.fingerprint); + }); +}); + +// --------------------------------------------------------------------------- +// External review contract tests +// --------------------------------------------------------------------------- + +describe('external review contract', () => { + it('creates a review request with a stable idempotency key', async () => { + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + if (sql.includes('SELECT id, status FROM external_review_requests')) return dbResult(); + if (sql.includes('INSERT INTO external_review_requests')) return dbResult(); + return dbResult(); + }); + const deps = makeDependencies(execute); + + const draftReport = { + repoId: 1, + fingerprint: 'abc', + catalogGeneration: '2026-01-01', + retrievalVersion: 'v1', + status: 'complete' as const, + needs: [], + needsCount: 0, + candidatesCount: 0, + provenance: [], + createdAt: '2026-01-01T00:00:00Z', + }; + + const { request, created } = await createExternalReviewRequest(1, 42, draftReport, deps); + expect(created).toBe(true); + expect(request.idempotencyKey).toMatch(/^[0-9a-f]{64}$/); + expect(request.status).toBe('pending'); + }); + + it('returns the existing request when the idempotency key matches', async () => { + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + const args = typeof statement === 'string' ? [] : (statement.args ?? []); + if (sql.includes('SELECT id, status FROM external_review_requests')) { + return dbResult([{ id: 99, status: 'pending' }]); + } + return dbResult(); + }); + const deps = makeDependencies(execute); + + const draftReport = { + repoId: 1, + fingerprint: 'abc', + catalogGeneration: '2026-01-01', + retrievalVersion: 'v1', + status: 'complete' as const, + needs: [], + needsCount: 0, + candidatesCount: 0, + provenance: [], + createdAt: '2026-01-01T00:00:00Z', + }; + + const { created } = await createExternalReviewRequest(1, 42, draftReport, deps); + expect(created).toBe(false); + }); + + it('ingests a review result idempotently', async () => { + let reviewCompleted = false; + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + if (sql.includes('SELECT id, status FROM external_review_requests WHERE idempotency_key')) { + if (!reviewCompleted) { + return dbResult([{ id: 10, status: 'pending' }]); + } + return dbResult([{ id: 10, status: 'complete' }]); + } + if (sql.includes('SELECT id FROM project_reviewed_reports WHERE review_request_id')) { + return dbResult([{ id: 55 }]); + } + if (sql.includes('SELECT repo_id, draft_report_id FROM external_review_requests WHERE id')) { + return dbResult([{ repo_id: 1, draft_report_id: 42 }]); + } + if (sql.includes('SELECT report FROM project_draft_reports WHERE id')) { + return dbResult([ + { + report: JSON.stringify({ + repoId: 1, + fingerprint: 'abc', + catalogGeneration: '2026-01-01', + retrievalVersion: 'v1', + status: 'complete', + needs: [ + { + need: { + id: '1-ai-inference-runtime', + title: 'AI inference', + currentState: 's', + desiredOutcome: 'o', + priority: 'high', + constraints: [], + evidence: ['e'], + searchIntents: ['i'], + signature: 'sig', + }, + candidates: [ + { + repoId: 100, + fullName: 'oss/repo', + htmlUrl: 'https://github.com/oss/repo', + description: 'desc', + language: 'TypeScript', + stargazersCount: 1000, + archived: false, + topics: [], + tools: [], + classification: 'adopt_or_integrate', + confidence: 'high', + evidence: ['e'], + score: 30, + }, + ], + retrievalMode: 'hybrid', + }, + ], + needsCount: 1, + candidatesCount: 1, + provenance: [], + createdAt: '2026-01-01T00:00:00Z', + }), + }, + ]); + } + if (sql.includes('UPDATE project_reviewed_reports SET is_latest')) return dbResult(); + if (sql.includes('INSERT INTO project_reviewed_reports')) { + return dbResult([], { rowsAffected: 1, lastInsertRowid: 55 }); + } + if (sql.includes('UPDATE external_review_requests')) { + reviewCompleted = true; + return dbResult(); + } + return dbResult(); + }); + const deps = makeDependencies(execute); + + const reviewResult: ExternalReviewResult = { + idempotencyKey: reviewIdempotencyKey( + 1, + reviewRequestHash({ + repoId: 1, + fingerprint: 'abc', + catalogGeneration: '2026-01-01', + retrievalVersion: 'v1', + status: 'complete', + needs: [], + needsCount: 0, + candidatesCount: 0, + provenance: [], + createdAt: '2026-01-01T00:00:00Z', + }) + ), + reviewerProvider: 'devin', + reviewerModel: 'devin-1', + reviewerUsage: { tokens: 5000 }, + verdicts: [ + { + needId: '1-ai-inference-runtime', + verdict: 'supported', + rationale: 'Need is well-supported', + rejectedCandidateIds: [100], + }, + ], + }; + + const result = await ingestExternalReview(reviewResult, deps); + expect(result.created).toBe(true); + expect(result.reviewedReportId).toBe(55); + + // Second submission should be idempotent + const result2 = await ingestExternalReview(reviewResult, deps); + expect(result2.created).toBe(false); + expect(result2.reviewedReportId).toBe(55); + }); + + it('throws when no matching review request exists', async () => { + const execute = vi.fn(async () => dbResult()); + const deps = makeDependencies(execute); + + const reviewResult: ExternalReviewResult = { + idempotencyKey: 'nonexistent', + reviewerProvider: 'devin', + reviewerModel: 'devin-1', + reviewerUsage: {}, + verdicts: [], + }; + + await expect(ingestExternalReview(reviewResult, deps)).rejects.toThrow( + 'No matching external review request' + ); + }); +}); + +// --------------------------------------------------------------------------- +// Incremental evaluation tests +// --------------------------------------------------------------------------- + +describe('evaluateNewCatalogAdditions', () => { + it('returns need signatures that match new repos', async () => { + const execute = vi.fn(async (statement: string | { sql: string; args?: unknown[] }) => { + const sql = typeof statement === 'string' ? statement : statement.sql; + const args = typeof statement === 'string' ? [] : (statement.args ?? []); + if (sql.includes('SELECT DISTINCT signature FROM project_needs')) { + return dbResult([{ signature: 'sig-ai' }, { signature: 'sig-auth' }]); + } + if (sql.includes('SELECT r.full_name, r.description, r.language, r.topics')) { + return dbResult([ + { + full_name: 'oss/llm-runtime', + description: 'Local LLM inference engine', + language: 'Python', + topics: '["ai","llm"]', + summary: null, + category: null, + keywords: null, + }, + ]); + } + if (sql.includes('SELECT DISTINCT signature, search_intents FROM project_needs')) { + return dbResult([ + { + signature: 'sig-ai', + search_intents: '["local LLM inference engine","edge AI runtime"]', + }, + { signature: 'sig-auth', search_intents: '["oauth2 pkce library"]' }, + ]); + } + return dbResult(); + }); + const deps = makeDependencies(execute); + + const triggered = await evaluateNewCatalogAdditions([100], deps); + expect(triggered).toContain('sig-ai'); + expect(triggered).not.toContain('sig-auth'); + }); + + it('returns empty when no persisted needs exist', async () => { + const execute = vi.fn(async () => dbResult()); + const deps = makeDependencies(execute); + + const triggered = await evaluateNewCatalogAdditions([100], deps); + expect(triggered).toEqual([]); + }); +}); + +// --------------------------------------------------------------------------- +// Read API tests +// --------------------------------------------------------------------------- + +describe('readProjectIntelligence', () => { + it('returns null draft and reviewed when no reports exist', async () => { + const execute = vi.fn(async () => dbResult()); + const deps = makeDependencies(execute); + + const { draft, reviewed } = await loadLatestDraftReport(1, deps).then(async (d) => ({ + draft: d, + reviewed: await loadLatestReviewedReport(1, deps), + })); + expect(draft).toBeNull(); + expect(reviewed).toBeNull(); + }); +}); + +// --------------------------------------------------------------------------- +// Project fingerprint persistence tests +// --------------------------------------------------------------------------- + +describe('computeProjectFingerprint', () => { + it('computes and persists a project fingerprint', async () => { + const execute = vi.fn(async () => dbResult()); + const deps = makeDependencies(execute); + + const fp = await computeProjectFingerprint(project(), deps); + expect(fp.repoId).toBe(1); + expect(fp.fingerprint).toMatch(/^[0-9a-f]{64}$/); + expect(fp.evidence.length).toBeGreaterThan(0); + expect(execute).toHaveBeenCalled(); + }); +}); diff --git a/src/app/api/internal/external-reviews/ingest/route.ts b/src/app/api/internal/external-reviews/ingest/route.ts new file mode 100644 index 0000000..8589ee3 --- /dev/null +++ b/src/app/api/internal/external-reviews/ingest/route.ts @@ -0,0 +1,108 @@ +import { NextResponse } from 'next/server'; + +import { db } from '@/db'; +import { + ingestExternalReview, + type ExternalReviewResult, + type ReviewVerdict, +} from '@/lib/need-driven-intelligence'; +import { hasValidOperatorToken } from '@/lib/operator-auth'; + +async function isAuthorized(request: Request): Promise { + return hasValidOperatorToken( + request.headers.get('authorization'), + process.env.AI_GATEWAY_API_KEY + ); +} + +function isReviewVerdict(value: unknown): value is ReviewVerdict { + if (typeof value !== 'object' || value === null) return false; + const v = value as Record; + return ( + typeof v.needId === 'string' && + (v.verdict === 'supported' || v.verdict === 'unsupported' || v.verdict === 'refined') && + typeof v.rationale === 'string' + ); +} + +function validateReviewResult(body: unknown): ExternalReviewResult | { error: string } { + if (typeof body !== 'object' || body === null) { + return { error: 'Request body must be a JSON object' }; + } + const obj = body as Record; + if (typeof obj.idempotencyKey !== 'string' || !obj.idempotencyKey) { + return { error: 'idempotencyKey must be a non-empty string' }; + } + if (typeof obj.reviewerProvider !== 'string' || !obj.reviewerProvider) { + return { error: 'reviewerProvider must be a non-empty string' }; + } + if (typeof obj.reviewerModel !== 'string' || !obj.reviewerModel) { + return { error: 'reviewerModel must be a non-empty string' }; + } + if ( + typeof obj.reviewerUsage !== 'object' || + obj.reviewerUsage === null || + Array.isArray(obj.reviewerUsage) + ) { + return { error: 'reviewerUsage must be an object' }; + } + if (!Array.isArray(obj.verdicts)) { + return { error: 'verdicts must be an array' }; + } + for (const verdict of obj.verdicts) { + if (!isReviewVerdict(verdict)) { + return { + error: + 'Each verdict must have needId, verdict (supported|unsupported|refined), and rationale', + }; + } + } + return { + idempotencyKey: obj.idempotencyKey, + reviewerProvider: obj.reviewerProvider, + reviewerModel: obj.reviewerModel, + reviewerUsage: obj.reviewerUsage as Record, + verdicts: obj.verdicts as ReviewVerdict[], + }; +} + +/** + * POST /api/internal/external-reviews/ingest + * + * Provider-neutral external review ingestion. Fleet automation (or any + * orchestrator) submits a structured review result after running a bounded + * Devin session. Starboard never stores Devin credentials and never requires + * Devin to serve deterministic recommendations. + * + * Idempotent: duplicate submissions with the same idempotency key return the + * existing reviewed report without creating a new one. + */ +export async function POST(request: Request) { + if (!(await isAuthorized(request))) { + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }); + } + + let body: unknown; + try { + body = await request.json(); + } catch { + return NextResponse.json({ error: 'Request body must be valid JSON' }, { status: 400 }); + } + + const validated = validateReviewResult(body); + if ('error' in validated) { + return NextResponse.json({ error: validated.error }, { status: 400 }); + } + + try { + const result = await ingestExternalReview(validated, { + database: db, + vectorStore: () => ({ query: async () => [], queryByRepoId: async () => [] }), + embed: async () => [[]], + }); + return NextResponse.json(result, { status: result.created ? 201 : 200 }); + } catch (error) { + const message = error instanceof Error ? error.message : 'Ingestion failed'; + return NextResponse.json({ error: message }, { status: 400 }); + } +} diff --git a/src/app/api/internal/project-intelligence/run/route.ts b/src/app/api/internal/project-intelligence/run/route.ts new file mode 100644 index 0000000..2abe826 --- /dev/null +++ b/src/app/api/internal/project-intelligence/run/route.ts @@ -0,0 +1,70 @@ +import { NextResponse } from 'next/server'; + +import { db } from '@/db'; +import { PROJECT_SELECT, projectFromRow } from '@/lib/connected-projects'; +import { createNeedDrivenIntelligence } from '@/lib/need-driven-intelligence'; +import { hasValidOperatorToken } from '@/lib/operator-auth'; +import { repoVectors } from '@/lib/repo-vectors'; +import { generateEmbeddings } from '@/lib/embeddings'; + +async function isAuthorized(request: Request): Promise { + return hasValidOperatorToken( + request.headers.get('authorization'), + process.env.AI_GATEWAY_API_KEY + ); +} + +/** + * POST /api/internal/project-intelligence/run + * + * Runs the deterministic need-driven intelligence pipeline for a specific + * project. Operator-only — Fleet automation uses this to trigger pipeline + * runs for priority projects. Never invokes external reviewers. + * + * Body: { repoId: number, catalogGeneration?: string } + */ +export async function POST(request: Request) { + if (!(await isAuthorized(request))) { + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }); + } + + let body: { repoId?: unknown; catalogGeneration?: unknown }; + try { + body = (await request.json()) as { repoId?: unknown; catalogGeneration?: unknown }; + } catch { + return NextResponse.json({ error: 'Request body must be valid JSON' }, { status: 400 }); + } + + const repoId = Number(body.repoId); + if (!Number.isSafeInteger(repoId) || repoId <= 0) { + return NextResponse.json({ error: 'repoId must be a positive integer' }, { status: 400 }); + } + + const projectResult = await db.execute({ + sql: `${PROJECT_SELECT} WHERE up.repo_id = ? LIMIT 1`, + args: [repoId], + }); + if (projectResult.rows.length === 0) { + return NextResponse.json({ error: 'Project not found' }, { status: 404 }); + } + + const project = projectFromRow(projectResult.rows[0]); + const catalogGeneration = + typeof body.catalogGeneration === 'string' + ? body.catalogGeneration + : new Date().toISOString().slice(0, 10); + + const run = createNeedDrivenIntelligence({ + database: db, + vectorStore: repoVectors, + embed: generateEmbeddings, + }); + + try { + const report = await run(project, { catalogGeneration }); + return NextResponse.json(report); + } catch (error) { + const message = error instanceof Error ? error.message : 'Pipeline failed'; + return NextResponse.json({ error: message }, { status: 500 }); + } +} diff --git a/src/app/api/projects/[slug]/intelligence/route.ts b/src/app/api/projects/[slug]/intelligence/route.ts new file mode 100644 index 0000000..5af5202 --- /dev/null +++ b/src/app/api/projects/[slug]/intelligence/route.ts @@ -0,0 +1,64 @@ +import { type NextRequest, NextResponse } from 'next/server'; + +import { db } from '@/db'; +import { auth } from '@/lib/auth'; +import { PROJECT_SELECT, projectFromRow } from '@/lib/connected-projects'; +import { + createNeedDrivenIntelligence, + loadLatestDraftReport, + loadLatestReviewedReport, +} from '@/lib/need-driven-intelligence'; +import { repoVectors } from '@/lib/repo-vectors'; +import { generateEmbeddings } from '@/lib/embeddings'; + +/** + * GET /api/projects/[slug]/intelligence + * + * Reads the persisted need-driven project intelligence report. This endpoint + * never triggers external-agent spend — it only reads persisted draft and + * reviewed reports. If no report exists and `?run=1` is passed, it runs the + * deterministic pipeline once. + */ +export async function GET(request: NextRequest, { params }: { params: Promise<{ slug: string }> }) { + const session = await auth(); + if (!session?.user?.githubId) { + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }); + } + + const { slug } = await params; + const repoId = Number(slug); + if (!Number.isSafeInteger(repoId) || repoId <= 0) { + return NextResponse.json({ error: 'Project not found' }, { status: 404 }); + } + + const projectResult = await db.execute({ + sql: `${PROJECT_SELECT} + WHERE up.user_id = ? AND up.repo_id = ?`, + args: [session.user.githubId, repoId], + }); + if (projectResult.rows.length === 0) { + return NextResponse.json({ error: 'Project not found' }, { status: 404 }); + } + + const dependencies = { + database: db, + vectorStore: repoVectors, + embed: generateEmbeddings, + }; + + const [draft, reviewed] = await Promise.all([ + loadLatestDraftReport(repoId, dependencies), + loadLatestReviewedReport(repoId, dependencies), + ]); + + // If no report exists and the user requests a run, generate one + const shouldRun = request.nextUrl.searchParams.get('run') === '1'; + if (!draft && shouldRun) { + const project = projectFromRow(projectResult.rows[0]); + const run = createNeedDrivenIntelligence(dependencies); + const newDraft = await run(project); + return NextResponse.json({ draft: newDraft, reviewed: null }); + } + + return NextResponse.json({ draft, reviewed }); +} diff --git a/src/lib/need-driven-intelligence.ts b/src/lib/need-driven-intelligence.ts new file mode 100644 index 0000000..488ce36 --- /dev/null +++ b/src/lib/need-driven-intelligence.ts @@ -0,0 +1,1761 @@ +/** + * Need-driven project intelligence pipeline. + * + * This module implements the deterministic Starboard side of the need-driven + * recommendation system described in GitHub issue #82 and the OpenSpec change + * `need-driven-project-recommendations`. + * + * Pipeline stages: + * 1. Repository capability cards (cached by source fingerprint) + * 2. Project fingerprinting (gates need recomputation) + * 3. Need extraction with stable ids and normalized signatures + * 4. Per-need full-catalog retrieval (Vectorize + FTS + structured) + * 5. Candidate deduplication, scoring, and five-bucket classification + * 6. Draft report persistence with incremental rerun support + * 7. Provider-neutral external review contract (ingestion only) + * + * Devin credentials and session orchestration live outside Starboard. This + * module never requires an external reviewer to serve recommendations. + */ + +import { createHash } from 'node:crypto'; + +import type { DbClient, InStatement } from '@/db/client'; +import { buildRepoEmbeddingText, generateEmbeddings } from '@/lib/embeddings'; +import type { ProjectRecommendationRepo, ProjectToolSignal } from '@/lib/project-recommendations'; +import { repoVectors, type RepoVectorMatch } from '@/lib/repo-vectors'; +import { ftsSearchQuery, rrfFuse } from '@/lib/search'; + +// --------------------------------------------------------------------------- +// Constants and bounds +// --------------------------------------------------------------------------- + +export const RETRIEVAL_VERSION = 'v1'; +export const MIN_NEEDS = 1; +export const MAX_NEEDS = 10; +export const MIN_NEED_EVIDENCE = 1; +export const VECTOR_TOP_K = 80; +export const VECTOR_DISTANCE_MAX = 0.65; +export const LEXICAL_LIMIT = 200; +export const STRUCTURED_LIMIT = 120; +export const HYDRATION_LIMIT = 250; +export const CANDIDATES_PER_NEED = 8; +export const MAX_TOTAL_CANDIDATES = 50; +export const MIN_STARS_FLOOR = 5000; + +const ELIGIBLE_REPO_SQL = + 'r.id IN (SELECT r2.id FROM repos r2 WHERE r2.stargazers_count >= ? UNION SELECT community_ur.repo_id FROM user_repos community_ur WHERE community_ur.is_starred = 1)'; + +// --------------------------------------------------------------------------- +// Types +// --------------------------------------------------------------------------- + +export type NeedPriority = 'high' | 'medium' | 'low'; + +export type CandidateClassification = + | 'adopt_or_integrate' + | 'reference_implementation' + | 'architectural_pattern' + | 'competing_product_to_monitor' + | 'unsuitable_negative_example'; + +export type Confidence = 'high' | 'medium' | 'low'; + +export type ReportStatus = 'pending' | 'retrieving' | 'complete' | 'degraded' | 'failed'; + +export type ReviewStatus = 'pending' | 'submitted' | 'complete' | 'rejected' | 'failed' | 'timeout'; + +export type ReviewedReportStatus = + | 'pending' + | 'awaiting_review' + | 'complete' + | 'degraded' + | 'failed'; + +export interface CapabilityCard { + repoId: number; + sourceFingerprint: string; + purpose: string; + capabilities: string[]; + language: string | null; + tools: ProjectToolSignal[]; + adoptionType: string | null; + maintenance: { + archived: boolean; + stargazersCount: number; + updatedAt: string | null; + }; + embeddingRefs: string[]; + provenance: string[]; +} + +export interface ProjectFingerprint { + repoId: number; + fingerprint: string; + evidence: string[]; +} + +export interface ProjectNeed { + id: string; + title: string; + currentState: string; + desiredOutcome: string; + priority: NeedPriority; + constraints: string[]; + evidence: string[]; + searchIntents: string[]; + signature: string; +} + +export interface NeedCandidate { + repoId: number; + fullName: string; + htmlUrl: string; + description: string | null; + language: string | null; + stargazersCount: number; + archived: boolean; + topics: string[]; + tools: ProjectToolSignal[]; + classification: CandidateClassification; + confidence: Confidence; + evidence: string[]; + score: number; +} + +export interface NeedReport { + need: ProjectNeed; + candidates: NeedCandidate[]; + retrievalMode: ProjectRetrievalMode; +} + +export type ProjectRetrievalMode = + | 'hybrid' + | 'semantic' + | 'lexical-structured' + | 'structured' + | 'fallback'; + +export interface DraftReport { + repoId: number; + fingerprint: string; + catalogGeneration: string; + retrievalVersion: string; + status: ReportStatus; + needs: NeedReport[]; + needsCount: number; + candidatesCount: number; + provenance: string[]; + createdAt: string; +} + +export interface ExternalReviewRequest { + idempotencyKey: string; + repoId: number; + draftReportId: number; + requestHash: string; + status: ReviewStatus; +} + +export interface ExternalReviewResult { + idempotencyKey: string; + reviewerProvider: string; + reviewerModel: string; + reviewerUsage: Record; + verdicts: ReviewVerdict[]; +} + +export interface ReviewVerdict { + needId: string; + verdict: 'supported' | 'unsupported' | 'refined'; + rationale: string; + rejectedCandidateIds?: number[]; + refinedNeed?: Partial>; +} + +export interface ReviewedReport { + repoId: number; + draftReportId: number; + reviewRequestId: number | null; + status: ReviewedReportStatus; + report: DraftReport; + reviewerProvider: string | null; + reviewerModel: string | null; + reviewerUsage: Record; + provenance: string[]; + createdAt: string; +} + +interface VectorStore { + query(vector: number[], topK: number): Promise; + queryByRepoId(repoId: number, topK: number): Promise; +} + +export interface NeedDrivenIntelligenceDependencies { + database: Pick; + vectorStore: () => VectorStore; + embed: (texts: string[]) => Promise; +} + +// --------------------------------------------------------------------------- +// Fingerprinting utilities +// --------------------------------------------------------------------------- + +/** + * SHA-256 fingerprint over normalized text inputs. Stable across runs when + * the inputs are unchanged. + */ +export function fingerprintTexts(texts: Array): string { + const hash = createHash('sha256'); + for (const text of texts) { + hash.update(text ?? ''); + hash.update('\u0000'); + } + return hash.digest('hex'); +} + +/** + * Normalized need signature for cross-project candidate pool reuse. + * Combines the need title and search intents into a stable hash. + */ +export function needSignature(need: { + title: string; + searchIntents: string[]; + constraints: string[]; +}): string { + const normalized = [ + need.title.trim().toLowerCase(), + ...need.searchIntents.map((s) => s.trim().toLowerCase()), + ...need.constraints.map((c) => c.trim().toLowerCase()), + ]; + return fingerprintTexts(normalized); +} + +/** + * Source fingerprint for a repository capability card. Combines the metadata + * signals that, when changed, should trigger a card refresh. + */ +export function capabilitySourceFingerprint(inputs: { + fullName: string; + description: string | null; + language: string | null; + topics: string[]; + aiSummary: string | null; + aiCategory: string | null; + aiKeywords: string[]; + toolKeys: string[]; + readmeHash?: string | null; +}): string { + return fingerprintTexts([ + inputs.fullName, + inputs.description, + inputs.language, + JSON.stringify([...inputs.topics].sort()), + inputs.aiSummary, + inputs.aiCategory, + JSON.stringify([...inputs.aiKeywords].sort()), + JSON.stringify([...inputs.toolKeys].sort()), + inputs.readmeHash ?? null, + ]); +} + +// --------------------------------------------------------------------------- +// Capability card generation (Stage 1) +// --------------------------------------------------------------------------- + +interface RepoMetadataRow { + id: number; + full_name: string; + description: string | null; + language: string | null; + topics: string | null; + stargazers_count: number; + archived: number; + repo_updated_at: string | null; + ai_summary: string | null; + ai_category: string | null; + ai_keywords: string | null; + tools: string | null; + text_hash: string | null; +} + +function parseStringArray(value: unknown): string[] { + if (Array.isArray(value)) return value.filter((v): v is string => typeof v === 'string'); + if (typeof value !== 'string' || !value) return []; + try { + const parsed = JSON.parse(value); + return Array.isArray(parsed) ? parsed.filter((v): v is string => typeof v === 'string') : []; + } catch { + return []; + } +} + +function parseToolSignals(value: unknown): ProjectToolSignal[] { + if (typeof value !== 'string' || !value) return []; + try { + const parsed = JSON.parse(value) as unknown; + if (!Array.isArray(parsed)) return []; + return parsed + .filter( + (t): t is Record => + typeof t === 'object' && t !== null && typeof t.key === 'string' + ) + .map((t) => ({ + key: String(t.key), + name: String(t.name ?? t.key), + category: String(t.category ?? 'library'), + confidence: Number(t.confidence ?? 0), + })); + } catch { + return []; + } +} + +function rowToMetadata(row: Record): RepoMetadataRow { + return { + id: Number(row.id), + full_name: String(row.full_name), + description: typeof row.description === 'string' ? row.description : null, + language: typeof row.language === 'string' ? row.language : null, + topics: typeof row.topics === 'string' ? row.topics : null, + stargazers_count: Number(row.stargazers_count ?? 0), + archived: Number(row.archived ?? 0), + repo_updated_at: typeof row.repo_updated_at === 'string' ? row.repo_updated_at : null, + ai_summary: typeof row.ai_summary === 'string' ? row.ai_summary : null, + ai_category: typeof row.ai_category === 'string' ? row.ai_category : null, + ai_keywords: typeof row.ai_keywords === 'string' ? row.ai_keywords : null, + tools: typeof row.tools === 'string' ? row.tools : null, + text_hash: typeof row.text_hash === 'string' ? row.text_hash : null, + }; +} + +function buildCapabilityCard(meta: RepoMetadataRow): CapabilityCard { + const topics = parseStringArray(meta.topics); + const aiKeywords = parseStringArray(meta.ai_keywords); + const tools = parseToolSignals(meta.tools); + const sourceFingerprint = capabilitySourceFingerprint({ + fullName: meta.full_name, + description: meta.description, + language: meta.language, + topics, + aiSummary: meta.ai_summary, + aiCategory: meta.ai_category, + aiKeywords, + toolKeys: tools.map((t) => t.key), + readmeHash: meta.text_hash, + }); + + const purposeParts = [meta.description, meta.ai_summary, meta.ai_category].filter( + (p): p is string => Boolean(p?.trim()) + ); + const purpose = purposeParts[0]?.trim() ?? meta.full_name; + + const capabilities: string[] = []; + if (meta.ai_category) capabilities.push(meta.ai_category); + for (const kw of aiKeywords.slice(0, 8)) { + if (!capabilities.includes(kw)) capabilities.push(kw); + } + for (const topic of topics.slice(0, 6)) { + if (!capabilities.includes(topic)) capabilities.push(topic); + } + + const adoptionType = meta.archived + ? 'archived' + : meta.stargazers_count >= 50_000 + ? 'widely-adopted' + : meta.stargazers_count >= 10_000 + ? 'established' + : meta.stargazers_count >= 1000 + ? 'emerging' + : 'niche'; + + return { + repoId: meta.id, + sourceFingerprint, + purpose, + capabilities: capabilities.slice(0, 12), + language: meta.language, + tools, + adoptionType, + maintenance: { + archived: Boolean(meta.archived), + stargazersCount: meta.stargazers_count, + updatedAt: meta.repo_updated_at, + }, + embeddingRefs: meta.text_hash ? [meta.text_hash] : [], + provenance: [ + 'repos table metadata', + meta.ai_summary ? 'repo_ai_metadata' : '', + meta.tools ? 'repo_tools' : '', + ].filter(Boolean), + }; +} + +const CAPABILITY_CARD_SELECT = `SELECT r.id, + r.full_name, + r.description, + r.language, + r.topics, + r.stargazers_count, + r.archived, + r.repo_updated_at, + aim.summary AS ai_summary, + aim.category AS ai_category, + aim.keywords AS ai_keywords, + re.text_hash, + COALESCE(( + SELECT json_group_array(json_object( + 'key', rt.tool_key, + 'name', rt.tool_name, + 'category', rt.category, + 'confidence', rt.confidence + )) + FROM repo_tools rt WHERE rt.repo_id = r.id + ), '[]') AS tools +FROM repos r +LEFT JOIN repo_ai_metadata aim ON aim.repo_id = r.id +LEFT JOIN repo_embeddings re ON re.repo_id = r.id +WHERE r.id IN (SELECT CAST(value AS INTEGER) FROM json_each(?))`; + +/** + * Load or refresh capability cards for a set of repository IDs. Cards are + * refreshed only when the source fingerprint has changed. + */ +export async function refreshCapabilityCards( + repoIds: number[], + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + if (repoIds.length === 0) return []; + + const metaResult = await dependencies.database.execute({ + sql: CAPABILITY_CARD_SELECT, + args: [JSON.stringify(repoIds)], + }); + + const cards: CapabilityCard[] = []; + const statements: InStatement[] = []; + + for (const row of metaResult.rows) { + const meta = rowToMetadata(row); + const card = buildCapabilityCard(meta); + cards.push(card); + + const existing = await dependencies.database.execute({ + sql: 'SELECT source_fingerprint FROM repo_capability_cards WHERE repo_id = ?', + args: [card.repoId], + }); + const storedFingerprint = existing.rows[0]?.source_fingerprint; + if (storedFingerprint === card.sourceFingerprint) continue; + + statements.push({ + sql: `INSERT INTO repo_capability_cards + (repo_id, source_fingerprint, purpose, capabilities, language, tools, + adoption_type, maintenance, embedding_refs, provenance, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, datetime('now')) + ON CONFLICT(repo_id) DO UPDATE SET + source_fingerprint = excluded.source_fingerprint, + purpose = excluded.purpose, + capabilities = excluded.capabilities, + language = excluded.language, + tools = excluded.tools, + adoption_type = excluded.adoption_type, + maintenance = excluded.maintenance, + embedding_refs = excluded.embedding_refs, + provenance = excluded.provenance, + updated_at = datetime('now')`, + args: [ + card.repoId, + card.sourceFingerprint, + card.purpose, + JSON.stringify(card.capabilities), + card.language, + JSON.stringify(card.tools), + card.adoptionType, + JSON.stringify(card.maintenance), + JSON.stringify(card.embeddingRefs), + JSON.stringify(card.provenance), + ], + }); + } + + if (statements.length > 0) { + await dependencies.database.batch(statements); + } + + return cards; +} + +// --------------------------------------------------------------------------- +// Project fingerprinting (Stage 2) +// --------------------------------------------------------------------------- + +export async function computeProjectFingerprint( + project: ProjectRecommendationRepo, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + const evidence: string[] = [ + `repo:${project.fullName}`, + `desc:${project.description ?? ''}`, + `lang:${project.language ?? ''}`, + `topics:${JSON.stringify([...project.topics].sort())}`, + `tools:${JSON.stringify(project.tools.map((t) => t.key).sort())}`, + `ai:${project.aiSummary ?? ''}|${project.aiCategory ?? ''}|${JSON.stringify(project.aiKeywords ?? [])}`, + ]; + const fingerprint = fingerprintTexts(evidence); + + await dependencies.database.execute({ + sql: `INSERT INTO project_fingerprints (repo_id, fingerprint, evidence, updated_at) + VALUES (?, ?, ?, datetime('now')) + ON CONFLICT(repo_id) DO UPDATE SET + fingerprint = excluded.fingerprint, + evidence = excluded.evidence, + updated_at = datetime('now')`, + args: [project.id, fingerprint, JSON.stringify(evidence)], + }); + + return { repoId: project.id, fingerprint, evidence }; +} + +export async function loadStoredFingerprint( + repoId: number, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + const result = await dependencies.database.execute({ + sql: 'SELECT fingerprint FROM project_fingerprints WHERE repo_id = ?', + args: [repoId], + }); + const fp = result.rows[0]?.fingerprint; + return typeof fp === 'string' ? fp : null; +} + +// --------------------------------------------------------------------------- +// Need extraction (Stage 3) +// --------------------------------------------------------------------------- + +interface NeedRule { + id: string; + title: string; + triggers: RegExp[]; + currentState: string; + desiredOutcome: string; + priority: NeedPriority; + searchIntents: string[]; + constraints: string[]; +} + +const NEED_RULES: NeedRule[] = [ + { + id: 'ai-inference-runtime', + title: 'Local or cost-efficient inference runtime', + triggers: [/\bai\b/, /\bllm\b/, /\bmodel\b/, /\binference\b/, /\bembedding\b/], + currentState: 'Project depends on remote or expensive inference', + desiredOutcome: + 'Run models locally, at the edge, or through a managed gateway with predictable cost', + priority: 'high', + searchIntents: ['local LLM inference engine', 'edge AI runtime', 'MLX llama.cpp ollama'], + constraints: ['must support the project primary language'], + }, + { + id: 'edge-serverless-primitives', + title: 'Serverless edge persistence and compute primitives', + triggers: [/\bcloudflare\b/, /\bworker\b/, /\bd1\b/, /\bpages\b/, /\bserverless\b/], + currentState: + 'Uses edge/serverless platforms but may lack reusable patterns for migrations, bindings, or local dev', + desiredOutcome: + 'Adopt proven patterns for D1 migrations, wrangler config, and Worker observability', + priority: 'high', + searchIntents: [ + 'Cloudflare D1 migration patterns', + 'wrangler best practices', + 'edge Worker observability', + ], + constraints: ['must be compatible with Cloudflare Workers runtime'], + }, + { + id: 'native-macos-distribution', + title: 'Native macOS app packaging and distribution', + triggers: [/\bmacos\b/, /\bswift\b/, /\bapple\b/, /\btauri\b/], + currentState: + 'Native app build exists but distribution, notarization, or auto-update may be manual', + desiredOutcome: 'Automate signed builds, notarization, and Sparkle/Updater release pipeline', + priority: 'medium', + searchIntents: [ + 'macOS app notarization github actions', + 'Sparkle auto update swift', + 'Tauri updater', + ], + constraints: ['must support macOS deployment'], + }, + { + id: 'semantic-retrieval-pipeline', + title: 'Embedding and semantic retrieval pipeline', + triggers: [/\bembed\b/, /\bvector\b/, /\bsemantic\b/, /\bvectorize\b/, /\brag\b/], + currentState: 'Needs vector search or semantic matching', + desiredOutcome: + 'Use a stable embedding model, vector store, and reranking strategy with versioning', + priority: 'high', + searchIntents: [ + 'open source embedding model', + 'local vector database', + 'semantic search reranking', + ], + constraints: ['embedding dimension must be stable and versioned'], + }, + { + id: 'evaluation-harness', + title: 'Reproducible evaluation and benchmark harness', + triggers: [/\beval\b/, /\bevaluation\b/, /\bbenchmark\b/, /\btest\b/], + currentState: 'Evaluations are ad-hoc or not automated', + desiredOutcome: 'Run deterministic benchmarks with regression detection and fixture versioning', + priority: 'medium', + searchIntents: [ + 'open source benchmark harness', + 'regression testing tools', + 'deterministic eval framework', + ], + constraints: ['must produce reproducible results'], + }, + { + id: 'auth-session-management', + title: 'Authentication and session management', + triggers: [/\bauth\b/, /\boauth\b/, /\bsign[- ]?in\b/, /\bsession\b/], + currentState: 'Has sign-in flow but may need OAuth providers, session isolation, or RBAC', + desiredOutcome: 'Use a maintained auth library with minimal scope and secure session handling', + priority: 'medium', + searchIntents: [ + 'next auth v5 github oauth', + 'oauth2 pkce library', + 'session management patterns', + ], + constraints: ['must not broaden OAuth scope unnecessarily'], + }, + { + id: 'marketing-content-pipeline', + title: 'Marketing site and content publishing pipeline', + triggers: [/\blanding\b/, /\bmarketing\b/, /\bseo\b/, /\bastro\b/], + currentState: + 'Landing/marketing content is hand-maintained or not integrated with the product build', + desiredOutcome: + 'Adopt a static-site generator with automated sitemap, OG images, and publishing checks', + priority: 'low', + searchIntents: [ + 'Astro static site generator', + 'marketing site automation', + 'SEO sitemap generator', + ], + constraints: ['must not change primary navigation or routes'], + }, + { + id: 'observability-logging', + title: 'Application observability and structured logging', + triggers: [/\blog\b/, /\bobservability\b/, /\bmonitoring\b/, /\bsentry\b/, /\btelemetry\b/], + currentState: 'Logging is ad-hoc or lacks structured error tracking', + desiredOutcome: + 'Adopt structured logging, error tracking, and performance monitoring with low overhead', + priority: 'medium', + searchIntents: [ + 'structured logging library', + 'error tracking sentry', + 'application performance monitoring', + ], + constraints: ['must not leak secrets or PII'], + }, + { + id: 'ci-cd-automation', + title: 'CI/CD pipeline automation', + triggers: [/\bci\b/, /\bcd\b/, /\bgithub actions\b/, /\bpipeline\b/, /\bdeploy\b/], + currentState: 'Builds or deploys are manual or partially automated', + desiredOutcome: + 'Automate build, test, and deploy with SHA-tagged releases and green-main policy', + priority: 'medium', + searchIntents: [ + 'github actions ci cd pipeline', + 'automated deployment pipeline', + 'sha tagged release', + ], + constraints: ['production deploys remain manual'], + }, + { + id: 'data-migration-management', + title: 'Database schema migration management', + triggers: [/\bmigration\b/, /\bschema\b/, /\bd1\b/, /\bsqlite\b/, /\bturso\b/], + currentState: 'Schema changes are manual or lack ordered versioning', + desiredOutcome: 'Use ordered, additive SQL migrations with local and remote apply paths', + priority: 'medium', + searchIntents: [ + 'database schema migration tool', + 'ordered sql migrations', + 'sqlite migration patterns', + ], + constraints: ['migrations must be additive and reversible'], + }, +]; + +/** + * Extract evidence-backed needs from a project. Returns 1-10 needs; fewer when + * evidence is insufficient. Never invents needs to fill a quota. + */ +export function extractNeeds(project: ProjectRecommendationRepo): ProjectNeed[] { + const source = [ + project.name, + project.description, + project.language, + ...project.topics, + project.aiSummary, + project.aiCategory, + ...(project.aiKeywords ?? []), + ...project.tools.map((t) => `${t.key} ${t.name} ${t.category}`), + ] + .filter(Boolean) + .join('\n') + .toLowerCase(); + + if (!source.trim()) { + // Insufficient evidence — return a single conservative need. + const fallback: ProjectNeed = { + id: `${project.id}-health`, + title: 'Project health and dependency maintenance', + currentState: 'No specific needs were extracted from available evidence', + desiredOutcome: + 'Keep dependencies current, remove dead code, and monitor security advisories', + priority: 'low', + constraints: ['must not break existing behavior'], + evidence: ['Limited evidence available for targeted need extraction'], + searchIntents: ['dependency health tool', 'dead code detector', 'security audit automation'], + signature: '', + }; + fallback.signature = needSignature(fallback); + return [fallback]; + } + + const needs: ProjectNeed[] = []; + const seen = new Set(); + + for (const rule of NEED_RULES) { + if (needs.length >= MAX_NEEDS) break; + if (seen.has(rule.id)) continue; + + const matched = rule.triggers.some((re) => re.test(source)); + if (!matched) continue; + + const evidence: string[] = []; + for (const re of rule.triggers) { + const match = source.match(re); + if (match) evidence.push(`Evidence keyword: "${match[0]}"`); + } + if (project.aiCategory) evidence.push(`AI category: ${project.aiCategory}`); + if (project.topics.length > 0) + evidence.push(`Topics: ${project.topics.slice(0, 5).join(', ')}`); + + if (evidence.length < MIN_NEED_EVIDENCE) continue; + + const need: ProjectNeed = { + id: `${project.id}-${rule.id}`, + title: rule.title, + currentState: rule.currentState, + desiredOutcome: rule.desiredOutcome, + priority: rule.priority, + constraints: rule.constraints, + evidence, + searchIntents: rule.searchIntents, + signature: '', + }; + need.signature = needSignature(need); + seen.add(rule.id); + needs.push(need); + } + + // If no rules matched despite having some source text, return a single + // conservative need rather than an empty list. + if (needs.length === 0) { + const fallback: ProjectNeed = { + id: `${project.id}-health`, + title: 'Project health and dependency maintenance', + currentState: 'No specific needs were extracted from available evidence', + desiredOutcome: + 'Keep dependencies current, remove dead code, and monitor security advisories', + priority: 'low', + constraints: ['must not break existing behavior'], + evidence: ['Available evidence did not match any specific need rule'], + searchIntents: ['dependency health tool', 'dead code detector', 'security audit automation'], + signature: '', + }; + fallback.signature = needSignature(fallback); + return [fallback]; + } + + return needs.slice(0, MAX_NEEDS); +} + +/** + * Merge overlapping needs by signature. Retains the higher-priority need. + */ +export function mergeNeeds(needs: ProjectNeed[]): ProjectNeed[] { + const bySignature = new Map(); + const priorityWeight: Record = { high: 3, medium: 2, low: 1 }; + + for (const need of needs) { + const existing = bySignature.get(need.signature); + if (!existing || priorityWeight[need.priority] > priorityWeight[existing.priority]) { + bySignature.set(need.signature, need); + } + } + + return Array.from(bySignature.values()).sort( + (a, b) => priorityWeight[b.priority] - priorityWeight[a.priority] + ); +} + +/** + * Reject needs with insufficient evidence. A need must have at least + * MIN_NEED_EVIDENCE evidence entries to be retained. + */ +export function rejectUnsupportedNeeds(needs: ProjectNeed[]): { + retained: ProjectNeed[]; + rejected: ProjectNeed[]; +} { + const retained: ProjectNeed[] = []; + const rejected: ProjectNeed[] = []; + for (const need of needs) { + if (need.evidence.length >= MIN_NEED_EVIDENCE) { + retained.push(need); + } else { + rejected.push(need); + } + } + return { retained, rejected }; +} + +// --------------------------------------------------------------------------- +// Per-need retrieval (Stage 4) +// --------------------------------------------------------------------------- + +function projectText(project: ProjectRecommendationRepo): string { + return buildRepoEmbeddingText({ + full_name: project.fullName, + description: project.description, + language: project.language, + topics: project.topics, + ai: { + summary: project.aiSummary, + category: project.aiCategory, + keywords: project.aiKeywords, + }, + }); +} + +async function semanticCandidatesForNeed( + need: ProjectNeed, + embed: (texts: string[]) => Promise, + vectorStore: VectorStore +): Promise { + const queryText = need.searchIntents.join(' '); + const [embedding] = await embed([queryText]); + if (!embedding) return []; + const matches = await vectorStore.query(embedding, VECTOR_TOP_K); + return matches.filter((m) => m.distance <= VECTOR_DISTANCE_MAX).map((m) => m.repoId); +} + +async function lexicalCandidatesForNeed( + need: ProjectNeed, + database: NeedDrivenIntelligenceDependencies['database'] +): Promise { + const query = ftsSearchQuery(need.searchIntents.join(' ')); + if (!query) return []; + try { + const result = await database.execute({ + sql: `SELECT r.id, MIN(matches.rank) AS best_rank + FROM ( + SELECT repos_fts.rowid AS id, + bm25(repos_fts, 10.0, 14.0, 3.0, 1.5, 2.5) AS rank + FROM repos_fts + WHERE repos_fts MATCH ? + UNION ALL + SELECT repo_ai_metadata_fts.rowid AS id, + bm25(repo_ai_metadata_fts, 4.0, 3.0, 2.0, 2.0, 2.5) AS rank + FROM repo_ai_metadata_fts + WHERE repo_ai_metadata_fts MATCH ? + ) matches + JOIN repos r ON r.id = matches.id + WHERE r.archived = 0 AND ${ELIGIBLE_REPO_SQL} + GROUP BY r.id + ORDER BY best_rank ASC, r.stargazers_count DESC + LIMIT ?`, + args: [query, query, MIN_STARS_FLOOR, LEXICAL_LIMIT], + }); + return result.rows.map((row) => Number(row.id)).filter(Number.isSafeInteger); + } catch { + return []; + } +} + +async function structuredCandidatesForNeed( + _need: ProjectNeed, + project: ProjectRecommendationRepo, + database: NeedDrivenIntelligenceDependencies['database'] +): Promise { + if (!project.language) return []; + try { + const result = await database.execute({ + sql: `SELECT r.id + FROM repos r + WHERE r.archived = 0 + AND r.language = ? COLLATE NOCASE + AND ${ELIGIBLE_REPO_SQL} + ORDER BY r.stargazers_count DESC, r.full_name ASC + LIMIT ?`, + args: [project.language, MIN_STARS_FLOOR, STRUCTURED_LIMIT], + }); + return result.rows.map((row) => Number(row.id)).filter(Number.isSafeInteger); + } catch { + return []; + } +} + +async function hydrateCandidates( + ids: number[], + projectId: number, + database: NeedDrivenIntelligenceDependencies['database'] +): Promise { + if (ids.length === 0) return []; + const result = await database.execute({ + sql: `SELECT r.id, + r.name, + r.full_name, + r.html_url, + r.description, + r.language, + r.stargazers_count, + r.archived, + r.topics, + aim.summary AS ai_summary, + aim.category AS ai_category, + aim.keywords AS ai_keywords, + COALESCE(( + SELECT json_group_array(json_object( + 'key', rt.tool_key, + 'name', rt.tool_name, + 'category', rt.category, + 'confidence', rt.confidence + )) + FROM repo_tools rt WHERE rt.repo_id = r.id + ), '[]') AS tools + FROM repos r + LEFT JOIN repo_ai_metadata aim ON aim.repo_id = r.id + WHERE r.id != ? + AND r.archived = 0 + AND ${ELIGIBLE_REPO_SQL} + AND r.id IN (SELECT CAST(value AS INTEGER) FROM json_each(?))`, + args: [projectId, MIN_STARS_FLOOR, JSON.stringify(ids)], + }); + const byId = new Map(); + for (const row of result.rows) { + const repo: ProjectRecommendationRepo = { + id: Number(row.id), + name: String(row.name), + fullName: String(row.full_name), + htmlUrl: String(row.html_url), + description: typeof row.description === 'string' ? row.description : null, + language: typeof row.language === 'string' ? row.language : null, + stargazersCount: Number(row.stargazers_count ?? 0), + archived: Boolean(row.archived), + topics: parseStringArray(row.topics), + aiSummary: typeof row.ai_summary === 'string' ? row.ai_summary : null, + aiCategory: typeof row.ai_category === 'string' ? row.ai_category : null, + aiKeywords: parseStringArray(row.ai_keywords), + tools: parseToolSignals(row.tools), + }; + byId.set(repo.id, repo); + } + return ids.flatMap((id) => { + const repo = byId.get(id); + return repo ? [repo] : []; + }); +} + +// --------------------------------------------------------------------------- +// Classification (Stage 5) +// --------------------------------------------------------------------------- + +function classifyCandidate( + project: ProjectRecommendationRepo, + need: ProjectNeed, + candidate: ProjectRecommendationRepo +): { + classification: CandidateClassification; + confidence: Confidence; + evidence: string[]; + score: number; +} { + const evidence: string[] = []; + let score = 0; + + // Language match + if ( + project.language && + candidate.language && + project.language.toLowerCase() === candidate.language.toLowerCase() + ) { + score += 6; + evidence.push(`Same primary language: ${project.language}`); + } + + // Topic overlap + const projectTopics = new Set(project.topics.map((t) => t.toLowerCase())); + const candidateTopics = new Set(candidate.topics.map((t) => t.toLowerCase())); + const topicMatches = [...projectTopics].filter((t) => candidateTopics.has(t)).slice(0, 3); + if (topicMatches.length > 0) { + score += topicMatches.length * 10; + evidence.push(`Shared topics: ${topicMatches.join(', ')}`); + } + + // Tool overlap + const projectToolKeys = new Set(project.tools.map((t) => t.key.toLowerCase())); + const candidateToolKeys = new Set(candidate.tools.map((t) => t.key.toLowerCase())); + const toolMatches = [...projectToolKeys].filter((t) => candidateToolKeys.has(t)).slice(0, 3); + if (toolMatches.length > 0) { + score += toolMatches.length * 12; + const names = toolMatches.map( + (key) => candidate.tools.find((t) => t.key.toLowerCase() === key)?.name ?? key + ); + evidence.push(`Shared tools: ${names.join(', ')}`); + } + + // Need keyword overlap + const needText = `${need.title} ${need.searchIntents.join(' ')}`.toLowerCase(); + const candidateText = + `${candidate.fullName} ${candidate.description ?? ''} ${candidate.topics.join(' ')}`.toLowerCase(); + const needWords: string[] = needText.match(/[a-z0-9+#.-]{3,}/g) ?? []; + const overlap = needWords.filter((w) => candidateText.includes(w) && w.length > 3); + if (overlap.length > 0) { + score += Math.min(overlap.length, 5) * 4; + evidence.push(`Need keyword overlap: ${[...new Set(overlap)].slice(0, 5).join(', ')}`); + } + + // Maintenance signals + if (candidate.archived) { + score -= 20; + evidence.push('Repository is archived'); + } + if (candidate.stargazersCount >= 50_000) { + score += 4; + evidence.push(`High adoption: ${candidate.stargazersCount} stars`); + } + + // Classification logic + let classification: CandidateClassification; + let confidence: Confidence; + + const projectFullNameLower = project.fullName.toLowerCase(); + const candidateDescLower = (candidate.description ?? '').toLowerCase(); + const projectNameLower = project.name.toLowerCase(); + + // Competing product: solves the same end-user problem + if ( + candidate.fullName.toLowerCase().includes(projectNameLower) || + (projectNameLower.length > 3 && candidateDescLower.includes(projectNameLower)) + ) { + classification = 'competing_product_to_monitor'; + confidence = score >= 20 ? 'high' : 'medium'; + evidence.push('Name/description overlap with project — likely a competing product'); + } else if (candidate.archived || score < 0) { + classification = 'unsuitable_negative_example'; + confidence = 'high'; + if (!candidate.archived) evidence.push('Low relevance score'); + } else if (score >= 25 && toolMatches.length > 0) { + classification = 'adopt_or_integrate'; + confidence = score >= 35 ? 'high' : 'medium'; + } else if (score >= 15) { + classification = 'reference_implementation'; + confidence = score >= 25 ? 'high' : 'medium'; + } else if (score >= 8) { + classification = 'architectural_pattern'; + confidence = 'low'; + } else { + classification = 'reference_implementation'; + confidence = 'low'; + evidence.push('Weak signal — retained as a reference only'); + } + + return { classification, confidence, evidence, score }; +} + +function retrievalMode( + semantic: number, + lexical: number, + structured: number, + fallback: boolean +): ProjectRetrievalMode { + if (fallback) return 'fallback'; + if (semantic > 0 && (lexical > 0 || structured > 0)) return 'hybrid'; + if (semantic > 0) return 'semantic'; + if (lexical > 0) return 'lexical-structured'; + return 'structured'; +} + +// --------------------------------------------------------------------------- +// Candidate pool caching +// --------------------------------------------------------------------------- + +async function loadCachedCandidatePool( + need: ProjectNeed, + constraintsHash: string, + catalogGeneration: string, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + const result = await dependencies.database.execute({ + sql: `SELECT candidate_ids FROM need_candidate_pools + WHERE signature = ? AND retrieval_version = ? AND catalog_generation = ? AND constraints_hash = ?`, + args: [need.signature, RETRIEVAL_VERSION, catalogGeneration, constraintsHash], + }); + const ids = result.rows[0]?.candidate_ids; + if (typeof ids !== 'string') return null; + try { + const parsed = JSON.parse(ids) as unknown; + return Array.isArray(parsed) + ? parsed.filter((v): v is number => typeof v === 'number' && Number.isSafeInteger(v)) + : null; + } catch { + return null; + } +} + +async function storeCandidatePool( + need: ProjectNeed, + constraintsHash: string, + catalogGeneration: string, + candidateIds: number[], + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + await dependencies.database.execute({ + sql: `INSERT INTO need_candidate_pools + (signature, retrieval_version, catalog_generation, constraints_hash, candidate_ids, candidate_count) + VALUES (?, ?, ?, ?, ?, ?) + ON CONFLICT(signature, retrieval_version, catalog_generation, constraints_hash) + DO UPDATE SET candidate_ids = excluded.candidate_ids, candidate_count = excluded.candidate_count`, + args: [ + need.signature, + RETRIEVAL_VERSION, + catalogGeneration, + constraintsHash, + JSON.stringify(candidateIds), + candidateIds.length, + ], + }); +} + +function constraintsHash(project: ProjectRecommendationRepo): string { + return fingerprintTexts([ + project.language ?? '', + JSON.stringify(project.topics.sort()), + JSON.stringify(project.tools.map((t) => t.key).sort()), + ]); +} + +// --------------------------------------------------------------------------- +// Draft report persistence (Stage 6) +// --------------------------------------------------------------------------- + +async function markPreviousLatestDraft( + repoId: number, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + await dependencies.database.execute({ + sql: 'UPDATE project_draft_reports SET is_latest = 0 WHERE repo_id = ? AND is_latest = 1', + args: [repoId], + }); +} + +export async function persistDraftReport( + report: DraftReport, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + await markPreviousLatestDraft(report.repoId, dependencies); + const result = await dependencies.database.execute({ + sql: `INSERT INTO project_draft_reports + (repo_id, fingerprint, catalog_generation, retrieval_version, status, + report, needs_count, candidates_count, provenance, is_latest) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1)`, + args: [ + report.repoId, + report.fingerprint, + report.catalogGeneration, + report.retrievalVersion, + report.status, + JSON.stringify(report), + report.needsCount, + report.candidatesCount, + JSON.stringify(report.provenance), + ], + }); + return result.lastInsertRowid ?? 0; +} + +export async function loadLatestDraftReport( + repoId: number, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + const result = await dependencies.database.execute({ + sql: `SELECT report FROM project_draft_reports + WHERE repo_id = ? AND is_latest = 1 + ORDER BY created_at DESC LIMIT 1`, + args: [repoId], + }); + const raw = result.rows[0]?.report; + if (typeof raw !== 'string') return null; + try { + return JSON.parse(raw) as DraftReport; + } catch { + return null; + } +} + +// --------------------------------------------------------------------------- +// External review contract (Stage 7 — ingestion only) +// --------------------------------------------------------------------------- + +export function reviewRequestHash(report: DraftReport): string { + return fingerprintTexts([ + report.repoId.toString(), + report.fingerprint, + report.catalogGeneration, + JSON.stringify( + report.needs.map((n) => ({ + id: n.need.id, + candidates: n.candidates.map((c) => c.repoId), + })) + ), + ]); +} + +export function reviewIdempotencyKey(repoId: number, requestHash: string): string { + return fingerprintTexts([repoId.toString(), requestHash]); +} + +export async function createExternalReviewRequest( + repoId: number, + draftReportId: number, + report: DraftReport, + dependencies: NeedDrivenIntelligenceDependencies +): Promise<{ request: ExternalReviewRequest; created: boolean }> { + const requestHash = reviewRequestHash(report); + const idempotencyKey = reviewIdempotencyKey(repoId, requestHash); + + const existing = await dependencies.database.execute({ + sql: 'SELECT id, status FROM external_review_requests WHERE idempotency_key = ?', + args: [idempotencyKey], + }); + if (existing.rows.length > 0) { + return { + request: { + idempotencyKey, + repoId, + draftReportId, + requestHash, + status: existing.rows[0].status as ReviewStatus, + }, + created: false, + }; + } + + await dependencies.database.execute({ + sql: `INSERT INTO external_review_requests + (idempotency_key, repo_id, draft_report_id, request_hash, status) + VALUES (?, ?, ?, ?, 'pending')`, + args: [idempotencyKey, repoId, draftReportId, requestHash], + }); + + return { + request: { idempotencyKey, repoId, draftReportId, requestHash, status: 'pending' }, + created: true, + }; +} + +export interface ReviewIngestionResult { + reviewedReportId: number; + created: boolean; +} + +/** + * Ingest an external review result. Idempotent — duplicate submissions with + * the same idempotency key return the existing reviewed report. + */ +export async function ingestExternalReview( + result: ExternalReviewResult, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + // Check for existing submission + const existing = await dependencies.database.execute({ + sql: `SELECT id, status FROM external_review_requests WHERE idempotency_key = ?`, + args: [result.idempotencyKey], + }); + if (existing.rows.length === 0) { + throw new Error('No matching external review request for idempotency key'); + } + + const requestId = Number(existing.rows[0].id); + const existingStatus = existing.rows[0].status; + if (existingStatus === 'complete') { + // Idempotent — return existing reviewed report + const reviewed = await dependencies.database.execute({ + sql: `SELECT id FROM project_reviewed_reports WHERE review_request_id = ? ORDER BY created_at DESC LIMIT 1`, + args: [requestId], + }); + const reviewedId = Number(reviewed.rows[0]?.id ?? 0); + return { reviewedReportId: reviewedId, created: false }; + } + + // Load the draft report + const requestRow = await dependencies.database.execute({ + sql: 'SELECT repo_id, draft_report_id FROM external_review_requests WHERE id = ?', + args: [requestId], + }); + const repoId = Number(requestRow.rows[0]?.repo_id); + const draftReportId = Number(requestRow.rows[0]?.draft_report_id); + + const draftResult = await dependencies.database.execute({ + sql: 'SELECT report FROM project_draft_reports WHERE id = ?', + args: [draftReportId], + }); + const draftRaw = draftResult.rows[0]?.report; + if (typeof draftRaw !== 'string') { + throw new Error('Draft report not found for external review ingestion'); + } + const draft = JSON.parse(draftRaw) as DraftReport; + + // Apply verdicts to the draft + const appliedReport = applyReviewVerdicts(draft, result.verdicts); + + // Mark previous reviewed reports as non-latest + await dependencies.database.execute({ + sql: 'UPDATE project_reviewed_reports SET is_latest = 0 WHERE repo_id = ? AND is_latest = 1', + args: [repoId], + }); + + // Persist the reviewed report + const insertResult = await dependencies.database.execute({ + sql: `INSERT INTO project_reviewed_reports + (repo_id, draft_report_id, review_request_id, status, report, + reviewer_provider, reviewer_model, reviewer_usage, provenance, is_latest) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1)`, + args: [ + repoId, + draftReportId, + requestId, + 'complete', + JSON.stringify(appliedReport), + result.reviewerProvider, + result.reviewerModel, + JSON.stringify(result.reviewerUsage), + JSON.stringify( + ['external-review', result.reviewerProvider, result.reviewerModel].filter(Boolean) + ), + ], + }); + + // Update the review request status + await dependencies.database.execute({ + sql: `UPDATE external_review_requests + SET status = 'complete', result = ?, reviewer_provider = ?, reviewer_model = ?, + reviewer_usage = ?, completed_at = datetime('now') + WHERE id = ?`, + args: [ + JSON.stringify(result), + result.reviewerProvider, + result.reviewerModel, + JSON.stringify(result.reviewerUsage), + requestId, + ], + }); + + return { reviewedReportId: insertResult.lastInsertRowid ?? 0, created: true }; +} + +function applyReviewVerdicts(draft: DraftReport, verdicts: ReviewVerdict[]): DraftReport { + const verdictsByNeedId = new Map(verdicts.map((v) => [v.needId, v])); + const rejectedCandidateIds = new Set(); + for (const v of verdicts) { + for (const id of v.rejectedCandidateIds ?? []) { + rejectedCandidateIds.add(id); + } + } + + const needs: NeedReport[] = []; + for (const needReport of draft.needs) { + const verdict = verdictsByNeedId.get(needReport.need.id); + if (verdict?.verdict === 'unsupported') { + // Skip unsupported needs + continue; + } + const filteredCandidates = needReport.candidates.filter( + (c) => !rejectedCandidateIds.has(c.repoId) + ); + const refinedNeed = verdict?.refinedNeed + ? { ...needReport.need, ...verdict.refinedNeed } + : needReport.need; + needs.push({ ...needReport, need: refinedNeed, candidates: filteredCandidates }); + } + + return { + ...draft, + needs, + needsCount: needs.length, + candidatesCount: needs.reduce((sum, n) => sum + n.candidates.length, 0), + }; +} + +export async function loadLatestReviewedReport( + repoId: number, + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + const result = await dependencies.database.execute({ + sql: `SELECT id, draft_report_id, review_request_id, status, report, + reviewer_provider, reviewer_model, reviewer_usage, provenance, created_at + FROM project_reviewed_reports + WHERE repo_id = ? AND is_latest = 1 + ORDER BY created_at DESC LIMIT 1`, + args: [repoId], + }); + const row = result.rows[0]; + if (!row) return null; + try { + const report = JSON.parse(String(row.report)) as DraftReport; + return { + repoId, + draftReportId: Number(row.draft_report_id), + reviewRequestId: row.review_request_id ? Number(row.review_request_id) : null, + status: row.status as ReviewedReportStatus, + report, + reviewerProvider: typeof row.reviewer_provider === 'string' ? row.reviewer_provider : null, + reviewerModel: typeof row.reviewer_model === 'string' ? row.reviewer_model : null, + reviewerUsage: parseUsage(row.reviewer_usage), + provenance: parseStringArray(row.provenance), + createdAt: String(row.created_at), + }; + } catch { + return null; + } +} + +function parseUsage(value: unknown): Record { + if (typeof value !== 'string' || !value) return {}; + try { + const parsed = JSON.parse(value) as unknown; + if (typeof parsed === 'object' && parsed !== null && !Array.isArray(parsed)) { + return Object.fromEntries( + Object.entries(parsed).map(([k, v]) => [k, Number(v) ?? 0]) + ) as Record; + } + } catch { + // fall through + } + return {}; +} + +// --------------------------------------------------------------------------- +// Pipeline orchestration +// --------------------------------------------------------------------------- + +export interface NeedDrivenPipelineOptions { + catalogGeneration?: string; + candidatesPerNeed?: number; + maxTotalCandidates?: number; +} + +const defaultDependencies: NeedDrivenIntelligenceDependencies = { + database: undefined as never, // Set by createNeedDrivenIntelligence + vectorStore: repoVectors, + embed: generateEmbeddings, +}; + +export function createNeedDrivenIntelligence(dependencies: NeedDrivenIntelligenceDependencies) { + return async function runNeedDrivenIntelligence( + project: ProjectRecommendationRepo, + options: NeedDrivenPipelineOptions = {} + ): Promise { + const catalogGeneration = options.catalogGeneration ?? new Date().toISOString().slice(0, 10); + const candidatesPerNeed = options.candidatesPerNeed ?? CANDIDATES_PER_NEED; + const maxTotalCandidates = options.maxTotalCandidates ?? MAX_TOTAL_CANDIDATES; + const projectConstraintsHash = constraintsHash(project); + + // Stage 2: Fingerprint + const fingerprint = await computeProjectFingerprint(project, dependencies); + const storedFingerprint = await loadStoredFingerprint(project.id, dependencies); + + // Stage 3: Need extraction (with caching) + let needs = extractNeeds(project); + const merged = mergeNeeds(needs); + const { retained } = rejectUnsupportedNeeds(merged); + needs = retained; + + // Check if needs are cached and fingerprint unchanged + if (storedFingerprint === fingerprint.fingerprint) { + const cachedNeeds = await dependencies.database.execute({ + sql: 'SELECT need_id, title, current_state, desired_outcome, priority, constraints, evidence, search_intents, signature FROM project_needs WHERE repo_id = ? AND fingerprint = ?', + args: [project.id, fingerprint.fingerprint], + }); + if (cachedNeeds.rows.length > 0) { + needs = cachedNeeds.rows.map((row) => ({ + id: String(row.need_id), + title: String(row.title), + currentState: String(row.current_state), + desiredOutcome: String(row.desired_outcome), + priority: row.priority as NeedPriority, + constraints: parseStringArray(row.constraints), + evidence: parseStringArray(row.evidence), + searchIntents: parseStringArray(row.search_intents), + signature: String(row.signature), + })); + } else { + await persistNeeds(project.id, fingerprint.fingerprint, needs, dependencies); + } + } else { + await persistNeeds(project.id, fingerprint.fingerprint, needs, dependencies); + } + + // Stage 4+5: Per-need retrieval and classification + const needReports: NeedReport[] = []; + let totalCandidates = 0; + const allCandidateIds = new Set(); + const provenance: string[] = [ + `fingerprint:${fingerprint.fingerprint.slice(0, 16)}`, + `catalog:${catalogGeneration}`, + `retrieval:${RETRIEVAL_VERSION}`, + ]; + let hasDegradation = false; + + for (const need of needs) { + if (totalCandidates >= maxTotalCandidates) break; + + // Check cached candidate pool + let candidateIds = await loadCachedCandidatePool( + need, + projectConstraintsHash, + catalogGeneration, + dependencies + ); + + let mode: ProjectRetrievalMode; + if (candidateIds) { + mode = 'hybrid'; + provenance.push(`cached-pool:${need.id}`); + } else { + const [semanticResult, lexicalResult, structuredResult] = await Promise.allSettled([ + semanticCandidatesForNeed(need, dependencies.embed, dependencies.vectorStore()), + lexicalCandidatesForNeed(need, dependencies.database), + structuredCandidatesForNeed(need, project, dependencies.database), + ]); + const semanticIds = semanticResult.status === 'fulfilled' ? semanticResult.value : []; + const lexicalIds = lexicalResult.status === 'fulfilled' ? lexicalResult.value : []; + const structuredIds = structuredResult.status === 'fulfilled' ? structuredResult.value : []; + + if (semanticResult.status === 'rejected') hasDegradation = true; + if (lexicalResult.status === 'rejected') hasDegradation = true; + + const fused = rrfFuse([semanticIds, lexicalIds, structuredIds]).slice(0, HYDRATION_LIMIT); + candidateIds = fused; + mode = retrievalMode( + semanticIds.length, + lexicalIds.length, + structuredIds.length, + fused.length === 0 + ); + + if (fused.length > 0) { + await storeCandidatePool( + need, + projectConstraintsHash, + catalogGeneration, + fused, + dependencies + ); + } + } + + // Hydrate and classify + const hydrated = await hydrateCandidates(candidateIds, project.id, dependencies.database); + const classified: NeedCandidate[] = hydrated + .map((repo) => { + const result = classifyCandidate(project, need, repo); + return { + repoId: repo.id, + fullName: repo.fullName, + htmlUrl: repo.htmlUrl, + description: repo.description, + language: repo.language, + stargazersCount: repo.stargazersCount, + archived: repo.archived, + topics: repo.topics, + tools: repo.tools, + classification: result.classification, + confidence: result.confidence, + evidence: result.evidence, + score: result.score, + }; + }) + .sort((a, b) => b.score - a.score) + .slice(0, candidatesPerNeed); + + for (const c of classified) { + allCandidateIds.add(c.repoId); + } + totalCandidates += classified.length; + + needReports.push({ need, candidates: classified, retrievalMode: mode }); + } + + // Stage 1: Refresh capability cards for all candidates + if (allCandidateIds.size > 0) { + try { + await refreshCapabilityCards([...allCandidateIds], dependencies); + provenance.push('capability-cards:refreshed'); + } catch { + hasDegradation = true; + provenance.push('capability-cards:skipped'); + } + } + + const status: ReportStatus = hasDegradation ? 'degraded' : 'complete'; + const report: DraftReport = { + repoId: project.id, + fingerprint: fingerprint.fingerprint, + catalogGeneration, + retrievalVersion: RETRIEVAL_VERSION, + status, + needs: needReports, + needsCount: needReports.length, + candidatesCount: totalCandidates, + provenance, + createdAt: new Date().toISOString(), + }; + + // Stage 6: Persist draft report + await persistDraftReport(report, dependencies); + + return report; + }; +} + +async function persistNeeds( + repoId: number, + fingerprint: string, + needs: ProjectNeed[], + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + // Delete old needs for this repo + await dependencies.database.execute({ + sql: 'DELETE FROM project_needs WHERE repo_id = ?', + args: [repoId], + }); + if (needs.length === 0) return; + const statements: InStatement[] = needs.map((need) => ({ + sql: `INSERT INTO project_needs + (repo_id, need_id, title, current_state, desired_outcome, priority, + constraints, evidence, search_intents, signature, fingerprint) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + args: [ + repoId, + need.id, + need.title, + need.currentState, + need.desiredOutcome, + need.priority, + JSON.stringify(need.constraints), + JSON.stringify(need.evidence), + JSON.stringify(need.searchIntents), + need.signature, + fingerprint, + ], + })); + await dependencies.database.batch(statements); +} + +// --------------------------------------------------------------------------- +// Incremental evaluation (Stage 4.3) +// --------------------------------------------------------------------------- + +/** + * Evaluate whether a newly cataloged repository crosses the recommendation + * threshold for any persisted need signature. Returns the need signatures + * that should trigger a rerun. + */ +export async function evaluateNewCatalogAdditions( + newRepoIds: number[], + dependencies: NeedDrivenIntelligenceDependencies +): Promise { + if (newRepoIds.length === 0) return []; + + // Load all persisted need signatures + const signatures = await dependencies.database.execute({ + sql: 'SELECT DISTINCT signature FROM project_needs', + args: [], + }); + const persistedSignatures = signatures.rows + .map((row) => row.signature) + .filter((s): s is string => typeof s === 'string'); + + if (persistedSignatures.length === 0) return []; + + // Check if any new repos match existing need signatures via FTS + const triggered = new Set(); + for (const repoId of newRepoIds) { + const repoResult = await dependencies.database.execute({ + sql: `SELECT r.full_name, r.description, r.language, r.topics, + aim.summary, aim.category, aim.keywords + FROM repos r + LEFT JOIN repo_ai_metadata aim ON aim.repo_id = r.id + WHERE r.id = ?`, + args: [repoId], + }); + const row = repoResult.rows[0]; + if (!row) continue; + + const repoText = [ + row.full_name, + row.description, + row.language, + row.topics, + row.summary, + row.category, + row.keywords, + ] + .filter((v): v is string => typeof v === 'string' && v.length > 0) + .join(' ') + .toLowerCase(); + + // For each persisted need, check if the new repo is relevant + // by looking at the need's search intents stored in project_needs + const needsWithIntents = await dependencies.database.execute({ + sql: 'SELECT DISTINCT signature, search_intents FROM project_needs', + args: [], + }); + for (const needRow of needsWithIntents.rows) { + const sig = needRow.signature; + if (typeof sig !== 'string' || triggered.has(sig)) continue; + const intents = parseStringArray(needRow.search_intents); + const matches = intents.some((intent) => + intent.split(/\s+/).some((word) => word.length > 3 && repoText.includes(word.toLowerCase())) + ); + if (matches) { + triggered.add(sig); + } + } + } + + return [...triggered]; +} + +// --------------------------------------------------------------------------- +// Public read API (no external-agent spend) +// --------------------------------------------------------------------------- + +export async function readProjectIntelligence( + repoId: number, + dependencies: NeedDrivenIntelligenceDependencies +): Promise<{ draft: DraftReport | null; reviewed: ReviewedReport | null }> { + const [draft, reviewed] = await Promise.all([ + loadLatestDraftReport(repoId, dependencies), + loadLatestReviewedReport(repoId, dependencies), + ]); + return { draft, reviewed }; +}