feat: Add Workflow Analytics Dashboards with OpenSearch integration

[LuD1161]

Summary

This PR adds a Security Analytics platform to ShipSec Studio that enables users to index workflow output data into OpenSearch and visualize it through dashboards. This transforms raw scan outputs into actionable intelligence for security teams.

Commits (for reviewers)

#	Commit	Description
1	feat(infra)	Nginx reverse proxy and production security setup (20 files)
2	feat(workflows)	STALE status and status inference improvements (5 files)
3	feat(analytics)	Main feature: Security Analytics platform with OpenSearch (67 files)

Suggested review order: Start with commit 1 (infra), then 2 (small workflow change), then 3 (main feature).

Key Features

• Analytics Sink Component: New workflow node (core.analytics.sink) that indexes output data from any upstream node to OpenSearch

  • Supports array and object inputs with automatic bulk indexing
  • Auto-detects asset correlation keys (host, domain, subdomain, url, ip, etc.)
  • Configurable index suffix and fail-on-error modes
  • Fire-and-forget by default with retry logic (3 attempts with exponential backoff)

• OpenSearch Integration:

  • Daily index rotation pattern: security-findings-{orgId}-{YYYY.MM.DD}
  • Index template with standard metadata fields
  • Multi-tenant data isolation per organization

• Analytics API:

  • POST /api/v1/analytics/query endpoint supporting OpenSearch DSL
  • Auto-scopes queries to organization's index pattern
  • Rate limiting: 100 requests/minute per user

• Analytics Settings Page:

  • Tier-based retention configuration (Free: 30d, Pro: 90d, Enterprise: 365d)
  • Admin-only access controls

• UI Integration:

  • "Dashboards" link in sidebar (opens OpenSearch Dashboards in new tab)
  • "Analytics Settings" page for retention configuration
  • "View Analytics" button on workflow detail page

• Nginx Reverse Proxy:

  • Unified entry point at http://localhost
  • Routes: / (frontend), /api (backend), /analytics (OpenSearch Dashboards)
  • Proper CORS and proxy header configuration

• OpenSearch Dashboards basePath:

  • Configured with /analytics base path for reverse proxy compatibility
  • Updated init scripts and health checks

• Production Security Infrastructure:

  • TLS encryption for OpenSearch transport and HTTP layers
  • Security plugin with role-based access control
  • SaaS multitenancy with per-customer tenant isolation
  • Index patterns scoped by customer ID ({customer_id}-*)
  • Certificate generation script (just generate-certs)
  • Production deployment guide (docker/PRODUCTION.md)

• Workflow Status Improvements:

  • New STALE status for orphaned run records (DB/Temporal mismatch)
  • Improved status inference from trace events when Temporal workflow not found
  • Documentation for all execution statuses

• Component SDK Extensions:

  • generateFindingHash() utility for deduplication
  • Workflow context (workflowId, workflowName, organizationId) passed to components
  • Results output port added to nuclei, trufflehog, and supabase-scanner components
  • Support for optional inputs in components

New Commands

just dev              # Start dev with nginx reverse proxy
just prod-secure      # Start production with security & multitenancy
just generate-certs   # Generate TLS certificates for production

Files Changed

90+ files across backend, frontend, worker, component-sdk, docker, and documentation.

Test plan

• Run npm run typecheck to verify no type errors
• Run npm run lint to verify code quality
• Start infrastructure: just dev or docker compose -f docker/docker-compose.infra.yml up -d
• Run index template setup: OPENSEARCH_URL=http://localhost:9200 npm run --prefix backend setup:opensearch
• Test Analytics API endpoint: POST /api/v1/analytics/query (requires Basic Auth: admin:admin)
• Verify Dashboards accessible at http://localhost/analytics
• Verify nginx routing works for all paths
• Create workflow with Analytics Sink component and verify data indexed
• Test production security setup with just prod-secure

Screenshots

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 42044b8c24

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[betterclever]

Question: Scope of User Stories

Looking at tasks/prd.json, I see all user stories (US-001 through US-015) are marked with "passes": true, including:

• US-012: Analytics settings page UI ✅
• US-013: Retention settings API endpoints ✅

However, frontend/src/pages/AnalyticsSettingsPage.tsx still contains:

• Mock data for current tier and retention (line 42)
• TODO comments referencing US-013 for API integration (lines 58, 70)

Question: Are all 15 user stories expected to be completed in this PR, or is US-013 (the backend API) intentionally deferred to a later PR? If it's expected to be complete, the frontend may need to be wired up to the backend endpoints that were reportedly implemented.

[LuD1161]

Question: Scope of User Stories

Looking at tasks/prd.json, I see all user stories (US-001 through US-015) are marked with "passes": true, including:

* **US-012**: Analytics settings page UI ✅

* **US-013**: Retention settings API endpoints ✅

However, frontend/src/pages/AnalyticsSettingsPage.tsx still contains:

* Mock data for current tier and retention (line 42)

* TODO comments referencing US-013 for API integration (lines 58, 70)

Question: Are all 15 user stories expected to be completed in this PR, or is US-013 (the backend API) intentionally deferred to a later PR? If it's expected to be complete, the frontend may need to be wired up to the backend endpoints that were reportedly implemented.

This was a relic from the hottest trend in AI, of ralph . Henceforth removed :)