Skip to content

perf: add database indexes to commonly queried entity fields#217

Open
sameh-farouk wants to merge 1 commit intomaster-docs-updatefrom
master-add-db-indexes
Open

perf: add database indexes to commonly queried entity fields#217
sameh-farouk wants to merge 1 commit intomaster-docs-updatefrom
master-add-db-indexes

Conversation

@sameh-farouk
Copy link
Copy Markdown
Member

@sameh-farouk sameh-farouk commented Apr 2, 2026

Summary

Fixes #216. Adds 25 database indexes across 9 entities based on analysis of actual query patterns from all downstream consumers.

Research

Analyzed GraphQL queries and direct SQL access across:

  • tfgrid-sdk-ts (grid_client, playground, dashboard) — direct GraphQL queries
  • tfgrid-sdk-go (grid-client, relay-cache-warmer) — direct GraphQL queries
  • grid-proxy — reads the processor's PostgreSQL directly via GORM/SQL (does NOT use GraphQL API)

Grid-proxy already creates some indexes in its setup.sql, but they get lost on processor DB reset. This PR makes them part of the processor's migration.

Full research details in #216 comments.

Indexed fields (25 total)

Entity Fields
Twin twinID
Farm farmID, name, twinID, dedicatedFarm
PublicIp ip
Node nodeID, farmID, twinID, certification
NodeContract contractID, twinID, nodeID, state
NameContract contractID, twinID, state
RentContract contractID, twinID, nodeID, state
ContractBillReport contractID, timestamp
UptimeEvent nodeID, timestamp

What changed

  • schema.graphql — added @index directive to 25 fields
  • src/model/generated/*.model.ts — regenerated by yarn codegen with @Index_() decorators
  • db/migrations/1775123913074-Data.js — new migration with 25 CREATE INDEX statements

Backward compatibility

  • No table/column changes — migration contains only CREATE INDEX statements
  • Grid-proxy's setup.sql indexes use CREATE INDEX IF NOT EXISTS — they coexist without conflict
  • No processor reindex needed — indexes are added to existing data

Test plan

  • yarn build passes
  • Migration generated with exactly 25 CREATE INDEX statements, no other alterations
  • Apply migration on dev/qa and verify query performance improvement

@sameh-farouk sameh-farouk linked an issue Apr 2, 2026 that may be closed by this pull request
@sameh-farouk sameh-farouk force-pushed the master-add-db-indexes branch from 3ea7db0 to 319b7bb Compare April 3, 2026 23:08
@sameh-farouk sameh-farouk force-pushed the master-add-db-indexes branch from 319b7bb to 6bd4b06 Compare April 4, 2026 01:12
@sameh-farouk sameh-farouk force-pushed the master-add-db-indexes branch from 6bd4b06 to e92e002 Compare April 4, 2026 02:31
@sameh-farouk sameh-farouk force-pushed the master-add-db-indexes branch from e92e002 to 2a3a713 Compare April 5, 2026 00:03
@sameh-farouk sameh-farouk force-pushed the master-add-db-indexes branch from 2a3a713 to d93b222 Compare April 5, 2026 00:53
Add @index directives to schema.graphql for 25 fields across 9
entities, based on analysis of actual query patterns from all
downstream consumers:

- GraphQL API consumers: tfgrid-sdk-ts (grid_client, playground,
  dashboard), tfgrid-sdk-go (grid-client, relay-cache-warmer)
- Direct DB consumer: grid-proxy reads the processor's PostgreSQL
  directly via GORM/SQL

Indexed fields:
- Twin: twinID
- Farm: farmID, name, twinID, dedicatedFarm
- PublicIp: ip
- Node: nodeID, farmID, twinID, certification
- NodeContract: contractID, twinID, nodeID, state
- NameContract: contractID, twinID, state
- RentContract: contractID, twinID, nodeID, state
- ContractBillReport: contractID, timestamp
- UptimeEvent: nodeID, timestamp

No table/column changes — migration contains only CREATE INDEX
statements. Grid-proxy's setup.sql indexes (IF NOT EXISTS) coexist
without conflict. No processor reindex needed.

Fixes: #216

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sameh-farouk sameh-farouk force-pushed the master-add-db-indexes branch from d93b222 to 1e96cca Compare April 6, 2026 14:43
@sameh-farouk
Copy link
Copy Markdown
Member Author

sameh-farouk commented Apr 7, 2026

Tetsed on test vm
Test VM is consistently faster across all queries. The standout is ContractBillReports by contractID: 4.3x faster (177ms vs 754ms) that's the 8.2M row table where the index makes the biggest difference.
The general 1.6x improvement across all queries is likely a combination of the DB indexes, and The network latency difference (prod is behind HTTPS) also plays a role, But the contractBillReports result (4.3x) is clearly index-driven.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add database indexes to commonly queried entity fields

1 participant