Skip to content

chore: upgrade etl and adopt memory-based batching#48

Open
psteinroe wants to merge 1 commit intomainfrom
chore/upgrade-etl-to-latest-2
Open

chore: upgrade etl and adopt memory-based batching#48
psteinroe wants to merge 1 commit intomainfrom
chore/upgrade-etl-to-latest-2

Conversation

@psteinroe
Copy link
Owner

  • Bump etl and etl-postgres to the latest upstream revision and add sysinfo for memory-aware batch sizing
  • Replace count-based batch config with memory_budget_ratio and max_fill_ms across code, tests, and docs
  • Update replay batching to use byte budgets, emit on errors, and log memory-based batch settings
  • Remove local ETL migrations and switch PostgresStore::new call sites to async initialization

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant