Skip to content

Rust refactor: dedupe ingest filesystem walks + per-harness apply boilerplate #343

@willwashburn

Description

@willwashburn

Context

Two parallel duplications in the ingest module.

1. list_dirs / list_jsonl_files / walk_files exist three times.

  • crates/relayburn-sdk/src/ingest/walk.rs:14, 22-24
  • crates/relayburn-sdk/src/ingest/ingest.rs:933-952
  • crates/relayburn-sdk/src/ingest/reingest.rs:336-375

Plus a related case-sensitivity bug in walk.rs:14, 22-24: name.ends_with(".jsonl") skips .JSONL. Clippy already flags this.

2. Four ingest verbs and three ingest_*_into functions are 70-80% identical.

  • ingest/ingest.rs:159 ingest_all
  • :194 ingest_claude_projects
  • :213 ingest_codex_sessions
  • :232 ingest_opencode_sessions

…all run the same skeleton: cleanup_stale_pending_stamps → load_cursors → resolve_content_mode → one ingest call → emit_gap_warning → save_cursor_changes.

The per-harness ingest_*_into (:343, 485, 629) each end with a 6-call if !parsed.X.is_empty() { ledger.append_X(&parsed.X)?; } block — repeated yet again in ingest_claude_session:294-309.

Proposed fix

  1. Consolidate the three filesystem-walk copies into ingest/walk.rs. Use eq_ignore_ascii_case for .jsonl. Drop the duplicates.
  2. Extract apply_parsed(ledger, &parsed) -> Result<()> for the per-harness "if not empty append" block. Three call sites collapse to one line each.
  3. Extract wrap_one(adapter, body) (or a closure) for the verb-skeleton boilerplate. The four verbs become thin wrappers that supply the harness-specific body.

Adjacent: ingest/ingest.rs:262 does cwd.chars().map(|c| if c == '/' { '-' } else { c }).collect::<String>()cwd.replace('/', "-") is the idiomatic form.

References

  • crates/relayburn-sdk/src/ingest/walk.rs:14, 22-24
  • crates/relayburn-sdk/src/ingest/ingest.rs:159, 194, 213, 232, 262, 294-309, 343, 485, 629, 933-952
  • crates/relayburn-sdk/src/ingest/reingest.rs:336-375
  • Ingest review notes from the May 2026 Rust review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions