Skip to content

fix(cache): respect selected cache across Text2Model and PDF2Model - #540

Open
rikkurea-boop wants to merge 4 commits into
OpenDCAI:mainfrom
rikkurea-boop:publish/p08-pr-g2t-text-runtime-producer
Open

rikkurea-boop wants to merge 4 commits into
OpenDCAI:mainfrom
rikkurea-boop:publish/p08-pr-g2t-text-runtime-producer

Conversation

@rikkurea-boop

@rikkurea-boop rikkurea-boop commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR consolidates the related cache-routing fixes from #540 (Text2Model) and #541 (PDF2Model), following the maintainer's suggestion to group related changes for easier review.

Text2Model

  • Propagate the selected --cache directory through the training workflow.
  • Restore the missing built-in merge_json_jsonl.py producer.
  • Preserve existing default paths and helper compatibility.
  • Include regression coverage for the real CLI, helper, producer, and filesystem boundaries.

PDF2Model

  • Route the KBC and VQA Step-2 templates through the selected cache.
  • Read the correct detector manifest and write runtime/output artifacts under the selected cache.
  • Preserve legacy standalone behavior when --cache is omitted.
  • Include focused CLI, template, and cache handoff regression tests.

Validation

  • Rebased on the latest upstream main, including previously merged changes.
  • Combined validation: 6/6 environments passed.
  • Upstream PR CI: 6/6 checks passed.
  • Ubuntu Python 3.10: 180 passed, 1 skipped, 48 deselected.
  • Combined targeted suite: 31 passed (re-verified after rebase on Python 3.11).
  • No merge conflicts.

The changes remain logically separated in the commit history.

Limitations

External model/GPU dependencies are mocked in the relevant tests; this does not claim a live training run. Existing PDF2Model workspaces containing older copied templates may need a manual update or reinitialization.

The PDF2Model fix from #541 is now included in this PR.

rikkurea-boop added a commit to rikkurea-boop/DataFlow that referenced this pull request Oct 13, 2026
…nDCAI#540)

Integration candidate for joint cache-routing review. Original upstream PR: OpenDCAI#540
rikkurea-boop added a commit to rikkurea-boop/DataFlow that referenced this pull request Oct 13, 2026
rikkurea-boop added a commit to rikkurea-boop/DataFlow that referenced this pull request Oct 13, 2026
Bring OpenDCAI#536, OpenDCAI#553 and OpenDCAI#551 upstream changes into the existing PR branch
without rebasing away the previously reviewed Text2Model commits.
rikkurea-boop added a commit to rikkurea-boop/DataFlow that referenced this pull request Oct 13, 2026
Consolidate PDF Step-2 cache handoff with existing Text2Model cache routing
under PR OpenDCAI#540, keeping focused regression coverage and standalone defaults.
@rikkurea-boop rikkurea-boop changed the title fix(text2model): route selected cache and restore merge producer fix(cache): respect selected cache across Text2Model and PDF2Model Oct 13, 2026
rikkurea-boop added a commit to rikkurea-boop/DataFlow that referenced this pull request Oct 14, 2026
@rikkurea-boop
rikkurea-boop force-pushed the publish/p08-pr-g2t-text-runtime-producer branch from 4d58573 to 4e1554c Compare November 16, 2026 05:37

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant