Fix remote worker config handoff

[duroey]

Summary

• serialize the orchestrator-dispatched training config as JSON
• point the trainer at the job-scoped config through CONFIG_PATH and the JSON config backend
• remove the unsupported --config and --data_config arguments from the direct simpletuner.train invocation

Root cause

The remote worker received the complete training config, but wrote it as YAML and passed file arguments that the current training entry point does not consume. The subprocess then fell back to the default config/config.json instead of loading the dispatched job config.

This change only fixes the config handoff. Dataset resolution through data_backend_config remains unchanged.

Verification

• regression unit test in the packaged SimpleTuner 4.4.1 environment
• two-node Docker smoke test: the orchestrator dispatched a one-step SDXL LoRA job to a remote worker
• the remote job completed and produced both final LoRA weights and checkpoint-1; all 192 saved tensors were finite
• git diff --check

[Copilot]

Pull request overview

This PR fixes the remote worker → training subprocess configuration handoff by writing the dispatched training config as JSON and ensuring the training entrypoint loads it via environment variables rather than unsupported CLI flags.

Changes:

• Serialize the orchestrator-dispatched job config to config.json in a job-scoped temp directory.
• Launch python -m simpletuner.train without the unsupported --config/--data_config flags and instead provide CONFIG_PATH + SIMPLETUNER_CONFIG_BACKEND=json via the subprocess environment.
• Add an isolated async unit test verifying the config file contents and the subprocess command/env.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
simpletuner/worker_agent.py	Writes JSON config and switches the training subprocess to load configuration via env vars instead of unsupported CLI args.
tests/test_worker_agent.py	Adds regression coverage to ensure the worker writes the correct JSON config and passes the right env to the training subprocess.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.