Add local submission history log to guard daily submission slots

[k66inthesky]

Why

submission.md already warns "never rerun this script blindly — each successful submission call can spend one competition submission slot" and the skill's troubleshooting guidance tells agents to read existing logs before retrying, but nothing writes a structured log. When a submit_kernel.py run crashes mid-poll or its terminal output is gone, an agent has no local way to tell whether the daily slot was already spent, so the safe-retry guidance can't actually be followed.

This complements submission_quota.py (#9): quota asks Kaggle how many slots remain today; the history log shows what this workspace already submitted and how it scored, with no API call.

What

• submit_kernel.py appends each submission attempt (accepted/failed, with the error text) and each finished evaluation (complete/error/timeout, with the public score) to data/submissions.jsonl — the same project-root data/ convention as the kernel and discussion caches. Logging is best-effort and can never break an in-flight submission.
• New scripts/submission_log.py holds the append/read/merge logic as pure functions; evaluation events fold into their submission attempt by the same (competition, message) key submit_kernel.py uses to match submissions on Kaggle.
• New scripts/submission_history.py renders the merged history (Rich table, or --as-json), filterable by competition slug/URL, kernel ref, and limit. Local-only: no Kaggle API calls, no credentials.
• submission.md documents the log, the history command, and points the "never rerun blindly" guidance at it.

Testing

• 13 new offline unit tests in tests/test_submission_history.py: append/read roundtrip, malformed-line tolerance, merge semantics (including reused messages), filtering/limits, the submit_kernel.py hooks via a fake API object, and the CLI's JSON/empty-log output.
• uv run pytest: 55 passed, 12 skipped (baseline before this change: 42 passed, 12 skipped). No live-API tests needed.
• claude plugin validate .: passed.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds a local, append-only submission history log to the Kaggle skill so reruns can determine what has already been submitted/evaluated (and with what score) even if a prior submit_kernel.py run crashed or lost terminal output.

Changes:

• Add submission_log.py to append/read a JSONL log and merge submission+evaluation events into submission-attempt rows.
• Hook submit_kernel.py to log submission attempts (accepted/failed) and evaluation outcomes (complete/error/timeout).
• Add submission_history.py CLI and update submission.md, plus offline unit tests covering log behavior and CLI output.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_submission_history.py	Adds offline unit tests for logging, merging, filtering/limits, and CLI JSON/empty-log behavior.
skills/nvidia-kaggle-skill/submission.md	Documents the new local submission history log and the submission_history.py command.
skills/nvidia-kaggle-skill/scripts/submit_kernel.py	Adds best-effort logging hooks for submission attempts and evaluation outcomes.
skills/nvidia-kaggle-skill/scripts/submission_log.py	Introduces the JSONL append/read utilities and event-to-attempt merge logic.
skills/nvidia-kaggle-skill/scripts/submission_history.py	Adds a Rich/JSON CLI to render merged submission history from the local log.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[alessiodevoto]

Not sure here: should submission_attempts() skip failed submit attempts when folding evaluation events back into attempts ? Right now the scan matches the newest prior (competition, message) with no eval_status, regardless of whether Kaggle accepted that submit.

A possible bad sequence:

1. Run A submits message "baseline v1" successfully.
2. Run B retries with the same message and the submit call fails, e.g. quota/rejection, so it logs accepted: false.
3. Run A’s polling later writes the evaluation event for "baseline v1".

The current merge will attach Run A’s score to Run B’s failed submit row, because Run B is now the newest matching attempt. That makes the history misleading. I think the merge should ignore attempts where accepted is False, plus add a regression test for accepted submit -> failed submit with same message -> evaluation.

[k66inthesky]

Good catch — fixed in 04f3941: the merge now skips attempts with a falsy accepted, and added the regression test you suggested (accepted submit → failed retry with same message → evaluation lands on the accepted attempt).

[alessiodevoto]

Hi @k66inthesky thanks for opening this PR! The PR looks nice overall, I left a couple of small comments 🙂

[k66inthesky]

@alessiodevoto Both comments are addressed in 04f3941 — thanks for the catch!

• submission_attempts() now skips attempts with a falsy accepted, with a regression test for accepted submit → failed retry (same message) → evaluation.
• --as-json now always emits valid JSON ([] for an empty log), also covered by a test.

Local tests are fully passing:

$ pytest tests/test_submission_history.py -q
18 passed in 0.09s

$ pytest tests/ -q
60 passed, 12 skipped in 2.12s