Skip to content

add additional column case in plot_training - #2768

Open
SavvasMel wants to merge 1 commit into
ecmwf:developfrom
SavvasMel:SavvasMel/develop/fix_plot_training
Open

add additional column case in plot_training#2768
SavvasMel wants to merge 1 commit into
ecmwf:developfrom
SavvasMel:SavvasMel/develop/fix_plot_training

Conversation

@SavvasMel

@SavvasMel SavvasMel commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Description

Introduces a small "if statement" to cover channels with "." in the last parts of the string. This allow for:

image

Issue Number

Closes #2767

Is this PR a draft? Mark it as draft.

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

FastEvaluation

  • I have updated the public documentation if necessary

@github-actions github-actions Bot added the eval anything related to the model evaluation pipeline label Aug 18, 2026

@clessig clessig left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion but wouldn't it be best to sanitize the channel names when the data is loaded and, e.g., replace "." with "_"?

@jpolz jpolz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@SavvasMel

Copy link
Copy Markdown
Contributor Author

I don't have a strong opinion but wouldn't it be best to sanitize the channel names when the data is loaded and, e.g., replace "." with "_"?

The data has been transferred to ml_training folder so someone with write/read access rights needs to this, and also the sanitation needs to take place also for all the v3 zarr files. It is not a challenge or anything per se to have a script to do this automatically, or something that an agent cannot do fast and clean, but I find it time consuming, even only the effort on pinging people just for a ".".

Also to be fair I do not really find self explained having 2.5 as 2_5. What do you think?

@clessig

clessig commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

I don't have a strong opinion but wouldn't it be best to sanitize the channel names when the data is loaded and, e.g., replace "." with "_"?

The data has been transferred to ml_training folder so someone with write/read access rights needs to this, and also the sanitation needs to take place also for all the v3 zarr files. It is not a challenge or anything per se to have a script to do this automatically, or something that an agent cannot do fast and clean, but I find it time consuming, even only the effort on pinging people just for a ".".

Also to be fair I do not really find self explained having 2.5 as 2_5. What do you think?

No, I mean when we read in the training log files we should santize the column name to a valid format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval anything related to the model evaluation pipeline

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Update plot_training.py

3 participants