Skip to content

Improve plotting code - robustness, unification, remove code repetition#574

Open
keiran-rowell-unsw wants to merge 43 commits intonf-core:devfrom
Australian-Structural-Biology-Computing:improve_plotting
Open

Improve plotting code - robustness, unification, remove code repetition#574
keiran-rowell-unsw wants to merge 43 commits intonf-core:devfrom
Australian-Structural-Biology-Computing:improve_plotting

Conversation

@keiran-rowell-unsw
Copy link
Copy Markdown
Contributor

@keiran-rowell-unsw keiran-rowell-unsw commented Mar 31, 2026

Plotting refactor, drops duplicate files, ~300 lines of repetition beyond files, and reads more cleanly.
Implements #484, closes #485, and inches towards #373.
Will verify this implements #363

  • HTML report now drops sections where the metric isn't available ( e.g. ESMFold)
  • One generate_report.py, one report_template.html, all plotting logic in plot_utils.py; conceptually separates out the process constructing a report from plotting functions that use plain .tsvs
  • Uses a sort_structures_by_rank() utility in plot_utils.py to provide a place for program-specific ranking rather than use a matcher =~ with regex in the .nf pipeline.
    Each program sorting logic can be expanded upon in the util.
  • Injection of content in report_template.html now cleaner
  • Everything as plotly objects, no need to write-our and read-in .png files, users can save from the HTML report from plotly toolbar if they want to
  • post_processing.nf massively simplified due to removing program-specific exceptions
  • modeChannel() used consistently, cleaning many lines of adding in mode
  • ch_multiqc now cleared, was previously used to pass along plddt which is now simply extracted from structures with biopython, and makes way for a bulk folding report in Bulk MultiQC reporting as a pipeline-installed Python module #450.
  • Switches to unpkg.com as a more robust and up-to-date npm CDN. rawgit sunsetted in 2019

TODO:

  • Make a plotly sequence coverage object look as nice as the prior .png

TESTING:

  • In draft mode currently because I haven't run on all modes or validated comparison reports fully

PR checklist

  • This comment contains a description of changes (with reason).
  • CHANGELOG.md is updated.

@keiran-rowell-unsw keiran-rowell-unsw added this to the 2.1.0 milestone Mar 31, 2026
@keiran-rowell-unsw keiran-rowell-unsw self-assigned this Mar 31, 2026
@keiran-rowell-unsw keiran-rowell-unsw added the enhancement Improvement for existing functionality label Mar 31, 2026
@nf-core-bot
Copy link
Copy Markdown
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.5.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

Comment thread bin/plot_utils.py Outdated
Comment thread bin/plot_utils.py
z=pae,
colorscale="Greens_r",
zmin=0,
zmax=max_pae,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fixed (at 30) as it is on the main branch to give a consistent and interpretable color scale.

Copy link
Copy Markdown
Contributor Author

@keiran-rowell-unsw keiran-rowell-unsw Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I'll fix (at 31.75?). I also need to fix up the space given to the plots.
I'm also updating the sequence coverage plot.

The main thing I'm doing is clearing the pipeline logic, I'll switch out of draft when done testing

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks to be a nice clean-up! But I think it must forked from an old version because I noticed a few things like this that I have fixed in main which would get un-fixed here. It is already in draft but just flagging a few things that I noticed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep it was forked from a refactor from v1.1!

Thanks for spotting those ones, I was re-planting old code from refactor before v2.0 fixes, I'll get it more complete in a short while.

requested_modes_size
ch_report_input
ch_report_template
ch_comparison_template
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure of nf-core philosophy but my gut feel is that it is better to be explicit about dependencies rather than relying on globals within the subworkflow?

Comment thread modules/local/run_alphafold2/main.nf Outdated
Comment thread modules/local/run_alphafold3/main.nf Outdated
Comment thread bin/generate_report.py Outdated
@keiran-rowell-unsw keiran-rowell-unsw marked this pull request as ready for review April 2, 2026 04:57
@keiran-rowell-unsw
Copy link
Copy Markdown
Contributor Author

For reviewers: stylistic change

Since I'm using Plotly and directly injecting, the Sequence Coverage plot doesn't look the same as the matplotlib one. I tried to make a decent match a lock the axes, but open to aesthetic improvement

swappy-20260402-155845

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improvement for existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants