Skip to content

Add ncountr - #348

Open
princello wants to merge 5 commits into
scverse:mainfrom
princello:add-ncountr
Open

Add ncountr#348
princello wants to merge 5 commits into
scverse:mainfrom
princello:add-ncountr

Conversation

@princello

@princello princello commented Mar 24, 2026

Copy link
Copy Markdown

Add ncountr to the scverse ecosystem

ncountr is a Python package for end-to-end NanoString nCounter gene expression analysis. It provides RCC file parsing, quality control, normalization (positive-control, CodeSet content, and housekeeping), differential expression testing (per-gene Mann-Whitney U or t-test with FDR correction), gene set scoring, and GSEA.

AnnData integration

nCounter is a bulk, targeted-panel platform, so the parsing, platform-specific QC and normalization stages operate on ncountr's own NanostringExperiment container. AnnData is the interoperability layer: the module-level function ncountr.to_anndata(exp) exports the analysed experiment — normalized counts in .X, raw counts in .layers["raw"], sample metadata plus QC metrics and lane/cartridge information in .obs, and gene annotations including housekeeping flags in .var — so the result can be handed to scanpy and the rest of the scverse ecosystem without losing its sample or gene metadata.

The GSE275334 Long COVID vignette demonstrates that handoff end to end: PCA, principal components coloured by the QC and lane variables carried across in .obs, a group dendrogram, rank_genes_groups, and a matrixplot — plus a cross-check that re-running the same contrast in scanpy reproduces the ncountr.de() gene ranking (Spearman ρ = 0.999998, 19 of the top 20 genes shared).

Checklist

  • OSI-approved license: MIT
  • Versioned releases: published on PyPI
  • CI/CD: GitHub Actions (workflows)
  • API documentation: hosted on Read the Docs
  • AnnData integration: ncountr.to_anndata(exp) export

The full checklist from the repository README is filled in in this comment.

@muskanhashim

Copy link
Copy Markdown
Contributor

Thanks for the submission! I reviewed the code against our checklist and it's in great shape

We would like to flag, though, that the PR uses a shortened checklist, and it's missing the mandatory item

"I am an author or maintainer of the tool and agree on listing the package on the scverse website"

Could you paste the full checklist from the repo README and tick all mandatory items including this? That author-consent confirmation is required for us to list the package

Two more minor, non-blocking notes: the "automated tests" checklist item is also missing, though your tests clearly cover it; and the version field reads "v0.1.0" while PyPI lists it as "0.1.0". You may want to drop the "v"?

Other feedback (not related to the checklist): currently AnnData only comes in at the end via to_anndata(), while the analysis runs on the NanostringExperiment object. That's completely fine but it would just be nice if a tutorial showed how to take the exported AnnData onward into a scanpy workflow, so users can see how ncountr plugs into the broader scverse ecosystem!

Thanks again!

@princello

princello commented Aug 6, 2026

Copy link
Copy Markdown
Author

Thanks for the careful review, @muskanhashim — apologies for the shortened checklist. Here is the full one from the README with all mandatory items answered.

Also fixed in packages/ncountr/meta.yaml: the version field now reads 0.2.0 — I dropped the v as suggested, and bumped it to match the current PyPI release (0.2.0 was published after this PR was opened).

One correction to my own PR description above: I wrote NcountrExperiment.to_anndata(), which is wrong on both counts. The container class is NanostringExperiment, and to_anndata is a module-level function, so the actual call is ncountr.to_anndata(exp).


Checklist for adding packages

Mandatory

Name of the tool: ncountr

Short description: A Python package for end-to-end NanoString nCounter gene expression analysis — RCC file parsing, quality control, normalization (positive-control, CodeSet content, and housekeeping), differential expression testing (per-gene Mann-Whitney U or t-test with FDR correction), gene set scoring, and GSEA.

How does the package use scverse data structures: ncountr parses raw RCC files into a NanostringExperiment container, and exports the complete analysis state to AnnData via ncountr.to_anndata(exp): normalized counts in .X, raw counts in .layers["raw"], sample metadata plus QC metrics and lane/cartridge information in .obs, and gene annotations including housekeeping flags in .var. AnnData is the handoff point into scanpy and the rest of the scverse ecosystem for downstream analysis. NanoString nCounter is a bulk, targeted-panel platform (typically 50–800 genes), so the raw-parsing and platform-specific QC/normalization stages operate on the native container, and AnnData is used where it is the appropriate structure — as the interoperable representation of the analyzed experiment.

  • The code is publicly available under an OSI-approved license — MIT (LICENSE)
  • The package provides versioned releases — 0.1.0 and 0.2.0
  • The package can be installed from a standard registry (e.g. PyPI, conda-forge, bioconda) — PyPI: pip install ncountr (a conda-forge recipe is also in review at Add ncountr conda-forge/staged-recipes#32721)
  • Automated tests cover essential functions of the package and a reasonable range of inputs and conditions — 313 tests across 18 test modules in tests/, covering every public module (parsing, QC, normalization, DE, effect sizes, GSEA, pathway scoring, gene sets, plotting, AnnData export, GEO download, CLI)
  • Continuous integration (CI) automatically executes these tests on each push or pull request — GitHub Actions, on every push and PR to main, across Python 3.9–3.12
  • The package provides API documentation via a website or README — full API reference on Read the Docs
  • The package uses scverse datastructures where appropriate (i.e. AnnData, MuData or SpatialData and their modality-specific extensions) — AnnData, via ncountr.to_anndata(); see the description above
  • I am an author or maintainer of the tool and agree on listing the package on the scverse website

Recommended

  • Please announce this package on scverse communication channels (zulip, discourse, twitter)

  • Please tag the author(s) these announcements. Handles (e.g. @scverse_team) to include are:

    • GitHub: @princello (Zicheng Wang, Columbia University Irving Medical Center)
    • Zulip / Discourse / Mastodon / Bluesky / Twitter: not currently set up — GitHub is the best handle to use.
  • The package provides tutorials (or "vignettes") that help getting users started quickly — five vignettes on real public GEO datasets spanning 1,458 samples and five different nCounter panels (Long COVID / ME-CFS, HCC immunotherapy response, autoimmune retinopathy, interferon-stimulated macrophages, and a 1,321-sample PAM50 breast cancer set)

  • The package uses the scverse cookiecutter template — not used; the project structure predates the submission, though it follows the same src/ layout, pytest, and Sphinx/RTD conventions.


On the scanpy tutorial suggestion — that is a fair point, and you were right that it stopped at the export. This has since been implemented; see the follow-up comment below.

Thanks again for the review.

(Edited: corrected the differential-expression description — ncountr.de() uses a per-gene Mann-Whitney U or t-test with FDR correction, not a negative-binomial GLM — and refreshed the test count.)

princello added a commit to princello/ncountr that referenced this pull request Aug 7, 2026
* Add scanpy downstream section to GSE275334 vignette; fix broken plotting calls

Adds a section 9 to the Long COVID vignette carrying the exported AnnData into a
real scanpy workflow (PCA, QC-coloured components, group dendrogram,
rank_genes_groups, matrixplot), including a cross-check that re-running the
Long COVID vs Healthy contrast through scanpy reproduces the ncountr.de()
ranking (Spearman rho 0.999998, 19/20 top genes shared). Addresses reviewer
feedback on scverse/ecosystem-packages#348.

Also fixes API misuse that made four of the five vignettes fail on execution:

- plot_pathway_scores() was called with a DataFrame plus score_column=/
  group_column=, which it has never accepted. It takes a pd.Series of scores
  and a {group_name: [sample_ids]} dict.
- plot_heatmap() was called with the experiment plus genes=/samples=/z_score=.
  It takes a genes x samples DataFrame and the keyword is zscore.

All five vignettes now execute end to end under nbconvert.

* Harden vignette validation, expose tutorials in docs, sync ecosystem metadata

Follows up the scanpy section with the regression protection and documentation
work needed to make the five advertised tutorials trustworthy.

Scanpy section revisions:
- the canonical exported `adata` is no longer mutated; the scanpy walkthrough
  operates on an explicit `adata_sc` copy
- adds a cell showing the sample/QC/lane annotations carried across by the export
- states that no library-size renormalization is applied, since .X already holds
  positive-control + housekeeping normalized counts
- the UMAP/Leiden omission is now justified purely methodologically (46 bulk
  profiles on a targeted panel), with no claim about behaviour not exercised here

Regression prevention:
- tests/test_vignette_api.py statically checks every ncountr call in every
  vignette against the live signature, plus notebook hygiene (no committed
  outputs, execution counts or tracebacks). Runs in the existing fast CI.
  Verified against the pre-fix notebooks: it reports all five real mismatches.
- .github/workflows/vignettes.yml executes all five notebooks weekly and on
  demand on Python 3.11, one matrix job each, into a temp dir so sources are
  never modified.

Documentation:
- docs/vignettes/index.rst now lists all five tutorials with dataset, context,
  sample count, panel and links; previously an empty toctree, which is what the
  scverse tutorials_home URL points at
- api.rst/conf.py: drop the duplicate NanostringExperiment registration and
  render docstring Attributes as :ivar:. Sphinx -W warnings 28 -> 1 (the
  remainder is a third-party sphinx_autodoc_typehints deprecation)

Ecosystem metadata:
- scverse-submission/meta.yaml version v0.1.0 -> 0.2.0, matching the released
  package and the submitted PR
- conda-recipe/meta.yaml to 0.2.0 with the verified sdist sha256, drops the
  unused setuptools-scm host dep, adopts python_min, adds pip check

No changes under src/; package version stays 0.2.0.

* Migrate conda recipe to the CEP 13 v1 recipe.yaml format

conda-forge/staged-recipes now expects recipe.yaml: every recipe merged there
recently uses it, and the conda-build meta.yaml example is checked in as
'example-v0-deprecated'. Converts the mirrored recipe so it matches what is
actually submitted, and documents how to refresh version + sha256 together.
@muskanhashim

Copy link
Copy Markdown
Contributor

Great, @princello! Everything from the checklist side is resolved, thanks for the thorough follow-up and the Scanpy vignette!

In our attempt for users to be able to discover scverse ecosystem packages more easily, there is unfortunately a small new requirement on our side: The registry schema was just tightened (#392) after your PR was opened, so the metadata validation (mkjson + pre-commit) now fails. Two quick fixes needed in packages/ncountr/meta.yaml to match the new schema:

  1. tags must now come from the controlled vocabulary (the free-form ones no longer validate). For ncountr I'd recommend something like:
    tags:
    - bulk RNA-seq
    - quality control
    - differential expression
    - functional analysis
  2. two new required fields:
    primary_category: bulk RNA-seq
    language: Python

Please feel free to pick whichever tags/primary_category best fit from the allowed lists in the schema. Once that's in and the checks go green, this is good to merge. Thanks again for bearing with the moving target!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants