Skip to content

Fix MWE bugs across genotype preprocessing, enrichment, EMS, TWAS/colocalization, and molecular phenotype notebooks #1387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gaow merged 12 commits into from
Jul 8, 2026
Merged

Fix MWE bugs across genotype preprocessing, enrichment, EMS, TWAS/colocalization, and molecular phenotype notebooks #1387

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
35e93fd
fix(genotype-preprocessing): replace unused name placeholder with req…
Jul 8, 2026
a8bbd14
fix(enrichment): correct eoo_enrichment CLI params and sldsc postproc…
Jul 8, 2026
5db3145
fix(xqtl_modifier_score): fix config paths, silent-failure guard, and…
Jul 8, 2026
5b13245
fix(reference_data): remove trailing dot in rss_ld_sketch cohort-id; …
Jul 8, 2026
f7c846a
fix(mnm-and-pecotmr): parameter/API fixes across TWAS and colocalizat…
Jul 8, 2026
b5b01d5
fix(molecular-phenotypes): support gene-ID peaks without positional i…
Jul 8, 2026
51c9a11
fix(mash,graveyard): toy-data-compatible mash_input construction; pec…
Jul 8, 2026
ba6a6e4
fix(genotype-preprocessing): revert to single required --name parameter
Jul 8, 2026
4007740
revert: restore graveyard notebooks (MRAID_QTL, fastenloc_dap, polyfu…
Jul 8, 2026
d3f9a61
fix(mash-vignette): use correct pre-built 34-condition prior file ins…
Jul 8, 2026
11f7d6e
fix(mash-posterior-contrast): correct manifest-based invocation and a…
Jul 8, 2026
65bde1c
fix(apa-calling): remove hardcoded DaPars2 script path and fix GTF/wi…
Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 21 additions & 21 deletions code/SoS/association_scan/TensorQTL/TensorQTL.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
"data": {
"text/html": [
"<table class=\"dataframe\">\n",
"<caption>A data.table: 1 × 2</caption>\n",
"<caption>A data.table: 1 \u00d7 2</caption>\n",
"<thead>\n",
"\t<tr><th scope=col>#id</th><th scope=col>#dir</th></tr>\n",
"\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
Expand All @@ -67,7 +67,7 @@
"</table>\n"
],
"text/latex": [
"A data.table: 1 × 2\n",
"A data.table: 1 \u00d7 2\n",
"\\begin{tabular}{ll}\n",
" \\#id & \\#dir\\\\\n",
" <int> & <chr>\\\\\n",
Expand All @@ -77,7 +77,7 @@
],
"text/markdown": [
"\n",
"A data.table: 1 × 2\n",
"A data.table: 1 \u00d7 2\n",
"\n",
"| #id &lt;int&gt; | #dir &lt;chr&gt; |\n",
"|---|---|\n",
Expand All @@ -96,7 +96,7 @@
"data": {
"text/html": [
"<table class=\"dataframe\">\n",
"<caption>A data.table: 5 × 8</caption>\n",
"<caption>A data.table: 5 \u00d7 8</caption>\n",
"<thead>\n",
"\t<tr><th scope=col>#chr</th><th scope=col>start</th><th scope=col>end</th><th scope=col>ID</th><th scope=col>SAMPLE_001</th><th scope=col>SAMPLE_002</th><th scope=col>SAMPLE_003</th><th scope=col>SAMPLE_004</th></tr>\n",
"\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
Expand All @@ -111,7 +111,7 @@
"</table>\n"
],
"text/latex": [
"A data.table: 5 × 8\n",
"A data.table: 5 \u00d7 8\n",
"\\begin{tabular}{llllllll}\n",
" \\#chr & start & end & ID & SAMPLE\\_001 & SAMPLE\\_002 & SAMPLE\\_003 & SAMPLE\\_004\\\\\n",
" <chr> & <int> & <int> & <chr> & <dbl> & <dbl> & <dbl> & <dbl>\\\\\n",
Expand All @@ -125,7 +125,7 @@
],
"text/markdown": [
"\n",
"A data.table: 5 × 8\n",
"A data.table: 5 \u00d7 8\n",
"\n",
"| #chr &lt;chr&gt; | start &lt;int&gt; | end &lt;int&gt; | ID &lt;chr&gt; | SAMPLE_001 &lt;dbl&gt; | SAMPLE_002 &lt;dbl&gt; | SAMPLE_003 &lt;dbl&gt; | SAMPLE_004 &lt;dbl&gt; |\n",
"|---|---|---|---|---|---|---|---|\n",
Expand Down Expand Up @@ -192,7 +192,7 @@
"data": {
"text/html": [
"<table class=\"dataframe\">\n",
"<caption>A data.table: 6 × 2</caption>\n",
"<caption>A data.table: 6 \u00d7 2</caption>\n",
"<thead>\n",
"\t<tr><th scope=col>#id</th><th scope=col>#path</th></tr>\n",
"\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
Expand All @@ -208,7 +208,7 @@
"</table>\n"
],
"text/latex": [
"A data.table: 6 × 2\n",
"A data.table: 6 \u00d7 2\n",
"\\begin{tabular}{ll}\n",
" \\#id & \\#path\\\\\n",
" <int> & <chr>\\\\\n",
Expand All @@ -223,7 +223,7 @@
],
"text/markdown": [
"\n",
"A data.table: 6 × 2\n",
"A data.table: 6 \u00d7 2\n",
"\n",
"| #id &lt;int&gt; | #path &lt;chr&gt; |\n",
"|---|---|\n",
Expand Down Expand Up @@ -271,7 +271,7 @@
"data": {
"text/html": [
"<table class=\"dataframe\">\n",
"<caption>A matrix: 5 × 5 of type int</caption>\n",
"<caption>A matrix: 5 \u00d7 5 of type int</caption>\n",
"<thead>\n",
"\t<tr><th></th><th scope=col>SAMPLE_001</th><th scope=col>SAMPLE_002</th><th scope=col>SAMPLE_003</th><th scope=col>SAMPLE_004</th><th scope=col>SAMPLE_005</th></tr>\n",
"</thead>\n",
Expand All @@ -285,7 +285,7 @@
"</table>\n"
],
"text/latex": [
"A matrix: 5 × 5 of type int\n",
"A matrix: 5 \u00d7 5 of type int\n",
"\\begin{tabular}{r|lllll}\n",
" & SAMPLE\\_001 & SAMPLE\\_002 & SAMPLE\\_003 & SAMPLE\\_004 & SAMPLE\\_005\\\\\n",
"\\hline\n",
Expand All @@ -298,7 +298,7 @@
],
"text/markdown": [
"\n",
"A matrix: 5 × 5 of type int\n",
"A matrix: 5 \u00d7 5 of type int\n",
"\n",
"| <!--/--> | SAMPLE_001 | SAMPLE_002 | SAMPLE_003 | SAMPLE_004 | SAMPLE_005 |\n",
"|---|---|---|---|---|---|\n",
Expand Down Expand Up @@ -356,7 +356,7 @@
"data": {
"text/html": [
"<table class=\"dataframe\">\n",
"<caption>A data.table: 28 × 5</caption>\n",
"<caption>A data.table: 28 \u00d7 5</caption>\n",
"<thead>\n",
"\t<tr><th scope=col>#id</th><th scope=col>SAMPLE_001</th><th scope=col>SAMPLE_002</th><th scope=col>SAMPLE_003</th><th scope=col>SAMPLE_004</th></tr>\n",
"\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
Expand Down Expand Up @@ -394,7 +394,7 @@
"</table>\n"
],
"text/latex": [
"A data.table: 28 × 5\n",
"A data.table: 28 \u00d7 5\n",
"\\begin{tabular}{lllll}\n",
" \\#id & SAMPLE\\_001 & SAMPLE\\_002 & SAMPLE\\_003 & SAMPLE\\_004\\\\\n",
" <chr> & <dbl> & <dbl> & <dbl> & <dbl>\\\\\n",
Expand Down Expand Up @@ -431,7 +431,7 @@
],
"text/markdown": [
"\n",
"A data.table: 28 × 5\n",
"A data.table: 28 \u00d7 5\n",
"\n",
"| #id &lt;chr&gt; | SAMPLE_001 &lt;dbl&gt; | SAMPLE_002 &lt;dbl&gt; | SAMPLE_003 &lt;dbl&gt; | SAMPLE_004 &lt;dbl&gt; |\n",
"|---|---|---|---|---|\n",
Expand Down Expand Up @@ -559,7 +559,7 @@
"data": {
"text/html": [
"<table class=\"dataframe\">\n",
"<caption>A data.table: 6 × 4</caption>\n",
"<caption>A data.table: 6 \u00d7 4</caption>\n",
"<thead>\n",
"\t<tr><th scope=col>#chr</th><th scope=col>start</th><th scope=col>end</th><th scope=col>gene_id</th></tr>\n",
"\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
Expand All @@ -575,7 +575,7 @@
"</table>\n"
],
"text/latex": [
"A data.table: 6 × 4\n",
"A data.table: 6 \u00d7 4\n",
"\\begin{tabular}{llll}\n",
" \\#chr & start & end & gene\\_id\\\\\n",
" <chr> & <int> & <int> & <chr>\\\\\n",
Expand All @@ -590,7 +590,7 @@
],
"text/markdown": [
"\n",
"A data.table: 6 × 4\n",
"A data.table: 6 \u00d7 4\n",
"\n",
"| #chr &lt;chr&gt; | start &lt;int&gt; | end &lt;int&gt; | gene_id &lt;chr&gt; |\n",
"|---|---|---|---|\n",
Expand Down Expand Up @@ -737,12 +737,12 @@
"\n",
" **Computational strategy designed for trans analysis:**\n",
" \n",
" Trans analysis faces significant memory challenges as we calculate all associations between all molecular traits × all genetic variants across the genome, creating a massive computational burden. To address this challenge, we implement a two-stage chromosome-based parallelization approach:\n",
" Trans analysis faces significant memory challenges as we calculate all associations between all molecular traits \u00d7 all genetic variants across the genome, creating a massive computational burden. To address this challenge, we implement a two-stage chromosome-based parallelization approach:\n",
"\n",
" **Stage 1 (trans_1): Chromosome-based parallelization**\n",
" - Phenotype data is processed per chromosome (e.g., 22 separate jobs for autosomes)\n",
" - For each phenotype chromosome, we test associations against variants from all 22 chromosomes\n",
" - This creates phenotype_chr × genotype_chr combinations (e.g., phenotype chr1 vs genotype chr1-22); Garbage was collected between each chromosome combination caculation to release memory\n",
" - This creates phenotype_chr \u00d7 genotype_chr combinations (e.g., phenotype chr1 vs genotype chr1-22); Garbage was collected between each chromosome combination caculation to release memory\n",
" - Results are combined across all chromosome combinations and saved as compressed files\n",
"\n",
" **Stage 2 (trans_2): Significance filtering**\n",
Expand Down Expand Up @@ -933,7 +933,7 @@
"# Optional pattern to filter covariates (list of covariate prefixes or exact names)\n",
"parameter: covariate_pattern = []\n",
"# Prefix for the analysis output\n",
"parameter: name = \"\"\n",
"parameter: name = str\n",
"# An optional subset of regions of molecular features to analyze. The last column is the gene names\n",
"parameter: region_list = path()\n",
"parameter: region_list_phenotype_column = 4\n",
Expand Down
8 changes: 6 additions & 2 deletions code/SoS/data_preprocessing/genotype/GWAS_QC.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@
"sos run pipeline/GWAS_QC.ipynb qc_no_prune \\\n",
" --cwd output/gwas_qc/plink \\\n",
" --genoFile output/genotype_formatting/plink/protocol_example.genotype.merged.bed \\\n",
" --name protocol_example \\\n",
" --geno-filter 0.1 \\\n",
" --mind-filter 0.1 \\\n",
" --hwe-filter 1e-08 \\\n",
Expand Down Expand Up @@ -187,7 +188,8 @@
"sos run pipeline/GWAS_QC.ipynb genotype_phenotype_sample_overlap \\\n",
" --cwd output/gwas_qc/genotype \\\n",
" --genoFile output/gwas_qc/plink/protocol_example.genotype.merged.plink_qc.fam \\\n",
" --phenoFile input/rnaseq/protocol_example.rnaseq.bed.gz\n"
" --phenoFile input/rnaseq/protocol_example.rnaseq.bed.gz \\\n",
" --name protocol_example\n"
]
},
{
Expand Down Expand Up @@ -262,6 +264,7 @@
"sos run pipeline/GWAS_QC.ipynb qc \\\n",
" --cwd output/gwas_qc/genotype \\\n",
" --genoFile output/gwas_qc/kinship/protocol_example.genotype.merged.plink_qc.protocol_example.king.unrelated.bed \\\n",
" --name protocol_example \\\n",
" --mac-filter 5\n"
]
},
Expand Down Expand Up @@ -296,6 +299,7 @@
"sos run pipeline/GWAS_QC.ipynb qc \\\n",
" --cwd output/gwas_qc/genotype \\\n",
" --genoFile output/gwas_qc/plink/protocol_example.genotype.merged.plink_qc.bed \\\n",
" --name protocol_example \\\n",
" --mac-filter 5\n"
]
},
Expand Down Expand Up @@ -362,7 +366,7 @@
"# the output directory for generated files\n",
"parameter: cwd = path(\"output\")\n",
"# A string to identify your analysis run\n",
"parameter: name = \"\"\n",
"parameter: name = str\n",
"# PLINK binary files (either BED/BIM/FAM or PGEN/PVAR/PSAM format)\n",
"parameter: genoFile = paths\n",
"# The path to the file that contains the list of samples to remove (format FID, IID)\n",
Expand Down
Loading
Loading