|
18 | 18 | "- Recognise the datashader contrast caveat.\n", |
19 | 19 | "\n", |
20 | 20 | "We use the real Xenium `cells` dataset from `squidpy` (its `transcripts` element and the\n", |
21 | | - "`morphology_focus` image), downloaded and cached on first use." |
| 21 | + "`morphology_focus` image), downloaded and cached on first use.\n", |
| 22 | + "\n", |
| 23 | + "> **Watch out:** on the datashader backend a `PercentileNormalize` does not clip to its percentiles —\n", |
| 24 | + "> the single most common datashader-contrast surprise, covered in section 5." |
22 | 25 | ] |
23 | 26 | }, |
24 | 27 | { |
|
35 | 38 | "id": "879b4d13", |
36 | 39 | "metadata": { |
37 | 40 | "execution": { |
38 | | - "iopub.execute_input": "2026-08-18T22:57:59.500527Z", |
39 | | - "iopub.status.busy": "2026-08-18T22:57:59.499253Z", |
40 | | - "iopub.status.idle": "2026-08-18T22:58:05.670740Z", |
41 | | - "shell.execute_reply": "2026-08-18T22:58:05.670194Z" |
| 41 | + "iopub.execute_input": "2026-08-19T03:36:01.655642Z", |
| 42 | + "iopub.status.busy": "2026-08-19T03:36:01.655518Z", |
| 43 | + "iopub.status.idle": "2026-08-19T03:36:07.639521Z", |
| 44 | + "shell.execute_reply": "2026-08-19T03:36:07.638799Z" |
42 | 45 | } |
43 | 46 | }, |
44 | 47 | "outputs": [ |
|
97 | 100 | "sdata" |
98 | 101 | ] |
99 | 102 | }, |
| 103 | + { |
| 104 | + "cell_type": "markdown", |
| 105 | + "id": "46f63a20", |
| 106 | + "metadata": {}, |
| 107 | + "source": [ |
| 108 | + "The `cells` dataset is a small (~3 MB) crop of a real 10x **Xenium** section (also used in the\n", |
| 109 | + "*Colouring points* example). The reductions below aggregate its `transcripts` points, coloured by the\n", |
| 110 | + "quality column `qv`:" |
| 111 | + ] |
| 112 | + }, |
| 113 | + { |
| 114 | + "cell_type": "code", |
| 115 | + "execution_count": 2, |
| 116 | + "id": "2ee8a709", |
| 117 | + "metadata": { |
| 118 | + "execution": { |
| 119 | + "iopub.execute_input": "2026-08-19T03:36:07.641246Z", |
| 120 | + "iopub.status.busy": "2026-08-19T03:36:07.641106Z", |
| 121 | + "iopub.status.idle": "2026-08-19T03:36:07.644100Z", |
| 122 | + "shell.execute_reply": "2026-08-19T03:36:07.643490Z" |
| 123 | + } |
| 124 | + }, |
| 125 | + "outputs": [ |
| 126 | + { |
| 127 | + "data": { |
| 128 | + "text/plain": [ |
| 129 | + "['x',\n", |
| 130 | + " 'y',\n", |
| 131 | + " 'z',\n", |
| 132 | + " 'feature_name',\n", |
| 133 | + " 'cell_id',\n", |
| 134 | + " 'fov_name',\n", |
| 135 | + " 'codeword_index',\n", |
| 136 | + " 'qv',\n", |
| 137 | + " 'transcript_id',\n", |
| 138 | + " 'is_gene',\n", |
| 139 | + " 'codeword_category',\n", |
| 140 | + " 'nucleus_distance',\n", |
| 141 | + " 'overlaps_nucleus']" |
| 142 | + ] |
| 143 | + }, |
| 144 | + "execution_count": 2, |
| 145 | + "metadata": {}, |
| 146 | + "output_type": "execute_result" |
| 147 | + } |
| 148 | + ], |
| 149 | + "source": [ |
| 150 | + "list(sdata[\"transcripts\"].columns)" |
| 151 | + ] |
| 152 | + }, |
100 | 153 | { |
101 | 154 | "cell_type": "markdown", |
102 | 155 | "id": "7a08616e", |
|
111 | 164 | }, |
112 | 165 | { |
113 | 166 | "cell_type": "code", |
114 | | - "execution_count": 2, |
| 167 | + "execution_count": 3, |
115 | 168 | "id": "16d596ee", |
116 | 169 | "metadata": { |
117 | 170 | "execution": { |
118 | | - "iopub.execute_input": "2026-08-18T22:58:05.672268Z", |
119 | | - "iopub.status.busy": "2026-08-18T22:58:05.672122Z", |
120 | | - "iopub.status.idle": "2026-08-18T22:58:08.082570Z", |
121 | | - "shell.execute_reply": "2026-08-18T22:58:08.081976Z" |
| 171 | + "iopub.execute_input": "2026-08-19T03:36:07.645952Z", |
| 172 | + "iopub.status.busy": "2026-08-19T03:36:07.645825Z", |
| 173 | + "iopub.status.idle": "2026-08-19T03:36:10.143881Z", |
| 174 | + "shell.execute_reply": "2026-08-19T03:36:10.143080Z" |
122 | 175 | } |
123 | 176 | }, |
124 | 177 | "outputs": [ |
|
148 | 201 | "\n", |
149 | 202 | "The reduction changes both the picture and the colorbar scale. `max` reports the single brightest value\n", |
150 | 203 | "in a pixel (bounded by the data max), `mean` averages, and `sum` accumulates — so its colorbar runs far\n", |
151 | | - "higher wherever transcripts pile up." |
| 204 | + "higher wherever transcripts pile up.\n", |
| 205 | + "\n", |
| 206 | + "As a rule of thumb: `sum` for abundance, `mean` for a typical value, `max` for the brightest signal,\n", |
| 207 | + "and `std` (below) for heterogeneity." |
152 | 208 | ] |
153 | 209 | }, |
154 | 210 | { |
155 | 211 | "cell_type": "code", |
156 | | - "execution_count": 3, |
| 212 | + "execution_count": 4, |
157 | 213 | "id": "b143411e", |
158 | 214 | "metadata": { |
159 | 215 | "execution": { |
160 | | - "iopub.execute_input": "2026-08-18T22:58:08.084333Z", |
161 | | - "iopub.status.busy": "2026-08-18T22:58:08.084206Z", |
162 | | - "iopub.status.idle": "2026-08-18T22:58:09.854305Z", |
163 | | - "shell.execute_reply": "2026-08-18T22:58:09.853820Z" |
| 216 | + "iopub.execute_input": "2026-08-19T03:36:10.145859Z", |
| 217 | + "iopub.status.busy": "2026-08-19T03:36:10.145727Z", |
| 218 | + "iopub.status.idle": "2026-08-19T03:36:11.982901Z", |
| 219 | + "shell.execute_reply": "2026-08-19T03:36:11.982377Z" |
164 | 220 | } |
165 | 221 | }, |
166 | 222 | "outputs": [ |
|
209 | 265 | "source": [ |
210 | 266 | "## 3. `count` and `std` — structure without a value column\n", |
211 | 267 | "\n", |
212 | | - "`count` ignores the colour value and shows how many primitives fall in each pixel (a density, like the\n", |
213 | | - "*Point density maps* notebook). `std` shows the spread of the values per pixel — useful for finding\n", |
| 268 | + "`count` ignores the colour value and shows how many primitives fall in each pixel (a density — equivalent to\n", |
| 269 | + "`density=True` with no `density_how`, as in the *Point density maps* notebook). `std` shows the spread of the values per pixel — useful for finding\n", |
214 | 270 | "heterogeneous regions." |
215 | 271 | ] |
216 | 272 | }, |
217 | 273 | { |
218 | 274 | "cell_type": "code", |
219 | | - "execution_count": 4, |
| 275 | + "execution_count": 5, |
220 | 276 | "id": "26a8767f", |
221 | 277 | "metadata": { |
222 | 278 | "execution": { |
223 | | - "iopub.execute_input": "2026-08-18T22:58:09.856298Z", |
224 | | - "iopub.status.busy": "2026-08-18T22:58:09.856173Z", |
225 | | - "iopub.status.idle": "2026-08-18T22:58:11.072022Z", |
226 | | - "shell.execute_reply": "2026-08-18T22:58:11.071259Z" |
| 279 | + "iopub.execute_input": "2026-08-19T03:36:11.984867Z", |
| 280 | + "iopub.status.busy": "2026-08-19T03:36:11.984761Z", |
| 281 | + "iopub.status.idle": "2026-08-19T03:36:13.208799Z", |
| 282 | + "shell.execute_reply": "2026-08-19T03:36:13.208157Z" |
227 | 283 | } |
228 | 284 | }, |
229 | 285 | "outputs": [ |
|
263 | 319 | "## 4. Datashader for large images — `method=\"datashader\"`\n", |
264 | 320 | "\n", |
265 | 321 | "Reductions also apply to images: `method=\"datashader\"` renders large or sparse rasters by aggregating\n", |
266 | | - "pixels, instead of handing the full array to matplotlib." |
| 322 | + "pixels, instead of handing the full array to matplotlib.\n", |
| 323 | + "\n", |
| 324 | + "The image path takes a slightly different reduction vocabulary — it adds `mode`/`first`/`last` and has\n", |
| 325 | + "no `count`/`any` — though here we leave `datashader_reduction` at its default." |
267 | 326 | ] |
268 | 327 | }, |
269 | 328 | { |
270 | 329 | "cell_type": "code", |
271 | | - "execution_count": 5, |
| 330 | + "execution_count": 6, |
272 | 331 | "id": "09894db6", |
273 | 332 | "metadata": { |
274 | 333 | "execution": { |
275 | | - "iopub.execute_input": "2026-08-18T22:58:11.073801Z", |
276 | | - "iopub.status.busy": "2026-08-18T22:58:11.073674Z", |
277 | | - "iopub.status.idle": "2026-08-18T22:58:11.449224Z", |
278 | | - "shell.execute_reply": "2026-08-18T22:58:11.448612Z" |
| 334 | + "iopub.execute_input": "2026-08-19T03:36:13.210422Z", |
| 335 | + "iopub.status.busy": "2026-08-19T03:36:13.210310Z", |
| 336 | + "iopub.status.idle": "2026-08-19T03:36:13.598089Z", |
| 337 | + "shell.execute_reply": "2026-08-19T03:36:13.597603Z" |
279 | 338 | } |
280 | 339 | }, |
281 | 340 | "outputs": [ |
|
311 | 370 | }, |
312 | 371 | { |
313 | 372 | "cell_type": "code", |
314 | | - "execution_count": 6, |
| 373 | + "execution_count": 7, |
315 | 374 | "id": "65c8ca02", |
316 | 375 | "metadata": { |
317 | 376 | "execution": { |
318 | | - "iopub.execute_input": "2026-08-18T22:58:11.450993Z", |
319 | | - "iopub.status.busy": "2026-08-18T22:58:11.450884Z", |
320 | | - "iopub.status.idle": "2026-08-18T22:58:11.776013Z", |
321 | | - "shell.execute_reply": "2026-08-18T22:58:11.775507Z" |
| 377 | + "iopub.execute_input": "2026-08-19T03:36:13.599890Z", |
| 378 | + "iopub.status.busy": "2026-08-19T03:36:13.599766Z", |
| 379 | + "iopub.status.idle": "2026-08-19T03:36:13.934807Z", |
| 380 | + "shell.execute_reply": "2026-08-19T03:36:13.934204Z" |
322 | 381 | } |
323 | 382 | }, |
324 | 383 | "outputs": [ |
|
350 | 409 | "source": [ |
351 | 410 | "## Summary\n", |
352 | 411 | "\n", |
| 412 | + "- **Contrast caveat first:** datashader contrast follows the aggregate range, not norm percentiles —\n", |
| 413 | + " use the matplotlib backend when you need percentile (or any non-`vmin`/`vmax`) contrast.\n", |
353 | 414 | "- On the datashader backend, a continuous colour needs a **reduction** to combine the primitives that\n", |
354 | 415 | " share a pixel; `datashader_reduction` sets it (default `max`).\n", |
355 | | - "- `max`/`mean`/`sum` change the picture and the colorbar scale; `count` gives a density and `std` the\n", |
356 | | - " per-pixel spread.\n", |
357 | | - "- `method=\"datashader\"` applies to images too, for large or sparse rasters.\n", |
358 | | - "- Datashader contrast follows the aggregate range, not norm percentiles — use the matplotlib backend\n", |
359 | | - " for percentile contrast." |
| 416 | + "- `max`/`mean`/`sum` change the picture and the colorbar scale; `count` gives a density (same as\n", |
| 417 | + " `density=True`) and `std` the per-pixel spread.\n", |
| 418 | + "- `method=\"datashader\"` applies to images too, for large or sparse rasters (with a slightly different\n", |
| 419 | + " reduction vocabulary)." |
360 | 420 | ] |
361 | 421 | }, |
362 | 422 | { |
|
369 | 429 | }, |
370 | 430 | { |
371 | 431 | "cell_type": "code", |
372 | | - "execution_count": 7, |
| 432 | + "execution_count": 8, |
373 | 433 | "id": "9a906bc9", |
374 | 434 | "metadata": { |
375 | 435 | "execution": { |
376 | | - "iopub.execute_input": "2026-08-18T22:58:11.777869Z", |
377 | | - "iopub.status.busy": "2026-08-18T22:58:11.777766Z", |
378 | | - "iopub.status.idle": "2026-08-18T22:58:11.815022Z", |
379 | | - "shell.execute_reply": "2026-08-18T22:58:11.814271Z" |
| 436 | + "iopub.execute_input": "2026-08-19T03:36:13.937167Z", |
| 437 | + "iopub.status.busy": "2026-08-19T03:36:13.937056Z", |
| 438 | + "iopub.status.idle": "2026-08-19T03:36:14.008592Z", |
| 439 | + "shell.execute_reply": "2026-08-19T03:36:14.007897Z" |
380 | 440 | } |
381 | 441 | }, |
382 | 442 | "outputs": [ |
|
0 commit comments