Skip to content

feat: enable multiple sources, render as a mosaic - #140

Open
hrodmn wants to merge 11 commits into
mainfrom
feat/mosaic
Open

feat: enable multiple sources, render as a mosaic#140
hrodmn wants to merge 11 commits into
mainfrom
feat/mosaic

Conversation

@hrodmn

@hrodmn hrodmn commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • We want to be able to support tile requests for datasets that are stored in multiple icechunk stores on distinct spatial grids. We do this type of thing for COGs using titiler.mosaic.MosaicTilerFactory
  • This changes the API in a subtle way so that the url parameter can be provided multiple times instead of just once e.g. url=s3://path/to/store1&url=s3://path/to/store2. There are no other API changes that I am aware of due to switching to the MosaicTilerFactory.
    • This kind of mirrors the API described in the Collections Selections section of the OGC API - Tiles - Part 1: Core spec
image - Should we change this API to use `collections` instead of `url`? We don't need to decide now but I could see how these multidimensional datasets are much more like a collection than a single asset represented by a url.

Testing

  • Working locally on a sample of the NAQFC CONUS + AK Icechunk stores:
    • url=s3://naqfc/aqmv7/o3&url=s3://naqfc/aqmv7/o3_ak&variable=ozcon
  • Working on a fresh titiler-multidim deployment without a VPC and with no Redis cache

@maxrjones maxrjones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing this @hrodmn!

For a potential follow-up PR, it'd probably be worth caching the validation from src/titiler/multidim/mosaic.py:30-52 since that makes every tile fetch more expensive.

zarr.errors.GroupNotFoundError: status.HTTP_422_UNPROCESSABLE_ENTITY,
}
add_exception_handlers(app, error_codes)
add_exception_handlers(app, DEFAULT_STATUS_CODES)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
add_exception_handlers(app, DEFAULT_STATUS_CODES)
add_exception_handlers(app, MOSAIC_STATUS_CODES)

This will keep the error for out-of-coverage tile/point requests as a 404 (same as on main) rather than a 500


LoggingInstrumentor().instrument(set_logging_format=True)
HTTPXClientInstrumentor().instrument()
# HTTPXClientInstrumentor().instrument()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this just for dev purposes? should it get uncommented?

Comment thread pyproject.toml
"xarray>=2025.10.1",
"zarr>=3.2.0",
"titiler-mosaic==2.2.1",
"gribberish[zarr]>=1.7.0",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another motivation for shared codec implementations across formats/libraries. Needing gribberish for codecs isn't ideal, but a problem for another day.

Comment on lines +281 to +286
image, _ = src.part(
src.bounds,
pixel_selection=pixel_selection,
threads=MOSAIC_THREADS,
)
values = image.array.compressed()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
image, _ = src.part(
src.bounds,
pixel_selection=pixel_selection,
threads=MOSAIC_THREADS,
)
values = image.array.compressed()
# Antimeridian-crossing mosaics report wrapped bounds
# (west > east), which part() cannot window; read each side
# of the dateline separately and pool the unmasked values.
west, south, east, north = src.bounds
values = np.concatenate(
[
src.part(
(interval_west, south, interval_east, north),
pixel_selection=pixel_selection,
threads=MOSAIC_THREADS,
)[0].array.compressed()
for interval_west, interval_east in src._longitude_intervals(
west, east
)
]

There are definitely other solutions, but this is one option for handling input source that cross the antimeridian. Currently this is a regression from main.

Comment thread tests/test_mosaic.py
)
assert (
app.get("/variables", params=[("url", str(left)), ("url", str(mismatched))])
).status_code == 400

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
).status_code == 400
).status_code == 400
def test_histogram_supports_antimeridian_sources(app, tmp_path):
"""Histogram pools values from both sides of the antimeridian."""
west = tmp_path / "am-west.nc"
east = tmp_path / "am-east.nc"
write_dataset(west, 1, x=(172.5, 177.5))
write_dataset(east, 2, x=(-177.5, -172.5))
response = app.get(
"/histogram",
params=[("url", str(west)), ("url", str(east)), ("variable", "data")],
)
assert response.status_code == 200
histogram = response.json()
assert sum(bucket["value"] for bucket in histogram) > 0
assert histogram[0]["bucket"][0] == 1.0
assert histogram[-1]["bucket"][1] == 2.0
projected = tmp_path / "am-projected.nc"
write_antimeridian_dataset(projected)
response = app.get(
"/histogram", params=[("url", str(projected)), ("variable", "data")]
)
assert response.status_code == 200
histogram = response.json()
assert sum(bucket["value"] for bucket in histogram) > 0
assert histogram[0]["bucket"][0] == 0.5
assert histogram[-1]["bucket"][1] == 1.5

adds a test for inputs on different side of the antimeridian

reader_params=Depends(self.reader_dependency),
show_times: Annotated[
bool | None,
Query(description="Show info about the time dimension"),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Query(description="Show info about the time dimension"),
Query(description="Show info about the time dimension (only available for single URLs"),

@@ -15,16 +16,22 @@
)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from titiler.mosaic.errors import MOSAIC_STATUS_CODES

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable mosaic visualization of multidimensional sources

2 participants