Skip to content

Fix downloaded file names keeping the URL query string - #1340

Open
scks1 wants to merge 1 commit into
CLIMADA-project:mainfrom
scks1:fix/download-filename-query-string
Open

scks1 wants to merge 1 commit into
CLIMADA-project:mainfrom
scks1:fix/download-filename-query-string

Conversation

@scks1

@scks1 scks1 commented Oct 29, 2026 •

Copy link
Copy Markdown

Changes proposed in this PR:

  • Derive the local file name from the URL path only (PurePosixPath(urlsplit(url).path).name) in download_file (climada/util/files_handler.py) and Client._download (climada/util/api_client.py), so a query string or fragment no longer becomes part of the downloaded file name.

This PR fixes #1336

Summary

Fixes #1336. download_file (climada/util/files_handler.py) and Client._download (climada/util/api_client.py) derived the local file name with url.split("/")[-1], so a query string or fragment became part of the file name — e.g. the distance-to-coast dataset downloaded as GMT_intermediate_coast_distance_01d.zip?download=1, which is an invalid file name on Windows and made Centroids.get_dist_coast raise OSError: [Errno 22].

Both sites now derive the name from the URL path only:

PurePosixPath(urlsplit(url).path).name

Fix layer

The two sites do not share a helper today, and their post-processing differs, so each site derives the name locally with the same expression (the shape proposed in #1336):

  • files_handler.download_file never percent-decoded the name and still does not; it only stops treating the query/fragment as part of it.
  • Client._download keeps its existing unquote(...) of the derived name, so percent-encoded names decode exactly as before.

Behavior preserved / changed, explicitly:

  • Plain URLs: unchanged names at both sites.
  • Percent-encoded names: unchanged (decoded in api_client, left encoded in files_handler, as before).
  • files_handler: a URL whose path ends in /, or has no file name at all, still raises the documented ValueError ("cannot download … as a file") — the previous string split rejected directory URLs the same way, and this is preserved even when a query string is present.
  • URLs whose final path segment was previously polluted by ?query or #fragment now yield that segment — that is the bug fix.

One related spot was left alone on purpose: download_ftp in files_handler.py uses url.split("/")[-1] only as a tqdm progress-bar description, not as a file name.

Tests

New unit tests with mocked requests.get (no network, no data downloads):

  • climada/util/test/test_files.py — new TestDownloadFileName class: plain URL regression, query string (the issue's coast-distance file name), fragment, and ValueError for file-name-less URLs (directory + query, trailing slash, host-only).
  • climada/test/test_api_client.py — TestClient.test_download_file_name_from_url: plain URL regression, query string, fragment, and percent-encoded name still decoded.

Test-execution note, stated plainly: my local sandbox's package-index access degraded badly while preparing this (repeated proxy connection resets), so I could not finish installing CLIMADA's full dependency set to run the two modules under pytest yet. What I did run, against the real patched code: the new files_handler logic end-to-end with mocked requests.get (all cases above, 11/11 checks pass, including the issue's exact file name) and the api_client derivation expression verbatim. The mocked pytest modules are written to run in CI; if CI flags anything in them I will fix it immediately.

Disclosure

Disclosure: this contribution was written by Árbol, an AI assistant working under the direction of the account holder, who set the acceptance criteria and verified the results before submission.

PR Author Checklist

PR Reviewer Checklist

download_file (util/files_handler.py) and Client._download
(util/api_client.py) derived the local file name with
url.split("/")[-1], so a query string or fragment became part of the
file name. On Windows the resulting name (e.g.
"GMT_intermediate_coast_distance_01d.zip?download=1" for the
distance-to-coast dataset) is invalid and Centroids.get_dist_coast
raised OSError: [Errno 22].

Both sites now derive the name from the URL path only, via
PurePosixPath(urlsplit(url).path).name. Client._download keeps its
percent-decoding (unquote) of the file name; files_handler keeps not
decoding it, as before. In files_handler, a URL whose path ends in
"/" (or has no file name at all) still raises the documented
ValueError, as the previous string split did for directory URLs.

Adds unit tests with mocked requests for both sites: plain URL
(regression), query string, fragment, percent-encoded name
(api_client), and file-name-less URLs (files_handler).

Fixes CLIMADA-project#1336
@scks1

scks1 commented Oct 30, 2026

Copy link
Copy Markdown
Author

A note on the failing Read the Docs check, since the failure is not in this change: in the build log for this PR (https://app.readthedocs.org/projects/climada-python/builds/35502568/), every step through environment creation and pip install .[doc] succeeds. The build then fails in the Sphinx step itself, inside Read the Docs' own extension:

sphinx.errors.ExtensionError: Handler <function update_body ...> for event 'html-page-context' threw an exception (exception: 'StandaloneHTMLBuilder' object has no attribute 'css_files')

raised from readthedocs_ext/readthedocs.py under Sphinx 9.0.4 — an incompatibility between the RTD extension and the new Sphinx major version. This PR changes only climada/util/api_client.py, climada/util/files_handler.py, and tests; it touches no documentation files or docs configuration. (The log also shows docstring warnings in files this PR does not touch, e.g. climada/entity/exposures/base.py and climada/util/coordinates.py, but warnings are not what failed the build.)

So this check should go green on a rebuild once the RTD extension / Sphinx 9 combination is resolved on the docs side; there is nothing in this change set that can address it.

Disclosure: this comment was written by Árbol, an AI assistant working under the direction of the account holder, who reviewed it before posting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Downloaded file names keep the URL query string, invalid on Windows

1 participant