Modernize build#407
Closed
efredriksson wants to merge 16 commits into
Closed
Conversation
First step of the meson-python migration. Adds pyproject.toml with meson-python as build-backend, plus a minimal meson.build that installs the pure-Python tree (src/pyfmi -> pyfmi, src/common -> pyfmi/common). Cython extensions are not yet declared in meson; the legacy setup.py continues to build them when invoked directly. This is the C1 checkpoint per the modernization plan -- pip-install via mesonpy at this commit yields a wheel without compiled extensions on purpose. Build requires include assimulo-testing>=3.9.0b1 so build isolation resolves Assimulo's .pxd files in later commits. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Flattens the source layout so the entire installable package lives
under src/pyfmi/. Removes the pyfmi.common package_dir mapping from
setup.py and the second install_subdir call from meson.build -- a
single install_subdir('src/pyfmi') now produces the same on-disk
layout as before. Import paths (pyfmi.common.*) are unchanged.
Mirrors the simplification done in the Assimulo modernization fork
(git mv src assimulo).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires the 15 Cython extension modules into meson.build: - 14 plain modules (fmi_base, fmi, fmi1, fmi2, fmi3, fmi_util, fmi_extended, fmi_coupled, util, test_util, plus the four assimulo_interface variants in src/pyfmi/simulation/). - master.pyx with a WITH_OPENMP cython compile-time env and optional -fopenmp link/compile args, driven by the new -Dopenmp meson option. FMIL is located via the new -Dfmil_prefix meson option, falling back to the FMIL_HOME environment variable for parity with setup.py. The lib name is configurable via -Dfmil_name (default fmilib_shared) for the static-link case. NumPy include is resolved at configure time via `numpy.get_include()`, mirroring Assimulo's pattern. Likewise the assimulo package parent dir is added to Cython's include path so `cimport assimulo.problem` resolves at build time (assimulo-testing is in build-system.requires so it is present during build isolation). NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION is set globally via common_c_args, replacing the per-.pyx `# distutils:` headers functionally (the per-file headers remain harmless since meson ignores them). install_rpath/build_rpath point at the FMIL libdir for non-wheel Linux installs. Wheel-time bundling (with $ORIGIN rpath) lands in C4. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds '$ORIGIN' to install_rpath alongside the configured FMIL libdir. Source installs continue to resolve libfmilib_shared.so via the libdir entry. Wheel installs have FMIL repaired into the package by auditwheel (Linux) and delvewheel (Windows); '$ORIGIN' lets ld.so find the bundled copy next to the extension. Setup.py's manual shutil.copy2 of FMIL into src/pyfmi/ is therefore no longer needed -- the wheel repair tools handle bundling on both platforms. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drops the svnversion+version.txt write path from setup.py and the
file-read fallback in pyfmi/__init__.py. The version is now resolved
via importlib.metadata.version("PyFMI") (or "pyfmi-testing" during the
beta-name window), matching the source of truth in pyproject.toml.
Also adds an install_data step to meson.build so LICENSE and CHANGELOG
land inside the installed pyfmi/ package -- previously handled by
shutil.copy2 + package_data in setup.py.
__revision__ is kept as a constant "unknown" for backward compat with
any user code that imports it; PyFMI moved off subversion long ago so
the value was already practically constant.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
End of Phase 1. The meson-python backend is now the sole build entry point; pyproject.toml declares everything previously expressed in setup.cfg (dependencies, python_requires) and the Cython extension graph lives entirely in meson.build. MANIFEST.in is removed too -- meson-python's sdist builder uses git ls-files, making the manifest redundant. Build options previously surfaced as setup.py CLI flags are now meson options (-Dfmil_prefix, -Dfmil_name, -Dopenmp) or built-in meson features (-Dbuildtype=debug for the old --debug flag). The more obscure flags (--force-32bit, --copy-libgcc, --no-msvcr, --extra-c-flags) are dropped: 32-bit targets are out of scope for the modernization, MinGW DLL bundling is handled by delvewheel in CI, and ad-hoc CFLAGS can be passed via meson's standard CFLAGS env or -Dc_args. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Removes the source builds of SuperLU, SUNDIALS, and Assimulo from the dev image: the assimulo-testing wheel pulled in via pip build isolation already embeds SuperLU + SUNDIALS statically and dynamically links libopenblas only. The system therefore needs just libopenblas and gfortran at runtime. Drops the pre-install of Cython/numpy/scipy/matplotlib too -- pip build isolation resolves the build-time numpy/Cython from pyproject.toml, and the wheel install pulls in runtime numpy / scipy / matplotlib / assimulo-testing automatically. Keeps the FMI Library 3.0.4 build into /usr (still a system dep, detected by meson via -Dfmil_prefix=/usr). The venv is created without --system-site-packages now that runtime deps come from pip; this keeps the image self-contained and matches how CI-built wheels will resolve dependencies on end-user systems. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the old 'python setup.py build_ext install --fmil-home=/usr' incantation with a pip install that passes the FMIL prefix to meson via setup-args. Pip resolves build-system.requires from pyproject.toml in isolation, then invokes meson-python with the meson option set. Also drops the SETUPTOOLS_JFLAG variable (parallel build threading is now handled by ninja, which uses all cores by default) and switches .venv creation away from --system-site-packages (the image no longer preinstalls numpy/scipy/matplotlib system-wide). Test target gains an explicit tests/ path so we exercise the suite at its canonical location. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Adds workflow_dispatch trigger so the smoke workflow can be rerun manually. - Adds a concurrency group keyed on workflow+ref so superseded runs cancel themselves; PR pushes no longer queue stale builds. - Renames the workflow + job for clarity; keeps the canonical Py3.11 Docker smoke as the single linux job. - Adds a comment pointing readers at wheels.yml + cibuildwheel for the multi-Python matrix (cp311/312/313) per the modernization plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Targets cp311/312/313 on Linux and Windows; skips 32-bit and musllinux.
Linux uses a pre-built pyfmi-manylinux Docker image (created from
Dockerfile.manylinux in the next commit) that has FMIL 3.0.4 installed
in /usr, so each wheel build can find it via -Dfmil_prefix=/usr.
Windows expects FMIL to be built into C:/deps by the GHA workflow
(tools/wheels/build_dependencies_windows.sh) before cibuildwheel runs.
Wheels are repaired with auditwheel on Linux and the project-local
delvewheel wrapper script (tools/wheels/repair_windows.ps1) on
Windows.
Each built wheel is exercised by 'pytest {project}/tests' so a wheel
that imports but fails at runtime is caught in CI rather than after
publish.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a build-only docker image that compiles FMI Library 3.0.4 into /usr on top of the quay.io/pypa/manylinux_2_28_x86_64 base. cibuildwheel references this image via manylinux-x86_64-image = "pyfmi-manylinux" in pyproject.toml, so each cp311/312/313 wheel build can locate FMIL through -Dfmil_prefix=/usr (already configured in C11). tools/wheels/build_dependencies.sh is the single source of truth for the FMIL build; the manylinux image bakes it in at image-build time. Mirrors the assimulo-manylinux pattern but drops SuperLU/SUNDIALS, which the assimulo-testing wheel already embeds statically. Adds the build-manylinux-image Makefile target so it can be rebuilt locally with the same incantation CI uses. Verified locally: 'make build-manylinux-image' produces FMIL 3.0.4 at /usr/lib64/libfmilib_shared.so, and 'cibuildwheel --only cp311-manylinux_x86_64' against this image yields a 6.9 MB manylinux wheel that runs all 935 tests successfully (15 skipped, 0 failed). The repaired wheel bundles libfmilib_shared into pyfmi.libs/ via auditwheel. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
build_dependencies_windows.sh compiles FMI Library 3.0.4 into C:/deps using the MSYS2 UCRT64 toolchain (gcc + ninja + cmake). The wheels.yml workflow (C14) runs this script in an msys2 shell before invoking cibuildwheel, so the windows wheel build can find FMIL via -Dfmil_prefix=C:/deps (already configured in pyproject.toml). repair_windows.ps1 wraps delvewheel to bundle fmilib_shared.dll plus the MSYS2 UCRT64 runtime DLLs into each wheel; cibuildwheel invokes it via repair-wheel-command. Mirrors the assimulo-testing delvewheel wrapper pattern. Untested locally (no Windows host available); validation lands when the wheels.yml workflow is dispatched against a windows-2022 runner. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Builds cp311/312/313 wheels on Linux (manylinux_2_28_x86_64 via
pre-built pyfmi-manylinux image) and Windows (MSYS2 UCRT64 toolchain).
Each platform invokes cibuildwheel with the configuration in
pyproject.toml (C11), which runs the test suite against the
installed wheel before signing off.
Triggers:
- push / pull_request: build wheels on every change to catch
regressions before merge; no publish.
- workflow_dispatch with publish=true on a v* tag: download
artifacts, upload to PyPI via OIDC Trusted Publisher.
PyPI publishing uses the standard pypa/gh-action-pypi-publish action,
configured for the PyFMI project. The beta-rename step
(pyfmi -> pyfmi-testing) lands in C15 as a separate workflow input
so reviewers can opt into it without rewriting this baseline.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a 'test_release' workflow_dispatch input that gates a sed-based rewrite of the project name from 'PyFMI' to 'pyfmi-testing' in both pyproject.toml and meson.build. Runs only when publishing (publish=true && test_release=true), mirroring the assimulo-testing pattern. This lets us publish a parallel-installable beta to PyPI under 'pyfmi-testing' while the upstream PyFMI release stream is unaffected. The sed step is intended to be removed once the modernization branch is merged into modelon-community/PyFMI and released under the canonical name. Verified sed patterns locally against the current pyproject.toml line 13 and meson.build line 2. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
distutils was removed in Python 3.12 (PEP 632), so the wheel build's
post-install test step failed during collection of test_log.py when
running on cp312/cp313:
pyfmi/common/log/parser.py:24: ModuleNotFoundError: No module named 'distutils'
from distutils.util import strtobool
The only call site (parse_value) is guarded by a regex that limits
input to the literals True/true/False/false, so strtobool's broader
truthy/falsy mapping is overkill. Inline a direct string compare
instead.
Verified end-to-end: 'cibuildwheel --only cp312-manylinux_x86_64'
produces a wheel and its post-install pytest run completes with 937
passed / 13 skipped, same as the cp311 Docker smoke build.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors the workflow used in Assimulo so contributors get a reproducible dev venv regardless of when they clone. .venv is rebuilt whenever the lockfile changes; make compile-deps regenerates the lockfile from pyproject.toml.
efredriksson
force-pushed
the
dev-ef-modernize-build
branch
from
2c11b1f to
722591b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.