WIP: Add support for Windows

[zivy]

Add support for building and testing on Windows OS in a consistent manner to Linux and macOS.
Repository URL and tag are specified in a shared
configuration file (SITK_CONFIG) which is used
by both the configure and configure.win scripts.
Testing is performed in CircleCI as it offers a
more powerful build machine under the free plan.

[Artur-man]

trying now with my local win, but it seems MAKEJ and ADDITIONAL_SITK_MODULES are not being passed to cmake at all.

[Artur-man]

as a side note, configure.vars does not apply to windows ... thus these variables have to be set as environmental variables before installation
https://support.posit.co/hc/en-us/articles/200486518-Customizing-Package-Build-Options-in-the-RStudio-IDE

Sys.setenv("ADDITIONAL_SITK_MODULES"="-DSIMPLEITK_USE_ELASTIX=ON")
Sys.setenv("MAKEJ"="6")
remotes::install_github("SimpleITK/SimpleITKRInstaller")

[Artur-man]

i think the issue is with Rtools not seeing the make, there seems to be a quick fix for that, which i check before installing the package.
https://github.com/BIMSBbioinfo/SimpleITKRInstaller/tree/release

UPDATE: getting close, it seems working with powershell instead of bash is solving the problem. An issue could be to write a separate workflow for windows, I will try unfying mac and win runs afterwards.

[Artur-man]

It is working!, since configure.win is (because we set as so) working on bash, I moved setting RTOOLS_HOME to configure.win directly. Everything else is in powershell.

[Artur-man]

It seems windows require a separate cmake setup because i am getting these compilation errors:

[ 59%] Building C object Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/csa.c.obj
[ 59%] Building C object Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3.c.obj
[ 59%] Building C object Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3_io.c.obj
C:/bld/Build/ITK/Modules/ThirdParty/GDCM/src/gdcm/Utilities/gdcmext/mec_mr3_io.c: In function 'aligned_alloc_impl':
C:/bld/Build/ITK/Modules/ThirdParty/GDCM/src/gdcm/Utilities/gdcmext/mec_mr3_io.c:146:15: error: implicit declaration of function 'posix_memalign' [-Wimplicit-function-declaration]
  146 |   int error = posix_memalign(&allocPtr, alignment, size);
      |               ^~~~~~~~~~~~~~
make[6]: *** [Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/build.make:1625: Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3_io.c.obj] Error 1

UPDATE: looked over the web but couldn't find any specific solution, caused by gdcm ? Using -DITK_USE_SYSTEM_ZLIB:BOOL=ON \ skips one of the compilations that invokes posix but then the same error occurs somewhere else still.

[Artur-man]

@zivy know any fix to this posix_memalign error ?

[blowekamp]

The following check looks suspicious for a mingw compiler:
https://github.com/InsightSoftwareConsortium/ITK/blob/5be898bdeadd8c7d2de6daa9962489ed391782e6/Modules/ThirdParty/GDCM/src/gdcm/Utilities/gdcmext/mec_mr3_io.c#L315-L323

I'll have to check to see if ITK has any current build with this compiler.

[Artur-man]

I have tried the win build on my local win machine too and i am getting the same error, thus I will also try GHA in
https://github.com/Artur-man/ITKR/blob/master/configure.win

Getting the same error with the ITK build:
https://github.com/Artur-man/ITKR/actions/runs/21477502152/job/61865100103

C:\Users\runneradmin\AppData\Local\Temp\Rtmp6FJg3S\R.INSTALL2188552d2b27\ITKR\src\itks\Modules\ThirdParty\GDCM\src\gdcm\Utilities\gdcmext\mec_mr3_io.c
C:\Users\runneradmin\AppData\Local\Temp\Rtmp6FJg3S\R.INSTALL2188552d2b27\ITKR\src\itks\Modules\ThirdParty\GDCM\src\gdcm\Utilities\gdcmext\mec_mr3_io.c: In function 'aligned_alloc_impl':
C:\Users\runneradmin\AppData\Local\Temp\Rtmp6FJg3S\R.INSTALL2188552d2b27\ITKR\src\itks\Modules\ThirdParty\GDCM\src\gdcm\Utilities\gdcmext\mec_mr3_io.c:146:15: error: implicit declaration of function 'posix_memalign' [-Wimplicit-function-declaration]
  146 |   int error = posix_memalign(&allocPtr, alignment, size);
      |               ^~~~~~~~~~~~~~
mingw32-make[3]: *** [Modules\ThirdParty\GDCM\src\gdcm\Source\MediaStorageAndFileFormat\CMakeFiles\gdcmMSFF.dir\build.make:1627: Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3_io.c.obj] Error 1

[blowekamp]

What should be tested is building vanilla ITK with the mingw compiler.

I suspect that a the preprocessor logic around this function need adjustment:
https://github.com/search?q=repo%3Amalaterre%2FGDCM%20posix_memalign&type=code

[Artur-man]

Just tried on windows using
https://itk.org/ITKSoftwareGuide/html/Book1/ITKSoftwareGuide-Book1ch2.html#x22-200002.2.2

the same error persists.

$ git clone https://github.com/InsightSoftwareConsortium/ITK.git
$ where cmake
C:\rtools45\x86_64-w64-mingw32.static.posix\bin\cmake.exe
$ cmake -DBUILD_TESTING:BOOL=OFF ../ITK
$ cmake --build .

....

[ 58%] Building C object Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3.c.obj
[ 58%] Building C object Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3_io.c.obj
C:/ITK/Modules/ThirdParty/GDCM/src/gdcm/Utilities/gdcmext/mec_mr3_io.c: In function 'aligned_alloc_impl':
C:/ITK/Modules/ThirdParty/GDCM/src/gdcm/Utilities/gdcmext/mec_mr3_io.c:320:15: error: implicit declaration of function 'posix_memalign' [-Wimplicit-function-declaration]
  320 |   int error = posix_memalign(&allocPtr, alignment, size);
      |               ^~~~~~~~~~~~~~
make[2]: *** [Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/build.make:1625: Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/__/__/Utilities/gdcmext/mec_mr3_io.c.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:10275: Modules/ThirdParty/GDCM/src/gdcm/Source/MediaStorageAndFileFormat/CMakeFiles/gdcmMSFF.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

[Artur-man]

Would it work if we turn off ITK IO GDCM module ?
-DITK_USE_SYSTEM_GDCM=OFF
-DModule_ITKGDCM=OFF
-DModule_ITKIOGDCM=OFF

https://discourse.itk.org/t/how-to-disable-gdcm-functionality/6330

It seems that the GDCM is sooo deep into the dependancy tree that so many modules have to be manually turned on/off to get GDCM IO off.

I turned off a bunch of stuff in ITK build in
https://github.com/Artur-man/ITKR/actions/runs/21499621731/job/61942787094

Thus now the install completes as expected. If you guys have a easy way to disable GDCM without turning off major modules please let me know :D ... or we can do a bug report in GDCM as well.

[Artur-man]

It seems SimpleITK enforces GDCM:

-- Looking for include file strings.h - found
-- Looking for include file pwd.h
-- Looking for include file pwd.h - found
-- Looking for include file sys/select.h
-- Looking for include file sys/select.h - found
-- ITK fetched from repository and built from source
--   Source directory: /Users/amanuky/Dropbox/Research/MDC/Projects/SpatialData/test/OtherPackages/SimpleITK_build/SimpleITK_build/_deps/itk-src
--   Binary directory: /Users/amanuky/Dropbox/Research/MDC/Projects/SpatialData/test/OtherPackages/SimpleITK_build/SimpleITK_build/_deps/itk-build
CMake Error at CMake/sitkCheckForITKModuleDependencies.cmake:37 (message):
  SimpleITK requires that the ITKIOGDCM module be turned on in ITK
Call Stack (most recent call first):
  CMakeLists.txt:73 (include)

Unfortunately, the build will fail in R windows configure as long as GDCM is not fixed or enforced.

[blowekamp]

I installed a mingw compiler is a msys2 environment. With just compelling ITK, from either release or main I was able to reproduce this error and there were others too.

So there needs to be work done with ITK and it's third party libraries to support the mingw compiler.

[Artur-man]

Thanks @blowekamp. It is nice that we discovered this issue.

In my case, I need a remote win binary to decrease installation time (due to compilation). Thus as a last resort, I will try to point the compiler to an existing GDCM installation in a win machine. If that does not work, I will wait for the developments as you said.

We have a nice package that wraps SimpleITK together with OpenCV here:
https://github.com/BIMSBbioinfo/VoltRon

It is better for users to install binaries of dependancies instead of lengthy compilation.

[blowekamp]

I haven't had a chance to get back to this, but maybe try some older release version to see if they work with mingw.

[blowekamp]

I have made PR for ITK to address compilation error encountered: ITK/ITK#5774 and ITK/ITK#5773

[Artur-man]

Wow @blowekamp, thanks for the quick fix InsightSoftwareConsortium/ITK#5792. I will check the build again when I have the time!

[blowekamp]

When you configure the SimpleITK Superbuild you should be able to set ITK_GIT_TAG to ea28c4c7ff4041acbe425a302eca5be46e957def to build against those changes.

Unfortunately the current ITK main underdevelopment seems to have and issue with HDF5 and mingw. It is similar to the error reported here: HDFGroup/hdf5#5885 This issue is unresolved but that some suggested changes/hacks.

[Artur-man]

ITK now builds and SimpleITK keeps on going. The only problem with the win build is that it is slow, even with using all default number of cores which is 8, and the circleci puts 1 hour limit to a job on free plan. ITK builds in around 15 mins with 8 cores, but I still wait SimpleITK to finish.

UPDATE: Yeap, jobs is terminated before build could complete.

[blowekamp]

ITK now builds and SimpleITK keeps on going. The only problem with the win build is that it is slow, even with using all default number of cores which is 8, and the circleci puts 1 hour limit to a job on free plan. ITK builds in around 15 mins with 8 cores, but I still wait SimpleITK to finish.

UPDATE: Yeap, jobs is terminated before build could complete.

Yes, it has been tricky getting SimpleITK to build on some free resources. There have even a variety of limitations that have been encountered disk space, total CPU power available, memory for linking, but usually there are some tweaks and tunes to get it working. f I recall correctly @zivy said that the CircleCI builds were faster than GHA, but may GHA by have less CPUs over more time to allow the build to complete.

It can be tricky to determine the correct parallel level for different services. Do you have a link to the build so I can see how far it's progressed?

Also in the current SimpleITK main underdevelopment for SimpleITK 3.0, the super build is not longer needed. The code generation is now done through python, and support for getting swig from python is also there. So there are potential improvements in the build process also coming.

[Artur-man]

I think this link is public enough:
https://app.circleci.com/pipelines/github/BIMSBbioinfo/SimpleITKRInstaller/82/workflows/8666bc85-8f37-4e6a-9eb0-fa7ab5ff4ed4/jobs/92

[blowekamp]

That build does not look close at all. I'd double check that the correct parallelism is occurring through make. And there may be a need to switch to GHA for the windows build. The windows builds on for SimpleITK take just over 2hrs in the main repository.

https://github.com/SimpleITK/SimpleITK/actions/runs/21766261699

[Artur-man]

i might be able to do that, will test on my local win as well

[Artur-man]

Will check here, lets see if it is gonna finish, running on 8 cores:
https://github.com/BIMSBbioinfo/SimpleITKRInstaller/actions/runs/22228754864/job/64302321292

UPDATE: it was finished in 1.5 hours but there is a problem with R library installation, perhaps I haven't set env variables correctly.

[blowekamp]

It looks like R symbols were not found. Maybe the library isn't linked correctly?

[Artur-man]

I will test by building locally and parse the build from the manual location in configure.win. Hard to test when you wait 1.5 hours each time.

[Artur-man]

I am getting the same linking error during

cmake --build . --target SimpleITK-build -- -j"${MAKEJ}"

we get a bunch of undefined reference error, mostly (I assume) variables available in Rinternals.h. It seems to me that rtools does not like swig compiling simpleitk R C++ libraries on the configuration phase configure.win.

I checked ITKR again, and the build/install here happens in Makevars.win. Perhaps we should do something similar ?

https://github.com/ANTsX/ITKR/blob/master/configure.win
https://github.com/ANTsX/ITKR/blob/master/src/Makevars.win

Have you guys never wrapped R bindings of SimpleITK in any windows machine before? or is this the first time we are doing this ?

[richardbeare]

Sorry I haven't been participating in the discussion.
We got it running and passing all tests quite a few years ago - about the time that R was switching over to the new tool chain for v4. This wasn't from the SimpleITKRInstaller, but inside the main SimpleITK framework. At the time the issue was that it wasn't possible to get the link stage running on any of the standard CI VMs that were available because more than 8G was required for the linking step.

If you check https://github.com/richardbeare/SimpleITK and look for old branches with names like "RWindowsBattles", you may find some useful but possibly out of date clues to how we got there.

I recommend ignoring the installer at the start and focussing on getting a traditional SimpleITK build to work on a local machine over which you have control and with >8GB RAM, using the RTools 4+ and the toolchain that it comes with - I have definitely had success with that in the past. I recall having to contribute patches to the RTools project to bump versions of TBB.

Scarily, I checked my old email and the "passing all tests" emails were 2017!
This used a mix of msys and R toolchains and was pre RTools 4. I revisited later as rtools 4 was released, which was when I did the work on TBB etc, but never cracked the problem of memory requirements in order to create binaries via the release pipelines that were available at the time

[Artur-man]

Thanks @richardbeare, yes I ignored the installer and built the cmake and cmake --build that failed using rtools mingw32 compiler that throw those reference errors as well. I will check the branch now.

[Artur-man]

We currently do not link R.exe and R.dll, perhaps thats the issue ? ... I do not know about the ninja build though would that work with mingw32 ?

RR=$(which R)
R64=$(dirname $RR)/x64/R.exe
R64DLL=$(dirname $R64)/R.dll

cmake -G Ninja ../SimpleITK/SuperBuild -DCMAKE_CXX_FLAGS="--param ggc-min-expand=0 --param ggc-min-heapsize=2648000" -DR_COMMAND=${R64} -DR_LIBRARY_BASE=${R64DLL} -DWRAP_R=ON