Building on Windows with Eigen 3.4 takes a LOOOOOT of time

[cdcseacave]

Sometimes it is even crashing the machine as it runs out of memory even though I have 32GB RAM.
Most probably it is not particular to opengv but any library using heavy templates. Does anybody have any suggestion how to make it compile (not run OOM) or even speed up? Are there any flags I can use?

[zellx3]

same issue happen to me, Debug build was almost fast. but release taking more that 5 hours and I'm still waiting

[ryanalex98]

I am having this issue as well. Were any of you able to work around this? @irenji @cdcseacave
FYI - I am using a VM running Ubuntu 20.04 - I have it set up with 12 GB of RAM. I am building opengv as part of a larger overall build process (maplab) and this particular build step is the culprit - all 106 other libraries build successfully. It slowly consumes all my RAM (all 12 GB in about 4 mins, while only building 2%).

[MuyanXiaoXMU]

Same issue here, any solution?

[simogasp]

It seems to work fine on linux and osx, it seems that on windows the compiler runs out of resources. I experienced that on my machine and also on my attempt to add the CI on github actions
https://github.com/alicevision/opengv/actions/runs/5165375471/job/13981189036#step:6:227

Maybe trying with precompiled headers?

[javrtg]

Using the MinGW64 toolchain on Windows significantly speeds up the building process--it takes around 3 minutes on my machine. MinGW64 provides gcc, making the compilation process akin to a Linux environment.

In case it is helpful (and as a reference for myself in the future :)), below I add steps to get (py)opengv up and running.

Warning

There is a specific issue that needs to be solved first. OpenGV defines struct timeval and gettimeofday for Windows systems since they are native only to UNIX-like systems. Since MinGW already provides these, conflicts can occur during the build process. To prevent this, we need to modify L33 in test/time_measurement.cpp and L37 in test/time_measurement.hpp to change in both:

#ifdef WIN32

to

#if defined(WIN32) && !defined(__MINGW32__)

Setting up MinGW64 toolchain via MSYS2

To install the MinGW64 toolchain, a recommended approach is to first install MSYS2. MSYS2 comes with a package manager, pacman, useful for installing and managing additional dependencies like cmake and eigen. Alternatively, we can install these dependencies using other methods, such as within a conda or mamba environment.

following MSYS2 docs, after installing MSYS2, we need to launch the MSYS2 MSYS shell and update the package database by running:

pacman -Suy

As the docs say, we may be prompted to close all terminals if core packages are updated:

:: To complete this update all MSYS2 processes including this terminal will be closed.
   Confirm to proceed [Y/n]

If prompted, we need to close the terminal, reopen it, and run pacman -Suy again to update remaining packages.

Next, we need to install the gcc and g++ compilers using:

pacman -S --needed base-devel mingw-w64-x86_64-toolchain

Installing dependencies

Just as an example, below it is explained how to install the dependencies on a conda/mamba environment, but there are other alternatives, such as installing them in the MSYS2 MSYS shell.

Open a conda prompt and run the following commands:

# create fresh environment to build opengv (replace 3.10 to the desired python version).
(base) C:\random\path> mamba create -n opengv python=3.10 cmake ninja eigen -y

# activate the new environment
(base) C:\random\path> mamba activate opengv
(opengv) C:\random\path> 

Building OpenGV

First, to have access to the compilers, we need to add the MINGW64 binaries directory to the the PATH environment variable. If MSYS2 was installed using default options, this directory is typically C:\msys64\mingw64\bin.
We can temporarily modify the PATH variable as follows:

(opengv) C:\random\path> set PATH=%PATH%;C:\msys64\mingw64\bin

Finally, to build OpenGV:

# navigate to the cloned opengv repository.
(opengv) C:\random\path> cd  \path\to\opengv

# create build directory.
(opengv) \path\to\opengv> mkdir build && cd build

# Build using cmake and ninja (adjust flags as needed):
(opengv) \path\to\opengv> cmake -G Ninja .. -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=OFF -DBUILD_PYTHON=ON && ninja

The build process should take ~3 mins and the resulting compiled libraries will be stored in the folder \path\to\opengv\build\lib.

Warning

Since the Python extension isn't statically compiled, the compiled .pyd module (e.g. pyopengv.cp310-win_amd64.pyd) will rely on several MinGW .dll files at runtime. Thus, before importing pyopengv, we need to add the MinGW binaries directory to the DLL search path in Python. This can be done for for Python >= 3.8 as follows:

import os
with os.add_dll_directory("C:\\msys64\\mingw64\\bin"):
    import pyopengv

Alternatively, to automate this, we can move the .pyd module to a manually created pyopengv folder within the site-packages directory of our Python interpreter. Inside this pyopengv folder, besides having the .pyd file, we need to create a __init__.py file that contains something like:

import os
from pathlib import Path

# constant for MinGW64 directory path.
MINGW64_PATH = Path("C:\\msys64\\mingw64\\bin")


def import_pyopengv():
    """Import pyopengv, with optional modification to DLL search path."""
    try:
        # check if the MinGW64 directory exists
        if MINGW64_PATH.is_dir():
            # augment .dll search path to include MinGW64's bin directory.
            with os.add_dll_directory(str(MINGW64_PATH)):
                from . import pyopengv
        else:
            from . import pyopengv
        return pyopengv
    except ImportError as e:
        raise ImportError(f"Failed to import pyopengv: {e}")


import_pyopengv()

# clean up the namespace.
del os, Path, import_pyopengv, MINGW64_PATH

After this, pyopengv can be imported, from any directory, as follows:

from pyopengv import pyopengv

---

P.S. @simogasp I think the previous process can be included in a GHA workflow by using the action from MSYS2. The following is untested, but I believe something like it should work:

name: CI

on: [push]

jobs:
    build_opengv_on_windows:
        runs-on: windows-latest

        steps:

          # ... other needed steps ...

          - name: Checkout
            uses: actions/checkout@v3

          - name: Setup MSYS2 on Windows
            uses: msys2/setup-msys2@v2
            with:
              update: true
              msystem: 'MSYS'
              install: >-
                base-devel
                mingw-w64-x86_64-toolchain
                mingw-w64-x86_64-eigen3
                mingw-w64-x86_64-ninja
                mingw-w64-x86_64-cmake

          - name: build OpenGV
            shell: msys2 {0}
            env:
              MSYSTEM: MINGW64
            run: |
              mkdir build && cd build
              cmake -G Ninja .. -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=OFF -DBUILD_PYTHON=OFF && ninja