Use pip install ininstall_base.py; remove deprecated setup.py references (#399)

Update installation instructions to include verbose flag by default, showing CMake output.

Author: nayyirahsan

.github/workflows/array_api.yml

@@ -35,10 +35,9 @@ jobs:
           conda install -c conda-forge numpy mypy cmake pytest pybind11 scikit-build patchelf tqdm &&
           conda remove --force-remove -y ninja || true
       - name: Install pykokkos-base
-        working-directory: base
         run: |
           export CMAKE_BUILD_PARALLEL_LEVEL=2 &&
-          python setup.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=5
+          python install_base.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=5
       - name: Install pykokkos
         run: |
           python -m pip install .

.github/workflows/main_ci.yml

@@ -26,10 +26,9 @@ jobs:
           python -m pip install --upgrade pip
           python -m pip install --upgrade numpy mypy==1.0.1 cmake pytest pybind11 scikit-build patchelf
       - name: Install pykokkos-base
-        working-directory: base
         run: |
           export CMAKE_BUILD_PARALLEL_LEVEL=2
-          python setup.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=5
+          python install_base.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=5
       - name: Install pykokkos
         run: |
           python -m pip install .

.github/workflows/nightly.yml

@@ -37,10 +37,9 @@ jobs:
           cmake --build . --parallel 2
           cmake --install .
       - name: Install pykokkos-base
-        working-directory: base
         run: |
           export CMAKE_BUILD_PARALLEL_LEVEL=2
-          python setup.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DKokkos_DIR=/tmp/kokkos_install/lib/cmake/Kokkos/ -DENABLE_INTERNAL_KOKKOS=OFF
+          python install_base.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DKokkos_DIR=/tmp/kokkos_install/lib/cmake/Kokkos/ -DENABLE_INTERNAL_KOKKOS=OFF
       - name: Install pykokkos
         run: |
           python -m pip install .

README.md

@@ -36,7 +36,7 @@ conda env update -n pyk -f base/environment.yml
 conda activate pyk
 
 # Install pykokkos-base from the root directory
-python install_base.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=3 -DENABLE_CUDA=ON -DENABLE_THREADS=OFF -DENABLE_OPENMP=ON
+python install_base.py install --verbose -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF -DENABLE_VIEW_RANKS=3 -DENABLE_CUDA=ON -DENABLE_THREADS=OFF -DENABLE_OPENMP=ON
 ```
 
 #### Installing pykokkos

base/README.md

@@ -53,7 +53,7 @@ In order to write native Kokkos in Python, see [pykokkos](https://github.com/kok
 
 ## Installation
 
-You can install this package via CMake or Python's `setup.py`. The important cmake options are:
+You can install this package via CMake or via `pip install ./`. The important CMake options are:
 
 - `ENABLE_VIEW_RANKS` (integer)
 - `ENABLE_LAYOUTS` (bool)
@@ -82,8 +82,7 @@ If the `ENABLE_INTERNAL_KOKKOS` option is not specified the first time CMake is
 find an existing Kokkos installation. If no existing installation is found, it will build and install
 Kokkos from a submodule. When Kokkos is added as a submodule, you can configure the submodule
 as you would normally configure Kokkos. However, due to some general awkwardness configuring cmake
-from `setup.py` (especially via `pip install`), CMake tries to "automatically" configure
-reasonable default CMake settings for the Kokkos submodule.
+when configuring via pip builds, CMake tries to "automatically" configure reasonable default CMake settings for the Kokkos submodule.
 
 Here are the steps when Kokkos is added as a submodule:
 
@@ -108,28 +107,10 @@ Here are the steps when Kokkos is added as a submodule:
 cmake -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF /path/to/source
 ```
 
-### Configuring Options via `setup.py`
+### Configuring Options
 
-There are three ways to configure the options:
-
-1. Via the Python argparse options `--enable-<option>` and `--disable-<option>`
-2. Setting the `PYKOKKOS_BASE_SETUP_ARGS` environment variable to the CMake options
-3. Passing in the CMake options after a `--`
-
-All three lines below are equivalent (deprecated format):
-
-```console
-python setup.py install --enable-layouts --disable-memory-traits
-PYKOKKOS_BASE_SETUP_ARGS="-DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF" python setup.py install
-python setup.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF
-```
-
-### Configuring Options via `pip`
-
-Pip does not handle build options well. Thus, it is recommended to use the `PYKOKKOS_BASE_SETUP_ARGS`
-environment variable noted above. 
-
-We suggest using the following line to install pykokkos-base:
+For pip installs, set `PYKOKKOS_BASE_SETUP_ARGS` to a space-delimited list of CMake `-D...`
+options, then run:
 
 ```
 PYKOKKOS_BASE_SETUP_ARGS="-DKokkos_ENABLE_THREADS=OFF \
@@ -143,8 +124,7 @@ PYKOKKOS_BASE_SETUP_ARGS="-DKokkos_ENABLE_THREADS=OFF \
 
 `--verbose` is optional, but it shows installation progress in real time.
 
-> `pip install ./` will build against the latest release in the PyPi repository.
-> In order to pip install from this repository, use `pip install --user -e .`
+> `pip install ./` installs from this checkout. For editable development, use `pip install -e .`.
 
 ## Differences vs. Kokkos C++
 

base/setup.py

@@ -91,11 +91,8 @@ def add_arg_bool_option(lc_name, disp_name, default=None):
     default=[],
     type=str,
     nargs="*",
-    help="{}{}{}".format(
-        "Pass arguments to cmake. Use w/ pip installations",
-        "and --install-option, e.g. --install-option=--cmake-args=",
-        '"-DENABLE_LAYOUTS=ON -DKokkos_DIR=/usr/local/lib/cmake/Kokkos"',
-    ),
+    help="Pass arguments to CMake. For pip installs, "
+    "set PYKOKKOS_BASE_SETUP_ARGS instead.",
 )
 
 args, left = parser.parse_known_args()
@@ -139,8 +136,8 @@ def add_arg_bool_option(lc_name, disp_name, default=None):
     cmake_args += [f"-DCMAKE_OSX_DEPLOYMENT_TARGET={darwin_version}"]
 
 # DO THIS LAST!
-# support PYKOKKOS_BASE_SETUP_ARGS environment variables because
-#  --install-option for pip is a pain to use
+# Support PYKOKKOS_BASE_SETUP_ARGS environment variables so pip builds don't
+# need deprecated/awkward command-line build argument plumbing.
 # PYKOKKOS_BASE_SETUP_ARGS should be space-delimited set of cmake arguments, e.g.:
 #   export PYKOKKOS_BASE_SETUP_ARGS="-DENABLE_LAYOUTS=OFF -DENABLE_MEMORY_TRAITS=ON"
 env_cmake_args = os.environ.get("PYKOKKOS_BASE_SETUP_ARGS", None)

docs/installation.rst

@@ -77,7 +77,7 @@ install ``base`` with required CMake flags (example performs an install with  Op
 
 .. code-block:: bash
 
-   python install_base.py install -- \
+   python install_base.py install --verbose -- \
       -DENABLE_VIEW_RANKS=3 \             # maximum number of view ranks enabled
       -DENABLE_MEMORY_TRAITS=OFF \        # disable memory space concept
       -DENABLE_THREADS=OFF \              # disable pthreads execution space

install_base.py

@@ -8,7 +8,7 @@
     python install_base.py install -- [FLAGS]
 
 This is equivalent to:
-    cd base/ python setup.py install -- [FLAGS]
+    cd base/ python -m pip install .  (with PYKOKKOS_BASE_SETUP_ARGS set)
 """
 
 import os
@@ -24,20 +24,51 @@ def main():
     if not os.path.exists(base_dir):
         print(f"Error: base directory not found at {base_dir}")
         sys.exit(1)
-
-    setup_py = os.path.join(base_dir, "setup.py")
-    if not os.path.exists(setup_py):
-        print(f"Error: setup.py not found at {setup_py}")
-        sys.exit(1)
     original_dir = os.getcwd()
     os.chdir(base_dir)
 
     try:
-        # Run setup.py with all the arguments passed to this script
-        cmd = [sys.executable, "setup.py"] + sys.argv[1:]
+        argv = sys.argv[1:]
+        if not argv:
+            raise SystemExit(
+                "Usage: python install_base.py install -- -D<...> [other CMake -D flags]"
+            )
+
+        # Allow users to pass verbosity flags through this wrapper.
+        pip_verbose = ("--verbose" in argv) or ("-v" in argv)
+
+        # Parse CMake flags after `--`.
+        #
+        # Typical invocation:
+        #   python install_base.py install -- -DENABLE_LAYOUTS=ON -DENABLE_VIEW_RANKS=3
+        cmake_flags = []
+        if "--" in argv:
+            idx = argv.index("--")
+            cmake_flags = argv[idx + 1 :]
+        else:
+            # Backwards-compat: if someone calls `python install_base.py install -D...`
+            # treat everything except the leading `install` as CMake flags.
+            cmake_flags = [a for a in argv if a != "install"]
+
+        # Ensure wrapper verbosity flags don't leak into CMake arguments.
+        # These are only meant to affect `pip install`, not the -D... CMake args.
+        cmake_flags = [a for a in cmake_flags if a not in ("-v", "--verbose")]
+
+        env = os.environ.copy()
+        existing = env.get("PYKOKKOS_BASE_SETUP_ARGS", "").strip()
+        if cmake_flags:
+            cmake_str = " ".join(cmake_flags).strip()
+            env["PYKOKKOS_BASE_SETUP_ARGS"] = (
+                f"{existing} {cmake_str}".strip() if existing else cmake_str
+            )
+
+        cmd = [sys.executable, "-m", "pip", "install", "."]
+        if pip_verbose:
+            cmd.append("--verbose")
+
         print(f"Running: {' '.join(cmd)}")
         print(f"Working directory: {os.getcwd()}")
-        result = subprocess.run(cmd)
+        result = subprocess.run(cmd, env=env)
         sys.exit(result.returncode)
     finally:
         os.chdir(original_dir)