Merge pull request #2347 from borglab/feature/parrallelUpdate

dellaert · web-flow · commit bebf3f28175a · 2026-01-11T20:41:57.000-05:00
Parallel separator update in MFSolver
diff --git a/.github/README.md b/.github/README.md
@@ -0,0 +1,40 @@
+# GTSAM GitHub Configuration
+
+This directory contains GitHub-specific configuration, including issue templates and CI workflows.
+
+## Workflows Overview
+
+The `.github/workflows` directory contains definitions for the various CI/CD processes:
+
+-   **`build-linux.yml`**: Main Linux CI. Runs inside the pre-built Docker containers mentioned above to compile and test GTSAM with various compilers (GCC, Clang) and configurations.
+-   **`build-macos.yml`**: Compiles and tests GTSAM on macOS runners.
+-   **`build-windows.yml`**: Compiles and tests GTSAM on Windows runners using MSVC.
+-   **`build-python.yml`**: Verifies the Python wrapper compilation across Linux, macOS, and Windows.
+-   **`build-cibw.yml`**: Builds distributable Python wheels using `cibuildwheel`. It builds dependencies (like Boost) from source to ensure ABI compatibility.
+-   **`prod-cibw.yml`**: Used for production wheel builds (often triggered on release).
+-   **`deploy.yml`**: Handles deployment tasks (e.g., docs, release artifacts).
+
+## Docker CI Images (Linux Only)
+
+The **Linux CI** workflow (`build-linux.yml`) relies on pre-built Docker images to ensure consistency and speed up build times. These images are hosted in the [borglab/docker-images](https://github.com/borglab/docker-images) repository.
+
+*Note: macOS and Windows workflows use standard GitHub Actions runners and install dependencies (like Boost) via package managers (Homebrew, Chocolatey) or from source.*
+
+### Building and Updating Images
+
+If you need to update an existing CI image or add a new one (e.g., for a new Ubuntu version or compiler):
+
+1.  **Navigate to the `docker-images` repository:**
+    ```bash
+    cd ../docker-images/gtsam-ci
+    ```
+2.  **Add or modify a Dockerfile:**
+    -   Follow the naming convention: `ubuntu-<version>-<compiler>-<version>.Dockerfile`.
+    -   Base images are defined in `*-base.Dockerfile`.
+3.  **Build and Push:**
+    Use the provided script to build and push to Docker Hub (requires `borglab` permissions):
+    ```bash
+    ./build_and_push.sh <dockerhub-username>
+    ```
+
+For more details, see the [README in the docker-images repository](https://github.com/borglab/docker-images/blob/main/gtsam-ci/README.md).
diff --git a/containers/README.md b/containers/README.md
@@ -1,127 +1,131 @@
-# GTSAM Containers
+# GTSAM Docker Images
 
-- container files to build images
-- script to push images to a registry
-- instructions to pull images and run containers
+The official Docker images for GTSAM are maintained in the [borglab/docker-images](https://github.com/borglab/docker-images) repository.
 
-## Dependencies
+## Available Images
 
-- a container engine such as [`Docker Engine`](https://docs.docker.com/engine/install/)
+The following images are available on Docker Hub, primarily under the `borglab` namespace:
 
-## Pull from Docker Hub
+-   **[borglab/gtsam](https://hub.docker.com/r/borglab/gtsam)**:
+    A pre-compiled environment containing the latest `develop` branch of GTSAM. Useful for quick testing or as a base for downstream applications.
+    -   *Source:* [`docker-images/gtsam`](https://github.com/borglab/docker-images/tree/main/gtsam)
 
-Various GTSAM image configurations are available at [`docker.io/borglab/gtsam`](https://hub.docker.com/r/borglab/gtsam). Determine which [tag](https://hub.docker.com/r/borglab/gtsam/tags) you want and pull the image.
+-   **[borglab/gtsam-manylinux](https://hub.docker.com/r/borglab/gtsam-manylinux)**:
+    An environment based on `manylinux2014` tailored for building Python wheels for GTSAM.
+    -   *Source:* [`docker-images/gtsam-manylinux`](https://github.com/borglab/docker-images/tree/main/gtsam-manylinux)
 
-Example for pulling an image with GTSAM compiled with TBB and Python support on top of a base Ubuntu 22.04 image.
+-   **[borglab/ubuntu-boost-tbb](https://hub.docker.com/r/borglab/ubuntu-boost-tbb)**:
+    Base image (Ubuntu 24.04) with Boost and TBB libraries pre-installed.
+    -   *Source:* [`docker-images/ubuntu-boost-tbb`](https://github.com/borglab/docker-images/tree/main/ubuntu-boost-tbb)
+
+-   **CI Images**:
+    Various images used for Continuous Integration, covering different Ubuntu versions (22.04, 24.04) and compilers (Clang, GCC).
+    -   *Source:* [`docker-images/gtsam-ci`](https://github.com/borglab/docker-images/tree/main/gtsam-ci)
+
+## Usage
+
+### Running GTSAM
+
+To start an interactive shell in a container with GTSAM pre-installed:
 
 ```bash
-docker pull docker.io/borglab/gtsam:4.2.0-tbb-ON-python-ON_22.04
+docker run -it borglab/gtsam:latest
 ```
 
-[`docker.io/borglab/gtsam-vnc`](https://hub.docker.com/r/borglab/gtsam-vnc) is also provided as an image with GTSAM that will run a VNC server to connect to.
+### Using the Python Wrapper
 
-## Using the images
+The `borglab/gtsam` image typically includes Python bindings. To use them:
 
-### Just GTSAM
+1.  Start the container:
+    ```bash
+    docker run -it borglab/gtsam:latest
+    ```
+2.  Launch Python:
+    ```bash
+    python3
+    ```
+3.  Import GTSAM:
+    ```python
+    import gtsam
+    print(gtsam.Pose3())
+    ```
 
-To start the image, execute
+## Building Images
 
-```bash
-docker run -it borglab/gtsam:4.2.0-tbb-ON-python-OFF_22.04
-```
+To build these images locally or contribute changes, please refer to the **[borglab/docker-images](https://github.com/borglab/docker-images)** repository. It contains the Dockerfiles and build scripts for all the images listed above.
 
-after you will find yourself in a bash shell.
+### Legacy Configuration
 
-### GTSAM with Python wrapper
 
-To use GTSAM via the python wrapper, similarly execute
 
-```bash
-docker run -it borglab/gtsam:4.2.0-tbb-ON-python-ON_22.04
-```
+The following files in this directory are legacy artifacts and are **no longer actively maintained**:
 
-and then launch `python3`:
 
-```bash
-python3
->>> import gtsam
->>> gtsam.Pose2(1,2,3)
-(1, 2, 3)
-```
 
-### GTSAM with Python wrapper and VNC
+-   **`Containerfile`**: Build instructions for a standalone GTSAM image (cloning from git and building from source).
 
-First, start the image, which will run a VNC server on port 5900:
+-   **`compose.yaml`**: A Docker Compose wrapper used for configurable builds (via `.env` variables like `GTSAM_WITH_TBB`, `GTSAM_BUILD_PYTHON`) and standardized image tagging.
 
-```bash
-docker run -p 5900:5900 borglab/gtsam-vnc:4.2.0-tbb-ON-python-ON_22.04
-```
+-   **`hub_push.sh`**: A utility script to iterate through configuration matrices and push multiple image variants to Docker Hub.
 
-Then open a remote VNC X client, for example:
 
-#### Linux
 
-```bash
-sudo apt-get install tigervnc-viewer
-xtigervncviewer :5900
-```
+For official builds and the most up-to-date configurations, please refer to the **[borglab/docker-images](https://github.com/borglab/docker-images)** repository.
 
-#### Mac
 
-The Finder's "Connect to Server..." with `vnc://127.0.0.1` does not work, for some reason. Using the free [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/), enter `0.0.0.0:5900` as the server.
 
-## Build images locally
+> **TODO**: Consider migrating the configurable build and matrix-pushing functionality from these legacy files into the `docker-images` repository to support more flexible local builds.
 
-### Build Dependencies
 
-- a [Compose Spec](https://compose-spec.io/) implementation such as [docker-compose](https://docs.docker.com/compose/install/)
 
-### `gtsam` image
+## VNC Support (`gtsam-vnc`)
 
-#### `.env` file
 
-- `GTSAM_GIT_TAG`: [git tag from the gtsam repo](https://github.com/borglab/gtsam/tags)
-- `UBUNTU_TAG`: image tag provided by [ubuntu](https://hub.docker.com/_/ubuntu/tags) to base the image off of
-- `GTSAM_WITH_TBB`: to build GTSAM with TBB, set to `ON`
-- `GTSAM_BUILD_PYTHON`: to build python bindings, set to `ON`
-- `CORES`: number of cores to compile with
 
-#### Build `gtsam` image
+The **gtsam-vnc** image configuration is available locally in the [`gtsam-vnc`](./gtsam-vnc) subdirectory. This image extends the official `borglab/gtsam` image by adding a VNC server, allowing you to view GUI applications (like Matplotlib plots) running inside the container.
 
-```bash
-docker compose build
-```
 
-### `gtsam-vnc` image
 
-#### `gtsam-vnc/.env` file
+### Building and Running VNC Image
 
-- `GTSAM_TAG`: image tag provided by [gtsam](https://hub.docker.com/r/borglab/gtsam/tags)
 
-#### Build `gtsam-vnc` image
 
-```bash
-docker compose --file gtsam-vnc/compose.yaml build
-```
+1.  **Navigate to the directory:**
 
-## Push to Docker Hub
+    ```bash
 
-Make sure you are logged in via: `docker login docker.io`.
+    cd gtsam-vnc
 
-### `gtsam` images
+    ```
 
-Specify the variables described in the `.env` file in the `hub_push.sh` script.
-To push images to Docker Hub, run as follows:
 
-```bash
-./hub_push.sh
-```
 
-### `gtsam-vnc` images
+2.  **Build the image:**
 
-Specify the variables described in the `gtsam-vnc/.env` file in the `gtsam-vnc/hub_push.sh` script.
-To push images to Docker Hub, run as follows:
+    You can build it using Docker Compose or directly with Docker.
 
-```bash
-./gtsam-vnc/hub_push.sh
-```
+    ```bash
+
+    # Example using docker build
+
+    docker build -t gtsam-vnc .
+
+    ```
+
+
+
+3.  **Run with Port Forwarding:**
+
+    Map port 5900 to access the VNC server.
+
+    ```bash
+
+    docker run -p 5900:5900 gtsam-vnc
+
+    ```
+
+
+
+4.  **Connect:**
+
+    Use a VNC client to connect to `localhost:5900`.
diff --git a/gtsam/base/SymmetricBlockMatrix.h b/gtsam/base/SymmetricBlockMatrix.h
@@ -270,6 +270,12 @@ namespace gtsam {
     void updateFromOuterProductBlocks(const VerticalBlockMatrix& other,
                                       const std::vector<DenseIndex>& blockIndices);
 
+    /// Add the upper-triangular part of another symmetric block matrix.
+    void addUpperTriangular(const SymmetricBlockMatrix& other) {
+      assert(nBlocks() == other.nBlocks());
+      full().triangularView<Eigen::Upper>() += other.full();
+    }
+
     /// @}
     /// @name Accessing the full matrix.
     /// @{
diff --git a/gtsam/base/tests/testPriorityScheduler.cpp b/gtsam/base/tests/testPriorityScheduler.cpp
@@ -62,15 +62,19 @@ TEST(PriorityScheduler, VoidPriorityOrderingSingleWorker) {
   scheduler.waitForAllTasks();
   EXPECT_LONGS_EQUAL(3, executionOrder.size());
   // With a single worker, tasks are not preempted: the first scheduled task may
-  // start running before later (higher-priority) tasks are submitted. Once all
-  // tasks are enqueued, remaining work should respect priority order.
+  // start running before later (higher-priority) tasks are submitted. Also,
+  // since the third task is submitted last, the worker may run the first two
+  // tasks before the third is enqueued.
   const bool strictPriority =
       (executionOrder.at(0) == 1 && executionOrder.at(1) == 3 &&
        executionOrder.at(2) == 5);
   const bool firstTaskRanImmediately =
       (executionOrder.at(0) == 5 && executionOrder.at(1) == 1 &&
        executionOrder.at(2) == 3);
-  EXPECT(strictPriority || firstTaskRanImmediately);
+  const bool thirdTaskEnqueuedLate =
+      (executionOrder.at(0) == 1 && executionOrder.at(1) == 5 &&
+       executionOrder.at(2) == 3);
+  EXPECT(strictPriority || firstTaskRanImmediately || thirdTaskEnqueuedLate);
 }
 
 /* ************************************************************************* */
diff --git a/gtsam/linear/MultifrontalClique.cpp b/gtsam/linear/MultifrontalClique.cpp
diff --git a/gtsam/linear/MultifrontalClique.h b/gtsam/linear/MultifrontalClique.h
