Extraction of Python installer into a separate package (#995)

## **Note: This PR needs to be merged after
[xmippCore-223](I2PC/xmippCore#223) and
[xmippViz-73](I2PC/xmippViz#73
The new installer is located in [this
repository](https://github.com/I2PC/xmipp3-installer).

## Changes
This greadtly simplifies the code complexity of the installer in this
repository, and allows it to evolve at different rates, being able to
fix issues or extend the installer capabilities without relying on
making releases of Xmipp.

With that change, comes a new paradigm on how source dependencies are
handled in xmipp (xmippViz/xmippCore/scipion-em-xmipp):
- scipion-em-xmipp will no longer be automatically installed by xmipp.
This is because the plugin depends on xmipp, not the other way around,
so the plugin must be the one installing xmipp (in production only).
- xmippCore & xmippViz are still installed by the installer of xmipp,
but now, instead of cloning specific fixed versions (for example,
`v3.24.0`), it will do so with a dynamically updated tag pointing to the
latest major release of such dependency (for example, `v3`, which match
the content of the latest `v3.X.Y` release).

## New release paradigm
This change enables skipping the release process in sources that don't
actually need it, and that it was happening until now because of a
simple update in the dependency number. Now that release cascade will
only be needed if a major release happens.

### Example of how it was until now (with any kind of release):
- `xmippCore`, `xmippViz`, `xmipp` and `scipion-em-xmipp` are all in
version `v3.24.0` initially.
- `xmippCore` creates release `v3.25.0`.
- `xmipp` needs to create release `v3.25.0` to update the version to be
downloaded of `xmippCore`, from `v3.24.0` to `v3.25.0`.
- `scipion-em-xmipp` needs to do the same thing to retrieve the latest
release of `xmipp`.

### With the new process (minor or patch release):
- `xmippCore`, `xmippViz`, `xmipp` and `scipion-em-xmipp` are all in
version `v3.24.0` initially.
- `xmippCore` creates release `v3.25.0`.
- Tag `v3` of `xmippCore` is updated to match the content of tag
`v3.25.0`.
- `xmipp` is pointing to release `v3`, so, whenever a new install is
triggered, it will already automatically download the newest version of
`xmippCore`, so no changes are required in `xmipp`'s side.
- `scipion-em-xmipp` also does not need to create a new release.

### With the new process (major release):
- `xmippCore`, `xmippViz`, `xmipp` and `scipion-em-xmipp` are all in
version `v3.24.0` initially.
- `xmippCore` creates release `v4.0.0`.
- Tag `v4 of `xmippCore` is created. It matches the content of tag
`v4.0.0`.
- `xmipp v3.24.0` is currently pointing at `xmippCore v3`, so, this
version will not be affected by the breaking changes introduced in the
`v4.0.0` release of `xmippCore`. To point to that new major release,
version `v3.24.1` can be created. This new version does not need to be a
new major (`v4.0.0`) since `xmipp` itself is not breaking any
compatibility towards its programs (in other words, `scipion-em-xmipp`
can still use the same `xmipp` programs without any code changes).
- When the new `v3.24.1` release is created, tag `v3` in `xmipp` will be
updated to match its content.
- `scipion-em-xmipp` does not need to implement any code changes, and,
therefore, no new release is needed.

## Conclusion
The improvements described above allow to generate more frequent
releases, with new functionality and bug fixes. As the pipeline process
of a release only takes ~5-10 minutes combined (adding up release times
of all xmipp sources & the plugin), the releases can be as frequent as
the development team wants, allowing users to experience almost
inmediately the newest develpments and patches, instead of needing to
wait ~6 months for each release.

---------

Co-authored-by: alberto <[email protected]>

Author: martin-s-a

.github/workflows/build.yml

@@ -25,6 +25,11 @@ jobs:
 
     runs-on: ${{ matrix.os }}
     steps:
+      - name: Extract branch name
+        shell: bash
+        run: echo "branch=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT
+        id: extract_branch
+
       - name: Install dependencies
         run: |
           sudo apt-get update
@@ -38,9 +43,11 @@ jobs:
       - uses: actions/setup-python@main
         with:
           python-version: '3.9.15'
-      - run: |
+      
+      - name: Install Python dependencies
+        run: |
           python -m pip install --upgrade pip
-          python -m pip install numpy
+          python -m pip install numpy xmipp3_installer
 
       - uses: Jimver/cuda-toolkit@master
         if: matrix.cuda.version != 'None'
@@ -53,14 +60,13 @@ jobs:
       - uses: actions/checkout@main
         with:
           fetch-depth: 0
-          ref: ${{ github.head_ref }}
+          ref: ${{ steps.extract_branch.outputs.branch }}
 
-      - name: Compile Xmipp with all dependencies in branch ${{ github.head_ref }} (if such branch exists, default is devel)
+      # Variable in the name is only evaluated once the workflow reaches this step
+      - name: Compile Xmipp with all dependencies in branch ${{ steps.extract_branch.outputs.branch }} (if such branch exists, default is devel)
         env:
-          BRANCH_NAME: ${{ github.head_ref }}
-          SEND_INSTALLATION_STATISTICS: 'OFF'
-        run: |
-          ./xmipp all -b $BRANCH_NAME --keep-output || (cat compilation.log && false)
+          BRANCH_NAME: ${{ steps.extract_branch.outputs.branch }}
+        run: echo "SEND_INSTALLATION_STATISTICS=OFF" > xmipp.conf && ./xmipp all -b $BRANCH_NAME --keep-output || (cat compilation.log && false)
 
       - name: Cat
         run: cat xmipp.conf

.github/workflows/release.yml

@@ -11,7 +11,13 @@ jobs:
     steps:
 
     - name: Checkout repository
-      uses: actions/checkout@main
+      uses: actions/checkout@v4
+
+    - name: Setup Python
+      uses: actions/setup-python@v5
+
+    - name: Install installer package
+      run: pip install xmipp3_installer
 
     - name: Retrieve tag name, release name & changelog
       run: |
@@ -28,9 +34,18 @@ jobs:
         github_token: ${{ secrets.GITHUB_TOKEN }}
         custom_tag: ${{ steps.variables.outputs.TAG_NAME }}
         tag_prefix: ''
+    
+    - name: Update major version tag
+      run: |
+        VERSION=${{ steps.variables.outputs.TAG_NAME }}
+        MAJOR=${VERSION%%.*}
+        git config --global user.name 'GitHub Actions'
+        git config --global user.email '[email protected]'
+        git tag -fa "${MAJOR}" -m 'Update major version tag with $VERSION'
+        git push origin "${MAJOR}" --force
         
     - name: Create a GitHub release
-      uses: ncipollo/release-action@main
+      uses: ncipollo/release-action@v1
       with:
         tag: ${{ steps.tag_version.outputs.new_tag }}
         name: ${{ steps.variables.outputs.RELEASE_NAME }}

.sonarcloud.properties

@@ -3,7 +3,7 @@ sonar.organization=i2pc
 
 # This is the name and version displayed in the SonarCloud UI.
 sonar.projectVersion=3.0
-sonar.sources=src/xmipp, scripts, installer
+sonar.sources=src/xmipp, scripts
 # Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
 
 # NOTE - be careful with excusion rules. If you include formerly excluded folder, all files that include