Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: trigger doc build on release tag #2301

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
1dc8c52
feature: trigger doc build on release-test tag
yuejiaointel Feb 6, 2025
ee953a5
fix: trigger by detect number.number format
yuejiaointel Feb 7, 2025
33710a5
feature: ci-test: get correct version and checkout to tag commit
yuejiaointel Feb 7, 2025
73c40e0
fci-test: export doc version as env and use in conf.py
yuejiaointel Feb 7, 2025
f0ca811
ci-test: print
yuejiaointel Feb 7, 2025
8452ecc
ci-test: print
yuejiaointel Feb 7, 2025
a874c91
ci-test: print
yuejiaointel Feb 7, 2025
6769c6f
ci-test: print
yuejiaointel Feb 7, 2025
bf977b0
ci-test: print
yuejiaointel Feb 7, 2025
f0934d5
ci-test: print
yuejiaointel Feb 7, 2025
d27c18f
ci-test: print
yuejiaointel Feb 7, 2025
8cd871d
ci-test: print
yuejiaointel Feb 7, 2025
c0f7dd4
ci-test: print
yuejiaointel Feb 7, 2025
7cdb5c8
ci-test: print
yuejiaointel Feb 7, 2025
2989c14
ci-test: print
yuejiaointel Feb 7, 2025
c3e3bc0
ci-test: print
yuejiaointel Feb 7, 2025
ed36ec8
ci-test: print
yuejiaointel Feb 7, 2025
3717eb3
ci-test: print
yuejiaointel Feb 7, 2025
4637d83
ci-test: print
yuejiaointel Feb 7, 2025
fe1fe57
fix: don't use pip to get newest version as fallaback
yuejiaointel Feb 7, 2025
71572e9
ci-test: print
yuejiaointel Feb 7, 2025
958f7ae
ci-test: print
yuejiaointel Feb 7, 2025
3d4b7c1
ci-test: print
yuejiaointel Feb 7, 2025
1cf77af
ci-test: print
yuejiaointel Feb 7, 2025
b3d906b
ci-test: print and tagged
yuejiaointel Feb 7, 2025
334782b
ci-test: print
yuejiaointel Feb 7, 2025
2e271ba
ci-test: print
yuejiaointel Feb 7, 2025
e009d6b
ci-test: print and tagged
yuejiaointel Feb 7, 2025
42a7cf2
fix: remove running ci on pr
yuejiaointel Feb 7, 2025
55814c3
ci-test: pritn
yuejiaointel Feb 7, 2025
b6e7017
ci-test: anohte way to trigger on tag
yuejiaointel Feb 8, 2025
6cc5f1b
ci-test: get pr seciton back
yuejiaointel Feb 8, 2025
d059818
ci-test: test
yuejiaointel Feb 8, 2025
e66e7d0
ci-test:print
yuejiaointel Feb 8, 2025
7bdf509
ci-test: try gitthub action for tag trigger
yuejiaointel Feb 10, 2025
468dabe
feature: add several steps needed
yuejiaointel Feb 11, 2025
3086f0f
fix: fix verison order
yuejiaointel Feb 11, 2025
f059967
fix: add env var to github env for other files
yuejiaointel Feb 11, 2025
bbc9f0f
ci-test: print
yuejiaointel Feb 11, 2025
36408eb
ci-test: print
yuejiaointel Feb 11, 2025
619f28b
ci-test:print
yuejiaointel Feb 11, 2025
e5f5f76
feature: rest of steps until push to ghpages
yuejiaointel Feb 11, 2025
23a5ed3
fix: revert docs.yml azure
yuejiaointel Feb 12, 2025
2cb0892
ci-test: commit test on gh page
yuejiaointel Feb 12, 2025
fcf3d2c
fix: creat fodler
yuejiaointel Feb 12, 2025
beb61dd
fix: add git config
yuejiaointel Feb 12, 2025
bf709bc
fix: try actions/deploy-pages on fork
yuejiaointel Feb 12, 2025
dae0aef
fix: need to use gh-page b/c we need to maintain the versions
yuejiaointel Feb 12, 2025
8324337
ci-test: test permissions
yuejiaointel Feb 12, 2025
91f24bb
fix: in case gh-pages does not exist create it
yuejiaointel Feb 12, 2025
ce941e6
ci-test: permission test
yuejiaointel Feb 13, 2025
1886f2e
fix: change latest to version before delete it
yuejiaointel Feb 13, 2025
dd2c91a
fix: fix tag trigger regex
yuejiaointel Feb 13, 2025
0e5152a
fix: fix tag trigger regex
yuejiaointel Feb 13, 2025
0520d26
fix: add old latest version to git
yuejiaointel Feb 13, 2025
c7783c2
fix: should have 2 same folder for new version
yuejiaointel Feb 13, 2025
0706bd5
fix: remove extra depen and escape . in tag trigger
yuejiaointel Feb 13, 2025
709f425
fix: remove versions.json generate it
yuejiaointel Feb 14, 2025
b41003d
fix: minor fix to add env var
yuejiaointel Feb 14, 2025
1563665
fix: make sure delete finish before cp
yuejiaointel Feb 14, 2025
ab1c9fd
fix: cp file
yuejiaointel Feb 14, 2025
e489519
fix: overwrite latest do not delete first
yuejiaointel Feb 14, 2025
d062b68
fix: try rsync instead cp
yuejiaointel Feb 14, 2025
01c6612
fix: cp also links
yuejiaointel Feb 14, 2025
a1a1623
ci-test:print
yuejiaointel Feb 14, 2025
c70d592
fix: add latest folder in case any new files in folder
yuejiaointel Feb 14, 2025
e0a1e99
fix: add switcher url back and other small fix
yuejiaointel Feb 14, 2025
5ff8ac8
ci-test: add all for latest and verison folder
yuejiaointel Feb 14, 2025
13d531f
ci-test: verison.json for swietcher url
yuejiaointel Feb 14, 2025
87235c1
ci-test: test layout.html
yuejiaointel Feb 14, 2025
a4fc3b1
fix: comment style fix
yuejiaointel Feb 14, 2025
1f793ea
ci-test: revert layout html
yuejiaointel Feb 15, 2025
2dc2a88
fix: revers order for versons
yuejiaointel Feb 21, 2025
0aa23eb
fix: add versions so version switcher could work locally
yuejiaointel Feb 25, 2025
4e0a3b9
fix: license check avoid verisons.json
yuejiaointel Feb 25, 2025
113bed7
fix: change versions to latest for local only versions.json
yuejiaointel Feb 25, 2025
c533711
fix: fix sort and mover versions
yuejiaointel Feb 25, 2025
ffa3733
fix: add versions to doc folder
yuejiaointel Feb 25, 2025
3198542
fix: put versions inside doc
yuejiaointel Feb 25, 2025
3cdb5f6
fix: add doc/versions.json to git
yuejiaointel Feb 25, 2025
d781667
fix: license check fix
yuejiaointel Feb 25, 2025
68b621d
ci-test to get rid of git config and change error message
yuejiaointel Feb 27, 2025
91e54d2
fix: revert git config
yuejiaointel Feb 27, 2025
4ad3fd1
ci-test: add new doc_release.sh for last step of doc_release.yml
yuejiaointel Feb 27, 2025
7fd43ba
ci-test: move check out scritp back
yuejiaointel Feb 27, 2025
99182a5
fix: cp doc_release.sh to a temp loc before swich branch
yuejiaointel Feb 28, 2025
3623e34
fix: move git config back to yml
yuejiaointel Mar 1, 2025
c82b91a
fix: trigger docs.yml when docs-relase.yml changes
yuejiaointel Mar 1, 2025
f706a82
fix: endline to doc_release
yuejiaointel Mar 5, 2025
e3dfaf6
ci-test: run again
yuejiaointel Mar 5, 2025
6c9d22a
ci-test: run again
yuejiaointel Mar 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .ci/pipeline/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ trigger:
- doc/
- .ci/pipeline/docs.yml
- .github/Pull_Request_template.md
- .github/workflows/docs-release.yml

pr:
branches:
Expand Down
71 changes: 71 additions & 0 deletions .github/scripts/doc_release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/bin/bash

#===============================================================================
# Copyright 2024 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================

# Ensure the build directory exists
BUILD_DIR="doc/_build/scikit-learn-intelex"
if [ ! -d "$BUILD_DIR" ]; then
echo "::error: Documentation build directory not found!"
exit 1
fi

# Copy built documentation to a temp location
DEPLOY_DIR="/tmp/gh-pages-deploy"
mkdir -p "$DEPLOY_DIR"
cp -R "$BUILD_DIR"/* "$DEPLOY_DIR"
ls $DEPLOY_DIR

# Checkout gh-pages branch
if ! git checkout gh-pages; then
echo "::error:: Could not checkout gh-pages branch!"
exit 1
fi

# Move the new versioned folder to the correct location
rm -Rf latest
cp -R "$DEPLOY_DIR/$SHORT_DOC_VERSION" "$SHORT_DOC_VERSION"
cp -R "$DEPLOY_DIR/$SHORT_DOC_VERSION" latest
cp "$DEPLOY_DIR/index.html" .
if ! diff -r "$SHORT_DOC_VERSION" latest > /dev/null; then
echo "::error: Content mismatch between $SHORT_DOC_VERSION and latest directories"
echo "Differences found:"
diff -r "$SHORT_DOC_VERSION" latest
exit 1
fi

# Generate versions.json by scanning for year.month folders
rm -f doc/versions.json
mkdir -p doc
echo "[" > doc/versions.json
# Add latest entry first
echo ' {"name": "latest", "version": "'$SHORT_DOC_VERSION'", "url": "/scikit-learn-intelex/latest/"},' >> doc/versions.json
# Add all year.month folders
for version in $(ls -d [0-9][0-9][0-9][0-9].[0-9]* 2>/dev/null || true); do
echo ' {"name": "'$version'", "version": "'$version'", "url": "/scikit-learn-intelex/'$version'/"},'
done | sort -rV >> doc/versions.json
# Remove trailing comma and close array
sed -i '$ s/,$//' doc/versions.json
echo "]" >> doc/versions.json
cat doc/versions.json

# Commit and push changes
git add -A "$SHORT_DOC_VERSION"
git add -A latest
git add doc/versions.json
git add index.html
git commit . -m "Automatic doc update for version $DOC_VERSION"
git push origin gh-pages
93 changes: 93 additions & 0 deletions .github/workflows/docs-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#===============================================================================
# Copyright 2024 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
name: Docs Release

on:
push:
tags:
- '[0-9][0-9][0-9][0-9]\.[0-9][0-9]?\.[0-9]' # Trigger on tag pushes

permissions:
contents: write

jobs:
build-docs:
runs-on: ubuntu-24.04

steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # Ensures all tags are fetched

- name: Set Up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"

- name: Install System Dependencies
run: sudo apt-get update && sudo apt-get install -y clang-format pandoc

- name: Extract Tag Version
run: |
export DOC_VERSION="${GITHUB_REF#refs/tags/}"
# Error out if cannot find version
if [ -z "$DOC_VERSION" ]; then
echo "::error: Failed to determine documentation version."
exit 1
fi
export SHORT_DOC_VERSION=$(echo "$DOC_VERSION" | awk -F'.' '{print $1"."$2}')
# export env var in other files
echo "DOC_VERSION=$DOC_VERSION" >> $GITHUB_ENV
echo "SHORT_DOC_VERSION=$SHORT_DOC_VERSION" >> $GITHUB_ENV
- name: Checkout release branch
run: |
if git checkout $DOC_VERSION 2>/dev/null; then
echo "Successfully checked out tag $DOC_VERSION."
else
echo "::error:: Tag $DOC_VERSION does not exist."
exit 1
fi
git branch
- name: Install Python Dependencies
run: |
pip install daal-devel impi-devel
pip install -r dependencies-dev
pip install -r requirements-doc.txt
- name: Build daal4py/sklearnex
run: |
export DALROOT=$(dirname $(dirname $(which python)))
export LD_LIBRARY_PATH=$(dirname $(dirname $(which python)))/lib:$LD_LIBRARY_PATH
./conda-recipe/build.sh
- name: Build scikit-learn-intelex Documentation
run: |
export LD_LIBRARY_PATH=$(dirname $(dirname $(which python)))/lib:$LD_LIBRARY_PATH
cd doc
./build-doc.sh --gh-pages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: divergence from docs CI by having the --gh-pages flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's because they do slightly different things. This one with --gh-pages uses different build options and generates a versioned folder - simply reusing the result from the docs CI and renaming it results in docs that don't render when put under gh-pages.

But that shouldn't be a problem because in the end both of them are using the same software to build the docs, with the same pinned versions from requirements-doc.txt.

- name: Deploy Documentation to gh-pages
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This run call is 57 lines long, ideally it should be a separate shell script located in .github/scripts/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved now I have a doc_release.sh in scripts

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the impression that the scripts were meant to be executable both locally and on GH.

This script would mostly be about git commands which are specific to this particular workflow and not reusable elsewhere. I think it makes sense to keep it here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Alexsandruss @homksei please weigh in on this, as we are both inferring about the rules of the codebase when it comes to this topic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@david-cortes-intel As far as I see, the script is a general deployment script which the only git commands which are specific to github is the setting the username and email, which could be maintained in the workflow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that sounds like a reasonable split.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx for the suggestions! Moved git config back to yml

# Copy the script to a temp location
# Otherwise it will get lost after switch branch
cp .github/scripts/doc_release.sh /tmp/doc_release.sh

# Set git auth for push changes
git config --global user.name "GitHub Actions"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this number come from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I searched git config and there is a .cache/per-commit/repondmp4e6s/workflows/main.yml so I tried to jsut copy it over, need this config to push the changes


chmod +x /tmp/doc_release.sh
/tmp/doc_release.sh
1 change: 0 additions & 1 deletion doc/build-doc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ if [[ "$*" == *"--gh-pages"* ]]; then
export SOURCEDIR=sources

sphinx-build -b html $SOURCEDIR $BUILDDIR/$SPHINXPROJ/$DOC_VERSION
cp versions.json $BUILDDIR/$SPHINXPROJ
echo "<meta http-equiv=\"refresh\" content=\"0; URL='/$SPHINXPROJ/$DOC_VERSION/'\" / >" >> $BUILDDIR/$SPHINXPROJ/index.html
else
make html
Expand Down
40 changes: 20 additions & 20 deletions doc/sources/_static/version_switcher.js
Original file line number Diff line number Diff line change
@@ -1,43 +1,43 @@
# Copyright contributors to the oneDAL project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
function load_versions(json){
// Copyright contributors to the oneDAL project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
function load_versions(json) {
var button = document.getElementById('version-switcher-button')
var container = document.getElementById('version-switcher-dropdown')
var loc = window.location.href;
var s = document.createElement('select');
s.style = "border-radius:5px;"
const versions = JSON.parse(json);
for (entry of versions){
for (entry of versions) {
var o = document.createElement('option');
var optionText = '';
if ('name' in entry){
if ('name' in entry) {
optionText = entry.name;
}else{
} else {
optionText = entry.version;
}
o.value = entry.url;
if (current_version == entry.version){
if (current_version == entry.version) {
o.selected = true;
}
o.innerHTML = optionText;
s.append(o);
}
s.addEventListener("change", ()=> {
s.addEventListener("change", () => {
var current_url = new URL(window.location.href);
var path = current_url.pathname;
//strip version from path
var page_path = path.substring(project_name.length+current_version.length+3);
var page_path = path.substring(project_name.length + current_version.length + 3);
window.location.href = s.value + page_path;
});
container.append(s);
Expand Down
36 changes: 19 additions & 17 deletions doc/sources/_templates/layout.html
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
{% extends "!layout.html" %}
{% block extrahead %}
<script defer type="text/javascript" src="https://www.intel.com/content/dam/www/global/wap/performance-config.js" ></script>
<script type="text/javascript">
fetch("{{ switcher_url }}").then(response => response.text()).then(respText=> load_versions(respText));
// Configure TMS settings
var wapLocalCode = 'us-en'; // Dynamically set per localized site, see mapping table for values
var wapSection = "scikit-learn"; // WAP team will give you a unique section for your site
// Load TMS
if(document.location.href.includes("uxlfoundation.github.io/scikit-learn-intelex")){
(function () {
var url = 'https://www.intel.com/content/dam/www/global/wap/tms-loader.js'; // WAP file URL
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = url;
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
}
</script>
<script defer type="text/javascript"
src="https://www.intel.com/content/dam/www/global/wap/performance-config.js"></script>
<script type="text/javascript">
fetch("{{ switcher_url }}").then(response => response.text()).then(respText => load_versions(respText));
// Configure TMS settings
var wapLocalCode = 'us-en'; // Dynamically set per localized site, see mapping table for values
var wapSection = "scikit-learn"; // WAP team will give you a unique section for your site
// Load TMS
if (document.location.href.includes("uxlfoundation.github.io/scikit-learn-intelex")) {
(function () {
var url = 'https://www.intel.com/content/dam/www/global/wap/tms-loader.js'; // WAP file URL
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = url;
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
}
</script>
{% endblock %}

{% block menu %}
{% include "versions.html" %}
{{ super() }}
{% include "versions.html" %}
{{ super() }}
{% endblock %}

11 changes: 4 additions & 7 deletions doc/sources/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
import os
import sys

# sys.path.insert(0, os.path.abspath("../"))
Expand All @@ -44,10 +44,8 @@
# The short X.Y version
# Note: it should not have more than two parts (year.month), otherwise the
# version switcher will not be able to pick it.
version = "2025.2"
# The full version, including alpha/beta/rc tags
release = "2025.2"

version = os.environ.get("SHORT_DOC_VERSION", "latest")
release = version

# -- General configuration ---------------------------------------------------

Expand Down Expand Up @@ -155,12 +153,11 @@
"titles_only": False,
}

switcher_url = "/scikit-learn-intelex/versions.json"

html_context = {
"current_version": version,
"project_name": "scikit-learn-intelex",
"switcher_url": switcher_url,
"switcher_url": "/scikit-learn-intelex/doc/versions.json",
}


Expand Down
34 changes: 2 additions & 32 deletions doc/versions.json
Original file line number Diff line number Diff line change
@@ -1,38 +1,8 @@
[
{
"name": "latest",
"version": "2025.2",
"version": "latest",
"url": "/scikit-learn-intelex/latest/"
},
{
"name": "2025.2",
"version": "2025.2",
"url": "/scikit-learn-intelex/2025.2/"
},
{
"name": "2025.1",
"version": "2025.1",
"url": "/scikit-learn-intelex/2025.1/"
},
{
"name": "2025.0",
"version": "2025.0",
"url": "/scikit-learn-intelex/2025.0/"
},
{
"version": "2024.7",
"url": "/scikit-learn-intelex/2024.7/"
},
{
"version": "2024.6",
"url": "/scikit-learn-intelex/2024.6/"
},
{
"version": "2024.3",
"url": "/scikit-learn-intelex/2024.3/"
},
{
"version": "2023.2",
"url": "/scikit-learn-intelex/2023.2/"
}
]

Loading