Skip to content

Update to be compatible with voc4cat-tool 1.0.0 (v1.0 formats) #107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dalito merged 1 commit into from
Dec 31, 2025
Merged

Update to be compatible with voc4cat-tool 1.0.0 (v1.0 formats) #107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 28 additions & 20 deletions .github/workflows/ci-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,25 @@ on:
# forks. Since the workflow only accesses the xlsx and turtle files, but does
# not run any code, it is safe to use pull_request_target. More info:
# https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/
# zizmor: ignore[dangerous-triggers] OK as per comment above
pull_request_target:
branches:
- main
types: [opened, reopened, synchronize]
workflow_dispatch:

env:
FORCE_COLOR: "1" # Make tool output pretty.
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_PROGRESS_BAR: "off"
LOGLEVEL: "DEBUG"
RUN_URL: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions: {}

jobs:
build_vocab:
name: Merge & build vocabulary
Expand All @@ -29,31 +35,33 @@ jobs:
timeout-minutes: 10

permissions:
# Give the default GITHUB_TOKEN write permission to commit to the PR branch.
contents: write
contents: write # Give the default GITHUB_TOKEN write permission to commit to the PR branch.

steps:
- name: Checkout PR branch into cwd
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
with:
# Checkout the fork/head-repository to push changes to the fork.
# Without this the base repository will be checked out and committed to.
repository: ${{ github.event.pull_request.head.repo.full_name }}

# Checkout the branch made in the fork.
ref: ${{ github.head_ref }}
# Be specific about keeping credentials (it is the default)
persist-credentials: true

- name: Checkout main branch into directory _main_branch/
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
with:
ref: main
path: _main_branch
# This sparse-checkout includes all toplevel files and everything below vocabularies/
sparse-checkout: |
vocabularies/
fetch-depth: 1
# Be specific about keeping credentials (it is the default)
persist-credentials: true

- uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548
with:
python-version: '3.13'

Expand All @@ -62,19 +70,16 @@ jobs:
python -VV
python -m pip install --upgrade pip setuptools wheel

# install tagged version
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v0.10.0
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main
# Select to install a stable tagged version or main branch (in development)
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v1.0.0
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main

# install custom pylode 2.x (adds sorted collections,
# uses URIs as fragment IDs and is git-installable)
python -m pip install git+https://github.com/dalito/pyLODE.git@nfdi4cat-2.x

# Example for installing from private repo:
# python -m pip install git+https://anon:${{ SECRETS.READ_PAT_VOC4CAT }}@github.com/dalito/voc4cat-tool.git@main

- name: Set dynamic environment variables.
run: |
run: | # zizmor: ignore[github-env] safe without user controllable input
echo "RUN_DATE=$(date +'%Y-%m-%dT%H%M')" >> $GITHUB_ENV

- name: Run voc4cat (pre-convert checks)
Expand All @@ -91,34 +96,37 @@ jobs:
voc4cat convert --config _main_branch/idranges.toml --logfile outbox/voc4cat.log --outdir outbox inbox-excel-vocabs/
if [ ! -f outbox/*.ttl ]; then
echo "No ttl file in outbox. Building joined vocabulary ttl-file from individual ttl-files in vocabulary."
voc4cat transform --join --logfile outbox/voc4cat.log --outdir outbox/ vocabularies/
voc4cat transform --join --config _main_branch/idranges.toml --logfile outbox/voc4cat.log --outdir outbox/ vocabularies/
fi

- name: Run voc4cat (post-convert checks)
run: |
# check all ttl file(s) in outbox
voc4cat check --config _main_branch/idranges.toml --logfile outbox/voc4cat.log outbox/
voc4cat check --redundant-hierarchies --config _main_branch/idranges.toml --logfile outbox/voc4cat.log outbox/
# check if vocabulary changes are allowed
voc4cat check --config _main_branch/idranges.toml --logfile outbox/voc4cat.log --ci-post _main_branch/vocabularies outbox/

- name: Run voc4cat (build HTML documentation)
run: |
voc4cat docs --logfile outbox/voc4cat.log --force outbox/

- name: Split vocabulary and merge into vocabulary dir (in PR branch)
- name: Split vocabulary, merge into vocabulary dir (in PR branch) and update provenance info
run: |
voc4cat transform --split --inplace --logfile outbox/voc4cat.log --outdir outbox_new_voc outbox/
voc4cat transform --split --inplace --config idranges.toml --logfile outbox/voc4cat.log --outdir outbox_new_voc outbox/
voc4cat-merge --logfile outbox/voc4cat.log outbox_new_voc/ vocabularies/
# copy xlsx to outbox so that the xlsx file is part of the artifact
find inbox-excel-vocabs -name '*.xlsx' -exec cp {} -t outbox/ \;
git status
# Add/update dct:created and dct:modified base on git history
voc4cat transform --prov-from-git --inplace --config idranges.toml --logfile outbox/voc4cat.log vocabularies/

- name: Run voc4cat (re-build updated Excel file and joined turtle vocabulary file)
# Passing the config is important to make use of the prefixes therein.
run: |
mkdir -p outbox/updated-xlsx-ttl
voc4cat convert --config idranges.toml --logfile outbox/voc4cat.log --template templates/voc4cat_template_043.xlsx outbox/updated-xlsx-ttl
voc4cat transform --config idranges.toml --logfile outbox/voc4cat.log -O outbox/updated-xlsx-ttl --join vocabularies/
voc4cat transform --join --config idranges.toml --logfile outbox/voc4cat.log --outdir outbox/updated-xlsx-ttl vocabularies/
# Using a template is optional and gives just a warning if missing.
voc4cat convert --config idranges.toml --logfile outbox/voc4cat.log --template templates/default_sheets.xlsx outbox/updated-xlsx-ttl

- name: Store artifacts
if: ${{ always() }}
Expand Down
31 changes: 20 additions & 11 deletions .github/workflows/merge.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,33 +21,41 @@ on:
workflow_dispatch:

env:
FORCE_COLOR: "1" # Make tool output pretty.
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_PROGRESS_BAR: "off"
LOGLEVEL: "DEBUG"

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions: {}

jobs:
build:
name: Development build of vocabulary & documentation
permissions:
# Required for peaceiris/actions-gh-pages below
contents: write
contents: write # Required for peaceiris/actions-gh-pages below
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
with:
# Using fetch-depth 0 is the only way to get all tags which are needed for building docs.
fetch-depth: 0
# Be specific about keeping credentials (it is the default)
persist-credentials: true

- name: Checkout gh-pages branch to dir publish/
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
with:
ref: gh-pages
path: publish
fetch-depth: 1
# Be specific about keeping credentials (it is the default)
persist-credentials: true

- uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c
- uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548
with:
python-version: '3.13'

Expand All @@ -56,9 +64,9 @@ jobs:
python -VV
python -m pip install --upgrade pip setuptools wheel

# install tagged version
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v0.10.0
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main
# Select to install a stable tagged version or main branch (in development)
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v1.0.0
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main

# install custom pylode 2.x
python -m pip install git+https://github.com/dalito/pyLODE.git@nfdi4cat-2.x
Expand All @@ -77,7 +85,7 @@ jobs:
find publish/dev/ -type f \( -name "*.xlsx" -o -name "*.ttl" \) -delete
cp -r vocabularies/. publish/dev/
# Build joined turtle
voc4cat transform --logfile publish/dev/voc4cat.log --join publish/dev/
voc4cat transform --join --config idranges.toml --logfile publish/dev/voc4cat.log publish/dev/

- name: Run voc4cat (build pyLODE HTML documentation)
run: |
Expand All @@ -88,7 +96,8 @@ jobs:
- name: Run voc4cat (build current Excel file)
# Passing the config is important to make use of the prefixes therein.
run: |
voc4cat convert --config idranges.toml --logfile publish/dev/voc4cat.log --template templates/voc4cat_template_043.xlsx publish/dev/
# Using a template is optional and gives just a warning if missing.
voc4cat convert --config idranges.toml --logfile publish/dev/voc4cat.log --template templates/default_sheets.xlsx publish/dev/

- name: Deploy updated gh-pages content
# This replaces all prior content in gh-pages branch. But we have
Expand Down
178 changes: 178 additions & 0 deletions .github/workflows/pr-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
# This workflow checks for common PR submission issues and provides helpful feedback
# - PRs from fork's main branch (helpful workflow suggestion)
# - Organization account submissions (critical - blocks CI due to GitHub limitation)
#
# Security Note: This workflow uses pull_request_target which runs in the context
# of the base repository, not the fork. User-controlled data (PR titles, descriptions,
# branch names, usernames) must be sanitized before use to prevent script injection.

name: PR Submission Checks

on:
pull_request_target:
types: [opened, reopened, synchronize]
branches:
- main

permissions: {}

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
check-pr-submission:
name: Check PR submission best practices
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # to post a message to PR

steps:
- name: Check if PR is from main branch of fork
id: check-main-branch
uses: actions/github-script@v8
with:
script: |
const pr = context.payload.pull_request;

// Get PR details from context (these are all user-controlled)
const headRef = pr.head.ref;
const headRepo = pr.head.repo.full_name;
const baseRepo = pr.base.repo.full_name;
const isFork = headRepo !== baseRepo;
const isFromMain = headRef === 'main';
const isFromOrg = pr.head.repo.owner.type === 'Organization';
const headOwnerLogin = pr.head.repo.owner.login;
const baseOwnerLogin = pr.base.repo.owner.login;

// Log details - no user input in template literals for security
console.log('PR number:', pr.number);
console.log('Head branch:', headRef);
console.log('Head repo:', headRepo);
console.log('Base repo:', baseRepo);
console.log('Is fork:', isFork);
console.log('From main branch:', isFromMain);
console.log('Owner type:', pr.head.repo.owner.type);
console.log('Head owner login:', headOwnerLogin);
console.log('Base owner login:', baseOwnerLogin);

core.setOutput('is_fork', isFork);
core.setOutput('is_from_main', isFromMain);
core.setOutput('is_from_org', isFromOrg);
core.setOutput('head_ref', headRef);
core.setOutput('head_repo', headRepo);
core.setOutput('head_owner_login', headOwnerLogin);
core.setOutput('base_owner_login', baseOwnerLogin);

return {
isFork,
isFromMain,
isFromOrg,
needsComment: isFork && isFromMain
};

- name: Post comment about main branch submission
if: steps.check-main-branch.outputs.is_fork == 'true' && steps.check-main-branch.outputs.is_from_main == 'true'
uses: actions/github-script@v8
with:
script: |
const pr = context.payload.pull_request;

// Check if we already posted this comment
const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number
});

const botComment = comments.data.find(comment =>
comment.user.type === 'Bot' &&
comment.body.includes('submitted from the main branch')
);

if (botComment) {
console.log('Comment about main branch already exists, skipping');
return;
}

// Sanitize username - GitHub usernames can only contain alphanumeric characters and hyphens
const username = pr.user.login.replace(/[^a-zA-Z0-9-]/g, '');

// Post helpful comment
const commentBody = `Hi @${username}! 👋

Thank you for your contribution to voc4cat!

> [!WARNING]
> We noticed that this pull request was submitted from the \`main\` branch of your fork.
> While this works, it can cause issues:

- It makes it harder to keep your fork updated with upstream changes
- You won't be able to work on multiple PRs at once
- Future contributions may be complicated by merge conflicts

This PR can still be merged, but please use feature branches going forward!

For more information, see our [How to contribute](https://nfdi4cat.github.io/voc4cat/docs_usage/how-to-contribute.html) guide.

*If you have any questions, please don't hesitate to ask!* 🚀`;

await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: commentBody
});

- name: Post info about organization account
if: steps.check-main-branch.outputs.is_from_org == 'true' && steps.check-main-branch.outputs.is_fork == 'true' && steps.check-main-branch.outputs.head_owner_login != steps.check-main-branch.outputs.base_owner_login
uses: actions/github-script@v8
with:
script: |
const pr = context.payload.pull_request;

// Check if we already posted this comment
const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number
});

const botComment = comments.data.find(comment =>
comment.user.type === 'Bot' &&
comment.body.includes('organization account')
);

if (botComment) {
console.log('Comment about organization account already exists, skipping');
return;
}

// Sanitize username - GitHub usernames can only contain alphanumeric characters and hyphens
const username = pr.user.login.replace(/[^a-zA-Z0-9-]/g, '');

const commentBody = `Hi @${username}! 👋

> [!CAUTION]
> We noticed that this pull request comes from a fork under an organization account,
> which will prevent our CI workflow from working correctly.

GitHub does not allow the "Allow edits from maintainers" option for forks in organizations (see [discussion](https://github.com/orgs/community/discussions/5634)).
Our CI needs this permission to commit turtle files and clean up Excel files.

**This PR cannot be merged as-is.** Please:
1. Fork voc4cat to your personal GitHub account
2. Create a feature branch with your changes
3. Submit a new PR from your personal fork
4. Close this PR

Sorry for the inconvenience - this is a GitHub limitation, not our choice!`;

await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: commentBody
});

Loading
Loading