Curation checks and community uploads #1297
Description
Hello I have forked an instance of Zenodo from version 22.0.0. Most things seem to work well, but one thing that has been inconsistent is the configuration of curation checks, specifically metadata checks. This could be due to the tooling I am using to create these checks as such tooling wasn't provided in the application code (to my knowledge anyway). Below, I'll go through the symptoms I am experiencing as well as providing a snippet of the python script I wrote to insert my curation checks.
Edit: I have had some insights since I opened this issue and there is a chance that this is an issue with invenio-rdm-records on this ticket, but I will keep this issue open in case there is Zenodo specific configuration/logic that has something to do with this.
The Symptoms
- When I go to edit the record for an already submitted record:
- And I re-save as so with one of the curation checks purposely unfulfilled (in this case it is checking if the description contains WORD, so we exclude it as a test). This seems to intermittently be the case when I start a new record in that community and click the "Save Draft" button, I suspect because I might have to something from the form validation and the subsequent likewise can be treated as alternative step 2 to replicate this:
- I get this blank screen with just the header and footer, and the header of the community. There is nothing below it when normally the form would be there, or the tabbed interface that shows curation checks but in my case nothing shows once I click "Save Draft".
The browser console shows Error: Minified React error #31 which contains Objects are not valid as a React child (found: object with keys {message, severity, description}). If you meant to render a collection of children, use an array instead.
- When I make sure the condition is met and I re-save the draft, it proceeds without that error and shows the checks view as expected.
- Likewise, you can repeat this, even when the previous save attempt fulfilled the curation check conditions.
The Curation Code
Like I mentioned, I wrote my own custom code to insert my curation checks into my communities. This may provide some context for whether I may be doing something wrong, or if it is simply a matter of this version of Zenodo being incompatible with the invenio-checks module.
setup-curation-checks.py (I run this using invenio shell)
import sys
import time
from invenio_communities.communities.records.models import CommunityFeatured
from invenio_communities.communities.records.api import Community
from invenio_communities.proxies import current_communities
from invenio_checks.models import CheckConfig, Severity
from invenio_db import db
# For the rr team working on this, it seems invenio-checks uses this logic for metadata check levels that don't line up with the documentation
# Refer to https://github.com/inveniosoftware/invenio-checks/blob/master/invenio_checks/templates/semantic-ui/invenio_checks/requests/metadata_tab.html
TARGET_SLUGS = [
"astrophysics",
"biophys-science",
"earth-science",
"heliophysics",
"planetary-science"
]
METADATA_PARAMS = {
"title": "Required Metadata",
"description": "Metadata requirements.",
"rules": [
{
"id": "check-title",
"title": "Title Required",
"message": "Title is required.",
"description": "Title is required.",
"level": "error",
"checks": [
{
"type": "field",
"path": "metadata.title",
"operator": "exists"
}
]
},
{
"id": "require-description",
"title": "Description Missing",
"message": "A description is required for this record.",
"level": "error",
"checks": [
{
"type": "field",
"path": "metadata.description",
"operator": "exists"
}
]
},
{
"id": "check-in-description",
"title": "WORD in Description",
"message": "The word WORD is required in the description field for this record.",
"description": "The word WORD is required in the description field for this record.",
"level": "info",
"checks": [
{
"type": "comparison",
"left": {
"type": "field",
"path": "metadata.description"
},
"operator": "~=",
"right": "WORD"
}
]
}
]
}
def resolve_uuid(slug, max_retries=30, delay=5):
"""
Tries to resolve the community UUID.
If not found, waits 'delay' seconds and tries again.
"""
for attempt in range(1, max_retries + 1):
try:
community = Community.pid.resolve(slug)
return community.id
except Exception:
print(f" ⏳ Attempt {attempt}/{max_retries}: '{slug}' not ready yet... waiting {delay}s")
time.sleep(delay)
return None
def configure_community(slug):
print(f"\n--- PROCESSING: {slug} ---")
# 1. RESOLVE UUID
uuid = resolve_uuid(slug)
if not uuid:
print(f" ❌ TIMEOUT: Could not find '{slug}'. Skipping.")
return
print(f" ✅ Found UUID: {uuid}")
# 2. FEATURE COMMUNITY LOGIC
try:
is_featured = CommunityFeatured.query.filter_by(community_id=uuid).first()
if not is_featured:
featured_entry = CommunityFeatured(
community_id=uuid,
start_date=None
)
db.session.add(featured_entry)
print(" -> Community marked as Featured in DB.")
else:
print(" -> Already featured in DB. Skipping.")
except Exception as e:
print(f" ❌ ERROR during featuring: {e}")
# 3. APPLY CURATION CHECKS
try:
# Metadata Check
existing_meta = CheckConfig.query.filter_by(community_id=uuid, check_id="metadata").first()
if not existing_meta:
config_meta = CheckConfig(
community_id=uuid,
check_id="metadata",
params=METADATA_PARAMS,
severity=Severity.WARN,
enabled=True,
)
db.session.add(config_meta)
print(" -> Metadata check staged.")
else:
# UPDATE EXISTING
# We explicitly re-assign the params to ensure DB picks up changes to the JSON
existing_meta.params = METADATA_PARAMS
existing_meta.severity = Severity.WARN
existing_meta.enabled = True
print(" -> Metadata check updated.")
# File Formats Check
existing_files = CheckConfig.query.filter_by(community_id=uuid, check_id="file_formats").first()
if not existing_files:
config_files = CheckConfig(
community_id=uuid,
check_id="file_formats",
params={},
severity=Severity.WARN,
enabled=True,
)
db.session.add(config_files)
print(" -> File check staged.")
else:
# UPDATE EXISTING
existing_files.params = {}
existing_files.severity = Severity.WARN
existing_files.enabled = True
print(" -> File check updated.")
# Commit DB changes
db.session.commit()
# 4. TRIGGER SEARCH INDEX REFRESH
# Essential for the community to appear in the homepage slider
current_communities.service.indexer.index_by_id(uuid)
print(" -> Search index updated.")
except Exception as e:
db.session.rollback()
print(f" ❌ ERROR configuring curation for '{slug}': {e}")
def run():
print("Starting Community Configuration...")
for slug in TARGET_SLUGS:
configure_community(slug)
print("\n--- DONE ---")
if __name__ == "__main__":
run()