feat: add CORE conference and journal backends (#1026)

* feat: add CORE conference and journal backends [AI-assisted]

Introduce CORE/ICORE conference and CORE journal integrations so ranked venues
can be used as first-class legitimate-list evidence in assessments.

This adds paginated portal sources, cached backends, config defaults,
and focused unit tests to keep behavior reliable as portal data evolves.

* fix: use urllib for CORE source fetching [AI-assisted]

Replace aiohttp/curl-specific fetch path with urllib-based page retrieval
for CORE conference and journal sources.

This avoids environment-specific aiohttp timeout behavior and keeps
cross-platform compatibility while preserving retry/backoff logic.

Also add a small connectivity diagnostics script for CORE endpoints.

* docs: add CORE integration documentation [AI-assisted]

Document CORE conference and journal integrations in README and
integration notes, including backend coverage, usage, and limitations.

* fix: resolve CORE lint and typing checks [AI-assisted]

Add SPDX header to CORE requests diagnostic script and make urllib read type explicit so mypy no-any-return passes consistently.

* fix: harden CORE fetch against unsafe URL schemes [AI-assisted]

Replace urllib urlopen with explicit HTTPSConnection requests and enforce https/host validation to satisfy Bandit B310 while keeping CORE sync behavior unchanged.

* docs: keep README backend docs generic [AI-assisted]

Remove the CORE-specific link from the main README and keep only the shared backend integration documentation entry. Also remove the temporary CORE connectivity test script no longer needed.

---------

Co-authored-by: florath-ai-assistant[bot] <Andreas.Florath@telekom.de>

Author: coding-ai-assistant[bot]

README.md

@@ -87,10 +87,12 @@ This tool acts as a **data aggregator** - it doesn't provide data itself, but co
 - **Kscien Hijacked Journals** - Legitimate journals that have been hijacked by predatory actors
 - **Kscien Predatory Conferences** - Database of predatory conferences
 - **DBLP Venues** - Curated computer science journals and conference series from DBLP XML
+- **CORE Conferences (ICORE/CORE)** - Ranked conference venues from the CORE portal
+- **CORE Journals (legacy)** - Ranked journals from the CORE journal portal (latest list: CORE2020)
 
 The tool analyzes publication patterns, citation metrics, and metadata quality to provide comprehensive coverage beyond traditional blacklist/whitelist approaches.
 
-**Note on Conference Assessment**: Conference checking now combines curated predatory conference signals (Kscien) with curated legitimacy signals from DBLP venue series. Coverage is stronger for computer science venues than for other domains.
+**Note on Conference Assessment**: Conference checking now combines curated predatory conference signals (Kscien) with curated legitimacy signals from DBLP and CORE/ICORE ranked venues. Coverage is strongest for computer science venues.
 
 ## Quick Start
 
@@ -122,6 +124,8 @@ These provide authoritative yes/no decisions for journals they cover:
 | **Kscien Hijacked Journals** | Hijacked journals | ~200 entries | Legitimate journals compromised by predatory actors |
 | **Kscien Predatory Conferences** | Predatory conferences | ~450 entries | Identified predatory conference venues |
 | **DBLP Venues** | Legitimate venues (CS) | dump-derived | Curated DBLP journals and conference series from local XML cache |
+| **CORE Conferences** | Legitimate ranked conferences | ~825 entries (ICORE2026 ranked) | CORE/ICORE conference rankings portal |
+| **CORE Journals (legacy)** | Legitimate ranked journals | ~582 entries (CORE2020 ranked) | CORE journal rankings portal (discontinued, no post-2020 updates) |
 | **Retraction Watch** | Quality indicator | ~27,000 journals | Retraction rates and patterns for quality assessment |
 | **Institutional Lists** | Custom whitelist/blacklist | Organization-specific | Local policy enforcement |
 
@@ -148,6 +152,7 @@ Journal Query → [Curated Databases + Pattern Analyzers] → Combined Assessmen
                  ├─ UGC-CARE discontinued lists
                  ├─ PredatoryJournals.org
                  ├─ Kscien databases
+                 ├─ CORE Conferences / CORE Journals
                  ├─ Retraction Watch (quality)
                  ├─ OpenAlex Analyzer (patterns)
                  ├─ Crossref Analyzer (metadata)
@@ -271,6 +276,7 @@ To enhance coverage with Scopus data:
   - [Backend API](docs/api-reference/backends.md) - Creating custom backends
   - [Data Models](docs/api-reference/models.md) - Core data structures
   - [Extending Guide](docs/api-reference/extending-guide.md) - Extension patterns
+- [Backend Integration Docs](dev-notes/integration/README.md) - Source-specific backend documentation
 - [Contributing Guide](.github/community/CONTRIBUTING.md) - Development setup and guidelines
 - [Coding Standards](dev-notes/CODING_STANDARDS.md) - Code quality requirements
 

dev-notes/integration/README.md

@@ -70,6 +70,7 @@ Each integration document should include:
 - **[DEPENDENCIES.md](../DEPENDENCIES.md)** - System and Python dependencies
 - **[CODING_STANDARDS.md](../CODING_STANDARDS.md)** - Code style and patterns
 - **[LOGGING_USAGE.md](../LOGGING_USAGE.md)** - Logging conventions
+- **[core.md](./core.md)** - CORE/ICORE conference and journal integration
 
 ## See Also
 

dev-notes/integration/core.md

@@ -0,0 +1,94 @@
+# CORE Integration
+
+## Overview
+
+The CORE integration adds ranked conference and journal venue signals from the CORE/ICORE portal to assessment decisions.
+
+- **Conference source**: ICORE/CORE conference rankings portal
+- **Journal source**: CORE journal rankings portal (legacy/discontinued dataset)
+- **Classification**: Legitimate-list evidence for ranked entries
+
+## Data Sources
+
+### Conferences
+- **Portal**: `https://portal.core.edu.au/conf-ranks/`
+- **Default source filter**: `ICORE2026`
+- **Format**: HTML (paginated table)
+- **Fields used**: title, acronym, source, rank
+
+### Journals
+- **Portal**: `https://portal.core.edu.au/jnl-ranks/`
+- **Default source filter**: `CORE2020`
+- **Format**: HTML (paginated table)
+- **Fields used**: title, source, rank
+
+## Architecture
+
+### Data Source Components
+- **Module**: `src/aletheia_probe/updater/sources/core.py`
+- **Classes**:
+  - `CoreConferenceSource`
+  - `CoreJournalSource`
+- **Base class**: `DataSource`
+- **Update cadence**: Monthly (`30` days)
+
+### Backend Components
+- **Conference backend**: `src/aletheia_probe/backends/core_conferences.py`
+- **Journal backend**: `src/aletheia_probe/backends/core_journals.py`
+- **Base class**: `CachedBackend`
+- **Evidence type**: `LEGITIMATE_LIST`
+- **Cache TTL**: `24 * 30` hours
+
+### Configuration
+Defined in `src/aletheia_probe/config.py` (`DataSourceUrlConfig`):
+- `core_conference_rankings_url`
+- `core_journal_rankings_url`
+- `core_conference_default_source`
+- `core_journal_default_source`
+
+## Data Processing Rules
+
+1. Fetch paginated portal pages (`50` rows per page).
+2. Parse table rows into normalized venue entries.
+3. Keep only ranked entries with these rank labels:
+   - `A*`, `A`, `B`, `C`, `Australasian B`, `Australasian C`
+4. Exclude non-ranked/non-classification statuses (for example `Unranked`, `National`, `Journal Published`, `Not ranked`, `not primarily CS`).
+5. Deduplicate by normalized name before writing to cache.
+
+## Metadata Stored
+
+Each CORE entry stores source metadata in `metadata`:
+- `source_url`
+- `core_entity_type` (`conference` or `journal`)
+- `core_source` (for example `ICORE2026`, `CORE2020`)
+- `core_rank`
+- `core_acronym` (conference source only)
+
+## Usage
+
+```bash
+# Sync only CORE conferences
+aletheia-probe sync core_conferences
+
+# Sync only CORE journals
+aletheia-probe sync core_journals
+
+# Normal sync includes CORE backends when enabled
+aletheia-probe sync
+```
+
+## Limitations
+
+1. CORE journal rankings are legacy/discontinued and should be interpreted accordingly.
+2. Parsing depends on portal table structure and rank label conventions.
+3. Matching is normalized name based; source-side aliases/variants may still miss edge cases.
+
+## References
+
+- `src/aletheia_probe/updater/sources/core.py`
+- `src/aletheia_probe/backends/core_conferences.py`
+- `src/aletheia_probe/backends/core_journals.py`
+- `tests/unit/updater/test_core_source.py`
+- `tests/unit/backends/test_core_backends.py`
+- https://portal.core.edu.au/conf-ranks/
+- https://portal.core.edu.au/jnl-ranks/

src/aletheia_probe/backends/__init__.py

@@ -5,6 +5,8 @@
 from . import (
     algerian_ministry,
     bealls,
+    core_conferences,
+    core_journals,
     crossref_analyzer,
     custom_list,
     dblp_venues,
@@ -28,6 +30,8 @@
 __all__ = [
     "algerian_ministry",
     "bealls",
+    "core_conferences",
+    "core_journals",
     "crossref_analyzer",
     "dblp_venues",
     "custom_list",

src/aletheia_probe/backends/core_conferences.py

@@ -0,0 +1,44 @@
+# SPDX-License-Identifier: MIT
+"""CORE/ICORE conference rankings backend for legitimate conference verification."""
+
+from typing import TYPE_CHECKING
+
+from ..enums import AssessmentType, EvidenceType
+from .base import CachedBackend, get_backend_registry
+
+
+if TYPE_CHECKING:
+    from ..updater.core import DataSource
+    from ..updater.sources.core import CoreConferenceSource
+
+
+class CoreConferencesBackend(CachedBackend):
+    """Backend that checks conferences against CORE/ICORE ranked venues."""
+
+    def __init__(self) -> None:
+        super().__init__(
+            source_name="core_conferences",
+            list_type=AssessmentType.LEGITIMATE,
+            cache_ttl_hours=24 * 30,
+        )
+        self._data_source: CoreConferenceSource | None = None
+
+    def get_name(self) -> str:
+        return "core_conferences"
+
+    def get_evidence_type(self) -> EvidenceType:
+        return EvidenceType.LEGITIMATE_LIST
+
+    def get_data_source(self) -> "DataSource | None":
+        if self._data_source is None:
+            from ..updater.sources.core import CoreConferenceSource
+
+            self._data_source = CoreConferenceSource()
+        return self._data_source
+
+
+get_backend_registry().register_factory(
+    "core_conferences",
+    lambda: CoreConferencesBackend(),
+    default_config={},
+)

src/aletheia_probe/backends/core_journals.py

@@ -0,0 +1,44 @@
+# SPDX-License-Identifier: MIT
+"""CORE journal rankings backend for legitimate journal verification."""
+
+from typing import TYPE_CHECKING
+
+from ..enums import AssessmentType, EvidenceType
+from .base import CachedBackend, get_backend_registry
+
+
+if TYPE_CHECKING:
+    from ..updater.core import DataSource
+    from ..updater.sources.core import CoreJournalSource
+
+
+class CoreJournalsBackend(CachedBackend):
+    """Backend that checks journals against CORE ranked journals."""
+
+    def __init__(self) -> None:
+        super().__init__(
+            source_name="core_journals",
+            list_type=AssessmentType.LEGITIMATE,
+            cache_ttl_hours=24 * 30,
+        )
+        self._data_source: CoreJournalSource | None = None
+
+    def get_name(self) -> str:
+        return "core_journals"
+
+    def get_evidence_type(self) -> EvidenceType:
+        return EvidenceType.LEGITIMATE_LIST
+
+    def get_data_source(self) -> "DataSource | None":
+        if self._data_source is None:
+            from ..updater.sources.core import CoreJournalSource
+
+            self._data_source = CoreJournalSource()
+        return self._data_source
+
+
+get_backend_registry().register_factory(
+    "core_journals",
+    lambda: CoreJournalsBackend(),
+    default_config={},
+)

src/aletheia_probe/config.py

@@ -112,6 +112,22 @@ class DataSourceUrlConfig(BaseModel):
         "https://dblp.org/xml/dblp.xml.gz",
         description="URL for DBLP full XML dump",
     )
+    core_conference_rankings_url: str = Field(
+        "https://portal.core.edu.au/conf-ranks/",
+        description="URL for CORE/ICORE conference rankings portal",
+    )
+    core_journal_rankings_url: str = Field(
+        "https://portal.core.edu.au/jnl-ranks/",
+        description="URL for CORE journal rankings portal",
+    )
+    core_conference_default_source: str = Field(
+        "ICORE2026",
+        description="Default source filter for CORE conference rankings",
+    )
+    core_journal_default_source: str = Field(
+        "CORE2020",
+        description="Default source filter for CORE journal rankings",
+    )
 
 
 class DataSourceProcessingConfig(BaseModel):

src/aletheia_probe/updater/sources/__init__.py

@@ -3,6 +3,7 @@
 
 from .algerian import AlgerianMinistrySource
 from .bealls import BeallsListSource
+from .core import CoreConferenceSource, CoreJournalSource
 from .custom import CustomListSource
 from .dblp import DblpVenueSource
 from .kscien_generic import KscienGenericSource
@@ -25,6 +26,8 @@
     "AlgerianMinistrySource",
     "BeallsListSource",
     "CustomListSource",
+    "CoreConferenceSource",
+    "CoreJournalSource",
     "DblpVenueSource",
     "KscienGenericSource",
     "KscienHijackedJournalsSource",