Commit 6e41a93
Consolidates duplicated code between kscien_predatory_conferences.py and kscien_generic.py
by creating a shared kscien_helpers.py utility module. This eliminates ~300 lines of
duplicate pagination, HTML parsing, and deduplication logic.
Changes:
- Add kscien_helpers.py with fetch_kscien_data(), deduplicate_entries() functions
- Remove kscien_predatory_conferences.py (replaced by KscienGenericSource)
- Refactor kscien_generic.py to use shared utilities
- Update source registration to use KscienGenericSource for predatory conferences
- Add comprehensive unit tests for refactored functionality
This refactoring improves maintainability, reduces code duplication, and aligns with
DRY principles while preserving all existing functionality.
Co-authored-by: florath-ai-assistant[bot] <Andreas.Florath@telekom.de>
1 parent 3e41559 commit 6e41a93
File tree
6 files changed
+483
-735
lines changed- src/aletheia_probe/updater
- sources
- tests/unit
6 files changed
+483
-735
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | | - | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
| 39 | + | |
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| 55 | + | |
55 | 56 | | |
56 | | - | |
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
7 | 8 | | |
8 | | - | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | | - | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
0 commit comments