Skip to content

Refactor the Linker, Ranker, Recogniser and Pipeline#282

Merged
thobson88 merged 196 commits intodevfrom
276-refactor
Mar 28, 2025
Merged

Refactor the Linker, Ranker, Recogniser and Pipeline#282
thobson88 merged 196 commits intodevfrom
276-refactor

Conversation

@thobson88
Copy link
Collaborator

@thobson88 thobson88 commented Dec 15, 2024 •
edited
Loading

Closes #276.

Major refactor. The main changes are:

  • new subclass structure in the Recogniser, Ranker & Linker (and common load and run interface)
  • new dataclasses representing data structures throughout the pipeline: Mention, Sentence, Candidates, Predictions, etc.
  • simplified Pipeline class, with all business logic moved into the Linker

@thobson88
Ignore .DS_Store files
00a3607
@thobson88
Refactor linking methods as Linker subclasses
a3c8c7a 
All tests not marked as 'deezy' are passing
@thobson88
Add RelDisamb Linker subclass & fix tests
a114f2c
@thobson88
Move rel_params to the RelDisambLinker class
769488e
@thobson88
Remove superfluous return value
5422243
@thobson88
Refactor Ranker into subclasses
e8282fb
@thobson88
Modify Ranker class hierarchy; fix unit tests
7a1e7a5
@thobson88
Fix method_name in DeezyMatchRanker
b42f6e5
@thobson88
Fix integration tests
3bf6fa6
@thobson88
Move train method to DeezyMatchRanker
936cc21
@thobson88
Fix docstrings
5219598
@thobson88
Move deezy-specific code to Ranker subclass
c94c166
@thobson88
Update docstrings
c3cf63b
@thobson88
Refactor the Recogniser class
daf97ad
@thobson88
Simplify variable names & rename module to avoid clash
b31d07e
@thobson88
Simplify Pipeline constructor
2e7d53d
@thobson88
Remove superfluous return value
dc97af8
@thobson88
Add Pipeline constructor unit test
ee02a65
@thobson88
Remove inconsistency in treating place_wqid
e13743f
@thobson88
Remove superfluous return value
d28f5ad
@thobson88
Add dataclasses to represent ranking candidates
7c39f6d
@thobson88
Integrate dataclasses into the Ranker
e42192d
@thobson88
Integrate dataclasses into Linker & Pipeline
2088beb
@thobson88
Add method return types
29de527
@thobson88
Move dataclasses to own module
1394718
@thobson88
Add new dataclasses for Ranker & Linker output
6d8fa19
@thobson88
Update dataclasses and integrate into Linker & Pipeline
34260f0
@thobson88
Reorder dataclasses module
004bd53
@thobson88
Rename Ranker cache & load method; Edit docstrings
d54392a
@thobson88
Replace Ranker method_name() with class attribute
ffe8a59
@thobson88
Fix missing return statement
d2e7093
@thobson88
Add GPU device param to Recogniser init
c7f48e0
@thobson88
Add GPU device config to Linker & Recogniser
2206164
@thobson88
Add linker model name in log message
c8f4813
@thobson88
Set Deezy config to verbose logging
ebe2ee5
@thobson88
Add Deezy model print statements
55919cb
@thobson88
Pass HuggingFace Dataset to recogniser pipeline
fe8f5de
@thobson88
Omit empty SentenceMentions in batch job
a9262b4
@thobson88
Allow partial DeezyMatch parameter config
a0e498f
@thobson88
Test non-default Deezy params in BatchJob
57b5865
@thobson88
Handle empty article text in BatchJob
f32ef40
@thobson88
Mark test as resource-dependent
64b9a2a
@thobson88
Remove print statements
96031b1
@thobson88
Add log message for REL device
687866f
@thobson88
Add log message for Recogniser device
cfd2f01
@thobson88
Add best coords methods in Predictions dataclass
baa6d07
@thobson88
Update Ranker API to take a list of toponym mentions
12e4d12
@thobson88
Add CombinedScores dataclass & integration test
3a6fe36
@thobson88
Add predicted coordinates in BatchJob CSV results
b2bd844
@thobson88
Add combined scores test with default place of pub
9d7c85b
@thobson88
Merge pull request #291 from Living-with-machines/274-gpu
9c8b20d 
GPU support
@thobson88
Write predictions.csv in batch job results
b7ab8ea
@thobson88
Add input row index column to BatchJob predictions CSV
31a6c83
@thobson88
Update docs: split Complete Tour into separate pages
352db8e
@thobson88
Add class diagrams in docs
f2ffcd4
@thobson88
Add dataclasses page in docs
b92c78a
@thobson88
Finish Batch Job docs & fix links
165918d
@thobson88
Update ci.yml
8e272e0 
Migrate from deprecated v2 to v4.
@thobson88 thobson88 merged commit 25e662f into dev Mar 28, 2025
1 check passed
@thobson88 thobson88 deleted the 276-refactor branch March 28, 2025 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rwood-97 rwood-97 rwood-97 left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Refactor the Linker, Ranker, Recogniser and Pipeline [PR open]

2 participants

@thobson88 @rwood-97