Relation extraction llama by vladd-bit · Pull Request #522 · CogStack/MedCAT

vladd-bit · 2025-03-10T10:04:27Z

This should be the final version of the relation extraction implementation, the following were implemented:

Llama model support
ModernBert model support
saving/loading for all three variations of models (BERT/Llama/ModernBERT)
improvements to documentation
fixes to dataset creation from trainer exports, the way the num non-relations were
fixes to dataset creation from spacy docs/fake docs, removed duplicates
fixes to inference (input data was not filtered/validated, many invalid combinations were present) resulting in a lot of garbage relations

…tion_extraction

mart-r · 2025-04-22T13:01:10Z

I fixed the linting issues from about (mostly to do with darglint / doc string alignment with method signatures).

I also fixed some test-time issues, such as import paths and init arguments.
And then I fixed some test-time property paths (i.e config is now at rel_cat.component.relcat_config and model is now at rel_cat.component.model).
However, the tests still fail at RelCATTests.test_train_csv_no_tags. And I'm not sure which changes I'd need to change to make this work here.
Perhaps it's the tokenizer that needs to be manually added or something?

To add some more context, the above failure causes the error in RelCATTests.test_train_mctrainer (since the model wasn't saved it cannot be loaded either) and the failure of RelCATTests.test_train_predict since this also relies on previous state.

mart-r · 2025-04-22T13:02:56Z

Also, wanted to comment that I'm not sure init_mode=True behaviour for BaseComponent_RelationExtraction makes a lot of sense. It creates the instance based on the default config, regardless of whether one was provided beforehand.

…ck/MedCAT into relation_extraction_llama

medcat/config_rel_cat.py

        validate_assignment = True
+
+    @classmethod
+    def load(cls, load_path: str = "./") -> "ConfigRelCAT":


mart-r

Looks really quite good to me.

Hopefully this allows the RelCAT side of things to be relatively easy to maintain and add on top in the future by moving the differences in the models and tokenizers away from the main logic and concentrating the choices to a few distinct places.

There were a few minor comments I had made before, but I'll leave them be. We can't keep looking for a subjective perfection forever.

mart-r · 2025-04-24T14:34:41Z

PS: I think Tom will take a look at this as well. Good to have another pair of eyes. Plus, some of this (although a very small - almost insignificant part) was done by me, so kind of weird for me to approve it.

tomolopolis

lgtm - potentially some more refactoring to be done especially between the Bert and ModernBERT implementations, but this is good to go as is for now

tomolopolis · 2025-04-24T15:52:35Z

.gitignore

 venv
 db.sqlite3
 .ipynb_checkpoints
+./install_requires.txt


don't think we want to ignore changes to this

Good catch - pushed a fix for this.

tomolopolis · 2025-04-24T15:53:58Z

medcat/config_rel_cat.py


    NB! For these changes to take effect, the pipe would need to be recreated."""
-    annotation_schema_tag_ids: List = []
+    annotation_schema_tag_ids: List = [30522, 30523, 30524, 30525]


can you add a comment line as to what these indices are, I assume they are the <s1>, <e1> , <s2>, <e2> tok ids?

Actually looking at it, you've got the listed below, so probably fine tbh..

* CU-8693bc9kc: Add python 3.12 support (#511) * CU-8693bc9kc: Add python 3.12 support * CU-8693bc9kc: Amend dependencies so as to be compatible with python 3.12 * Bump default spacy model version (to 3.8) * CU-8693bc9kc: Fix some typing issues due to numpy2 * CU-8693bc9kc: Fix some typing issues due to numpy2 (try 2) * CU-8693bc9kc: Change spacy models to 3.7.2 * CU-8693bc9kc: Pin numpy to v1 * CU-8693bc9kc: Fix numpy requirement comment * CU-8693bc9kc: Fix usage of old/deprecated assert methods in tests * CU-8693bc9kc: Update some requirement comments * CU-8697c86rf: Update docs build requirements (#514) * CU-8697c86rf: Update docs build requirements * CU-8697c86rf: Fix docs build requirements (hopefully) * CU-8697c86rf: Fix docs build requirements (hopefully) x2 * CU-8697x7y9x: Fix issue with transformers 4.47+ affecting DeID (#517) * CU-8697x7y9x: Fix issue with transformers 4.47+ affecting DeID * CU-8697x7y9x: Add type-ignore to module unrelated to current change * Updates for MetaCAT (#515) * Pushing update for MetaCAT - Addressing the multiple zero-division-error warnings per epoch while training - Accommodating the variations in category name and class name across NHS sites * Adding comments * Pushing requested changes * Pushing type fix * Pushing updates to metacat config * Support expansion of transformers ner models to include new concepts (#519) * CU-8697v6qr2 support expansion of transformers ner models to include new concepts * CU-8697v6qr2 add logging suggested by the review * CU-869805t7e alt names fixes (#520) * CU-869805t7e: Move getting of applicable category name to the config * CU-869805t7e: Use alternative category names in eval method * CU-869805t7e: Reduce indentation * CU-869805t7e: Reduce indentation (again) * CU-869805t7e: Some comment fixing due to rearrangements before * CU-869805t7e: Fix usage of matched class name when encoding category values * CU-869805t7e: Avoid duplicating exception message * CU-8697qfvzz train metacat on sup train (#516) * CU-8697qfvzz: Add new optional keyword argumnet to allow training MetaCAT models during supervised training * CU-8697qfvzz: Add tests regarding training meta-cats during supervised training * CU-8697qfvzz: Fix small typo in comment * CU-8697qfvzz: Allow using alternative category names if/when training meta cats through CAT.train_supervised * CU-8698ek477: Fix AdamW import from tranformers to torch (#523) * CU-8698ek477: Add TODO to MetaCAT ML utils regarding AdamW import * CU-8698ek477: Fix AdamW import (trf->torch) * CU-8698f8fgc: Fix negative sampling including indices for words without a vector (#524) * CU-8698f8fgc: Add new test to check that the negative sampling indices do not include non-vectored indices * CU-8698f8fgc: Add fix for negative sampling including indices for words without a vector * CU-8698f8fgc: Update tests to make sure index frequencies are respected * CU-8698f8fgc: Add 3.9-friendly counter totalling method * CU-8698gkrqa: Add argument to allow specifying the changes warrenting a model save (#525) * CU-8698hfkch: Add eval method to deid model * CU-8698hfkch: lint checks * CU-8698gqumv: Fix regression test vocab vector sizes (#526) * CU-8698gqumv: Add tests for Vocab upon regression testing * CU-8698gqumv: Fix regression time vocab data * CU-86983ruw9 Fix test train split (#521) * CU-86983ruw9: Fix train-test splitter leaving train set empty for smaller datasets * CU-86983ruw9: Add additional optional arguments to test-train splitting for minimum concept count and maximum test fraction * CU-86983ruw9: Add a few tests for test-train splitting * CU-8698hfkch: Add eval method to deid model (#527) * CU-8698hfkch: Add eval method to deid model * CU-8698hfkch: lint checks --------- Co-authored-by: Tom Searle <tom@cogstack.org> * CU-8698jzjj3: pass in extra param if ignore_extra_labels is set, and test * CU-8698mqu96 Transformers update (4.51.0) fix (#531) * CU-8698mqu96: Update special tokens lengths attribute * CU-8698mqu96: Update MetaCAT usage of BertTokenizer.from_pretrained for type safety * CU-8698mqu96: Ignore typing where mypy is wrong + add note in code * CU-8698mqu96: Ignore typing where mypy may be wrong + add comment * CU-8698mqu96: Fix tokenizer wrapper import for rel cat * CU-8698mqu96: Rename evaluation strategy keyword argument in line with changes * CU-8698mqu96: Type-ignore method where mypy says it does not exist * CU-8698mqu96: Fix TRF-NER output dir typing issue * CU-8698mqu96: Update a doc string for darglint * CU-8698mqu96: Fix typing issue for TrfNER trainer callback * Relation extraction llama (#522) * Added files. * More additions to rel extraction. * Rel base. * Update. * Updates. * Dependency parsing. * Updates. * Added pre-training steps. * Added training & model utils. * Cleanup & fixes. * Update. * Evaluation updates for pretraining. * Removed duplicate relation storage. * Moved RE model file location. * Structure revisions. * Added custom config for RE. * Implemented custom dataset loader for RE. * More changes. * Small fix. * Latest additions to RelCAT (pipe + predictions) * Setup.py fix. * RE utils update. * rel model update. * rel dataset + tokenizer improvements. * RelCAT updates. * RelCAT saving/loading improvements. * RelCAT saving/loading improvements. * RelCAT model fixes. * Attempted gpu learning fix. Dataset label generation fixes. * Minor train dataset gen fix. * Minor train dataset gen fix No.2. * Config updates. * Gpu support fixes. Added label stats. * Evaluation stat fixes. * Cleaned stat output mode during training. * Build fix. * removed unused dependencies and fixed code formatting * Mypy compliance. * Fixed linting. * More Gpu mode train fixes. * Fixed model saving/loading issues when using other baes models. * More fixes to stat evaluation. Added proper CAT integration of RelCAT. * Setup.py typo fix. * RelCAT loading fix. * RelCAT Config changes. * Type fix. Minor additions to RelCAT model. * Type fixes. * Type corrections. * RelCAT update. * Type fixes. * Fixed type issue. * RelCATConfig: added seed param. * Adaptations to the new codebase + type fixes.. * Doc/type fixes. * Fixed input size issue for model. * Fixed issue(s) with model size and config. * RelCAT: updated configs to new style. * RelCAT: removed old refs to logging. * Fixed GPU training + added extra stat print for train set. * Type fixes. * Updated dev requirements. * Linting. * Fixed pin_memory issue when training on CPU. * Updated RelCAT dataset get + default config. * Updated RelDS generator + default config * Linting. * Updated RelDatset + config. * Pushing updates to model Made changes to: 1) Extracting given number of context tokens left and right of the entities 2) Extracting hidden state from bert for all the tokens of the entities and performing max pooling on them * Fixing formatting * Update rel_dataset.py * Update rel_dataset.py * Update rel_dataset.py * RelCAT: added test resource files. * RelCAT: Fixed model load/checkpointing. * RelCAT: updated to pipe spacy doc call. * RelCAT: added tests. * Fixed lint/type issues & added rel tag to test DS. * Fixed ann id to token issue. * RelCAT: updated test dataset + tests. * RelCAT: updates to requested changes + dataset improvements. * RelCAT: updated docs/logs according to commends. * RelCAT: type fix. * RelCAT: mct export dataset updates. * RelCAT: test updates + requested changes p2. * RelCAT: log for MCT export train. * Updated docs + split train_test & dataset for benchmarks. * type fixes. * RelCAT: Initial Llama integration. * RelCAT: updates to Llama impl. * RelCAT: model typo fix. * RelCAT: label_id /sample no. mixup fix. * Updated cleaned up Relataset, added new ways to create relations via anno types (doc/export only for now). * Added option to predict any text /w annotations via RelCAT. MCT export train fixes. * RelCAT: added sample limiter / class, more logging info. * RelCAT: test/train ds shuffle update. * RelCAT: added option to keep original text when using reldataset class. * Pushing change for stratified batching Implement stratified batching for improved class representation and balanced training * RelCAT: fixed doc processing issue + class weights. * RelCAT: class weights addtions to cfg + param. * RelCAT: added config params for Adam optimizer. * RelCAT updated default config. * RelCAT: config update + optimizer change. * RelCAT: fixed model freeze flags. * RelCAT: model optimizer save/load fix. * RelCAT: added export ent tag check. * Fixed issues when saving/loading model for class weights + inference device cast. * RelCAT: bug fix for ents that are @ EoS. * Rel Dataset updates. * Rel Dataset updates. * Pushing change for ModernBERT * Bumped transformers version. * Updated rel dataset generation from fake Spacy Docs. * ModernBert updates. * Updated RelCAT model-load/save. * Minor relCAT updates, code format. * Type check updates. * Fixed inference issue. * RelCAT: testing updates. * Type fixes. * Type fixes. * Type fixes. * Type fixes IV. * Type fixes python 3.9. * RelCAT: flake8 fixes. * RelCAT: flake8 fixes. * RelCAT: Updates (fixed model loading after save). * Fixed test. * Update RelCAT stuff for improved abstraction * Move separate model implementations to separate packages * Some minor abstraction changes * Remove accidentally copied abstract method decorator * Fix import in test * Fix RelCAT impport in pipe tests * Update base relcat model implementation to include config * Latest RelCAT module updates. * Type fixes + run issues. * Type fixes. * Fixed Llama tokenizer. * Type fixes. * Type fixes: Python3.10 adjustements. * Linting. * Fix base flake8 lint issues * Fix doc string in ConfigRelCAT.load * Fix base component init doc string * Fixed BaseComponent.load method doc string * Fix doc strings in rel_cat ml_utils * Fix doc strings in rel_cat models module * Fix rel-cat test time import * Fix type casting * Align pipe tests with rel cat changes * Fix property paths in rel cat tests * Updates. * Fixed tests. * Fixed relCAT config save. * Latest fixes for model saving/loading. * Lint fix. * RelCAT cfg load test fix. * Remove install requirements from gitignore --------- Co-authored-by: Shubham Agarwal <66172189+shubham-s-agarwal@users.noreply.github.com> Co-authored-by: mart-r <mart.ratas@gmail.com> * CU-8698vewzp: Fix docs requirements (hopefully) (#534) * CU-8698veb6y: Use Ubuntu 24.04 for publishing to test PyPI (#533) --------- Co-authored-by: Shubham Agarwal <66172189+shubham-s-agarwal@users.noreply.github.com> Co-authored-by: Xi Bai <82581439+baixiac@users.noreply.github.com> Co-authored-by: Tom Searle <tom@cogstack.org> Co-authored-by: tomolopolis <tsearle88@gmail.com> Co-authored-by: Vlad Dinu <62345326+vladd-bit@users.noreply.github.com>

* CU-8693bc9kc: Add python 3.12 support (CogStack/MedCAT#511) * CU-8693bc9kc: Add python 3.12 support * CU-8693bc9kc: Amend dependencies so as to be compatible with python 3.12 * Bump default spacy model version (to 3.8) * CU-8693bc9kc: Fix some typing issues due to numpy2 * CU-8693bc9kc: Fix some typing issues due to numpy2 (try 2) * CU-8693bc9kc: Change spacy models to 3.7.2 * CU-8693bc9kc: Pin numpy to v1 * CU-8693bc9kc: Fix numpy requirement comment * CU-8693bc9kc: Fix usage of old/deprecated assert methods in tests * CU-8693bc9kc: Update some requirement comments * CU-8697c86rf: Update docs build requirements (CogStack/MedCAT#514) * CU-8697c86rf: Update docs build requirements * CU-8697c86rf: Fix docs build requirements (hopefully) * CU-8697c86rf: Fix docs build requirements (hopefully) x2 * CU-8697x7y9x: Fix issue with transformers 4.47+ affecting DeID (CogStack/MedCAT#517) * CU-8697x7y9x: Fix issue with transformers 4.47+ affecting DeID * CU-8697x7y9x: Add type-ignore to module unrelated to current change * Updates for MetaCAT (CogStack/MedCAT#515) * Pushing update for MetaCAT - Addressing the multiple zero-division-error warnings per epoch while training - Accommodating the variations in category name and class name across NHS sites * Adding comments * Pushing requested changes * Pushing type fix * Pushing updates to metacat config * Support expansion of transformers ner models to include new concepts (CogStack/MedCAT#519) * CU-8697v6qr2 support expansion of transformers ner models to include new concepts * CU-8697v6qr2 add logging suggested by the review * CU-869805t7e alt names fixes (CogStack/MedCAT#520) * CU-869805t7e: Move getting of applicable category name to the config * CU-869805t7e: Use alternative category names in eval method * CU-869805t7e: Reduce indentation * CU-869805t7e: Reduce indentation (again) * CU-869805t7e: Some comment fixing due to rearrangements before * CU-869805t7e: Fix usage of matched class name when encoding category values * CU-869805t7e: Avoid duplicating exception message * CU-8697qfvzz train metacat on sup train (CogStack/MedCAT#516) * CU-8697qfvzz: Add new optional keyword argumnet to allow training MetaCAT models during supervised training * CU-8697qfvzz: Add tests regarding training meta-cats during supervised training * CU-8697qfvzz: Fix small typo in comment * CU-8697qfvzz: Allow using alternative category names if/when training meta cats through CAT.train_supervised * CU-8698ek477: Fix AdamW import from tranformers to torch (CogStack/MedCAT#523) * CU-8698ek477: Add TODO to MetaCAT ML utils regarding AdamW import * CU-8698ek477: Fix AdamW import (trf->torch) * CU-8698f8fgc: Fix negative sampling including indices for words without a vector (CogStack/MedCAT#524) * CU-8698f8fgc: Add new test to check that the negative sampling indices do not include non-vectored indices * CU-8698f8fgc: Add fix for negative sampling including indices for words without a vector * CU-8698f8fgc: Update tests to make sure index frequencies are respected * CU-8698f8fgc: Add 3.9-friendly counter totalling method * CU-8698gkrqa: Add argument to allow specifying the changes warrenting a model save (CogStack/MedCAT#525) * CU-8698hfkch: Add eval method to deid model * CU-8698hfkch: lint checks * CU-8698gqumv: Fix regression test vocab vector sizes (CogStack/MedCAT#526) * CU-8698gqumv: Add tests for Vocab upon regression testing * CU-8698gqumv: Fix regression time vocab data * CU-86983ruw9 Fix test train split (CogStack/MedCAT#521) * CU-86983ruw9: Fix train-test splitter leaving train set empty for smaller datasets * CU-86983ruw9: Add additional optional arguments to test-train splitting for minimum concept count and maximum test fraction * CU-86983ruw9: Add a few tests for test-train splitting * CU-8698hfkch: Add eval method to deid model (CogStack/MedCAT#527) * CU-8698hfkch: Add eval method to deid model * CU-8698hfkch: lint checks --------- Co-authored-by: Tom Searle <tom@cogstack.org> * CU-8698jzjj3: pass in extra param if ignore_extra_labels is set, and test * CU-8698mqu96 Transformers update (4.51.0) fix (CogStack/MedCAT#531) * CU-8698mqu96: Update special tokens lengths attribute * CU-8698mqu96: Update MetaCAT usage of BertTokenizer.from_pretrained for type safety * CU-8698mqu96: Ignore typing where mypy is wrong + add note in code * CU-8698mqu96: Ignore typing where mypy may be wrong + add comment * CU-8698mqu96: Fix tokenizer wrapper import for rel cat * CU-8698mqu96: Rename evaluation strategy keyword argument in line with changes * CU-8698mqu96: Type-ignore method where mypy says it does not exist * CU-8698mqu96: Fix TRF-NER output dir typing issue * CU-8698mqu96: Update a doc string for darglint * CU-8698mqu96: Fix typing issue for TrfNER trainer callback * Relation extraction llama (CogStack/MedCAT#522) * Added files. * More additions to rel extraction. * Rel base. * Update. * Updates. * Dependency parsing. * Updates. * Added pre-training steps. * Added training & model utils. * Cleanup & fixes. * Update. * Evaluation updates for pretraining. * Removed duplicate relation storage. * Moved RE model file location. * Structure revisions. * Added custom config for RE. * Implemented custom dataset loader for RE. * More changes. * Small fix. * Latest additions to RelCAT (pipe + predictions) * Setup.py fix. * RE utils update. * rel model update. * rel dataset + tokenizer improvements. * RelCAT updates. * RelCAT saving/loading improvements. * RelCAT saving/loading improvements. * RelCAT model fixes. * Attempted gpu learning fix. Dataset label generation fixes. * Minor train dataset gen fix. * Minor train dataset gen fix No.2. * Config updates. * Gpu support fixes. Added label stats. * Evaluation stat fixes. * Cleaned stat output mode during training. * Build fix. * removed unused dependencies and fixed code formatting * Mypy compliance. * Fixed linting. * More Gpu mode train fixes. * Fixed model saving/loading issues when using other baes models. * More fixes to stat evaluation. Added proper CAT integration of RelCAT. * Setup.py typo fix. * RelCAT loading fix. * RelCAT Config changes. * Type fix. Minor additions to RelCAT model. * Type fixes. * Type corrections. * RelCAT update. * Type fixes. * Fixed type issue. * RelCATConfig: added seed param. * Adaptations to the new codebase + type fixes.. * Doc/type fixes. * Fixed input size issue for model. * Fixed issue(s) with model size and config. * RelCAT: updated configs to new style. * RelCAT: removed old refs to logging. * Fixed GPU training + added extra stat print for train set. * Type fixes. * Updated dev requirements. * Linting. * Fixed pin_memory issue when training on CPU. * Updated RelCAT dataset get + default config. * Updated RelDS generator + default config * Linting. * Updated RelDatset + config. * Pushing updates to model Made changes to: 1) Extracting given number of context tokens left and right of the entities 2) Extracting hidden state from bert for all the tokens of the entities and performing max pooling on them * Fixing formatting * Update rel_dataset.py * Update rel_dataset.py * Update rel_dataset.py * RelCAT: added test resource files. * RelCAT: Fixed model load/checkpointing. * RelCAT: updated to pipe spacy doc call. * RelCAT: added tests. * Fixed lint/type issues & added rel tag to test DS. * Fixed ann id to token issue. * RelCAT: updated test dataset + tests. * RelCAT: updates to requested changes + dataset improvements. * RelCAT: updated docs/logs according to commends. * RelCAT: type fix. * RelCAT: mct export dataset updates. * RelCAT: test updates + requested changes p2. * RelCAT: log for MCT export train. * Updated docs + split train_test & dataset for benchmarks. * type fixes. * RelCAT: Initial Llama integration. * RelCAT: updates to Llama impl. * RelCAT: model typo fix. * RelCAT: label_id /sample no. mixup fix. * Updated cleaned up Relataset, added new ways to create relations via anno types (doc/export only for now). * Added option to predict any text /w annotations via RelCAT. MCT export train fixes. * RelCAT: added sample limiter / class, more logging info. * RelCAT: test/train ds shuffle update. * RelCAT: added option to keep original text when using reldataset class. * Pushing change for stratified batching Implement stratified batching for improved class representation and balanced training * RelCAT: fixed doc processing issue + class weights. * RelCAT: class weights addtions to cfg + param. * RelCAT: added config params for Adam optimizer. * RelCAT updated default config. * RelCAT: config update + optimizer change. * RelCAT: fixed model freeze flags. * RelCAT: model optimizer save/load fix. * RelCAT: added export ent tag check. * Fixed issues when saving/loading model for class weights + inference device cast. * RelCAT: bug fix for ents that are @ EoS. * Rel Dataset updates. * Rel Dataset updates. * Pushing change for ModernBERT * Bumped transformers version. * Updated rel dataset generation from fake Spacy Docs. * ModernBert updates. * Updated RelCAT model-load/save. * Minor relCAT updates, code format. * Type check updates. * Fixed inference issue. * RelCAT: testing updates. * Type fixes. * Type fixes. * Type fixes. * Type fixes IV. * Type fixes python 3.9. * RelCAT: flake8 fixes. * RelCAT: flake8 fixes. * RelCAT: Updates (fixed model loading after save). * Fixed test. * Update RelCAT stuff for improved abstraction * Move separate model implementations to separate packages * Some minor abstraction changes * Remove accidentally copied abstract method decorator * Fix import in test * Fix RelCAT impport in pipe tests * Update base relcat model implementation to include config * Latest RelCAT module updates. * Type fixes + run issues. * Type fixes. * Fixed Llama tokenizer. * Type fixes. * Type fixes: Python3.10 adjustements. * Linting. * Fix base flake8 lint issues * Fix doc string in ConfigRelCAT.load * Fix base component init doc string * Fixed BaseComponent.load method doc string * Fix doc strings in rel_cat ml_utils * Fix doc strings in rel_cat models module * Fix rel-cat test time import * Fix type casting * Align pipe tests with rel cat changes * Fix property paths in rel cat tests * Updates. * Fixed tests. * Fixed relCAT config save. * Latest fixes for model saving/loading. * Lint fix. * RelCAT cfg load test fix. * Remove install requirements from gitignore --------- Co-authored-by: Shubham Agarwal <66172189+shubham-s-agarwal@users.noreply.github.com> Co-authored-by: mart-r <mart.ratas@gmail.com> * CU-8698vewzp: Fix docs requirements (hopefully) (CogStack/MedCAT#534) * CU-8698veb6y: Use Ubuntu 24.04 for publishing to test PyPI (CogStack/MedCAT#533) --------- Co-authored-by: Shubham Agarwal <66172189+shubham-s-agarwal@users.noreply.github.com> Co-authored-by: Xi Bai <82581439+baixiac@users.noreply.github.com> Co-authored-by: Tom Searle <tom@cogstack.org> Co-authored-by: tomolopolis <tsearle88@gmail.com> Co-authored-by: Vlad Dinu <62345326+vladd-bit@users.noreply.github.com>

* Added files. * More additions to rel extraction. * Rel base. * Update. * Updates. * Dependency parsing. * Updates. * Added pre-training steps. * Added training & model utils. * Cleanup & fixes. * Update. * Evaluation updates for pretraining. * Removed duplicate relation storage. * Moved RE model file location. * Structure revisions. * Added custom config for RE. * Implemented custom dataset loader for RE. * More changes. * Small fix. * Latest additions to RelCAT (pipe + predictions) * Setup.py fix. * RE utils update. * rel model update. * rel dataset + tokenizer improvements. * RelCAT updates. * RelCAT saving/loading improvements. * RelCAT saving/loading improvements. * RelCAT model fixes. * Attempted gpu learning fix. Dataset label generation fixes. * Minor train dataset gen fix. * Minor train dataset gen fix No.2. * Config updates. * Gpu support fixes. Added label stats. * Evaluation stat fixes. * Cleaned stat output mode during training. * Build fix. * removed unused dependencies and fixed code formatting * Mypy compliance. * Fixed linting. * More Gpu mode train fixes. * Fixed model saving/loading issues when using other baes models. * More fixes to stat evaluation. Added proper CAT integration of RelCAT. * Setup.py typo fix. * RelCAT loading fix. * RelCAT Config changes. * Type fix. Minor additions to RelCAT model. * Type fixes. * Type corrections. * RelCAT update. * Type fixes. * Fixed type issue. * RelCATConfig: added seed param. * Adaptations to the new codebase + type fixes.. * Doc/type fixes. * Fixed input size issue for model. * Fixed issue(s) with model size and config. * RelCAT: updated configs to new style. * RelCAT: removed old refs to logging. * Fixed GPU training + added extra stat print for train set. * Type fixes. * Updated dev requirements. * Linting. * Fixed pin_memory issue when training on CPU. * Updated RelCAT dataset get + default config. * Updated RelDS generator + default config * Linting. * Updated RelDatset + config. * Pushing updates to model Made changes to: 1) Extracting given number of context tokens left and right of the entities 2) Extracting hidden state from bert for all the tokens of the entities and performing max pooling on them * Fixing formatting * Update rel_dataset.py * Update rel_dataset.py * Update rel_dataset.py * RelCAT: added test resource files. * RelCAT: Fixed model load/checkpointing. * RelCAT: updated to pipe spacy doc call. * RelCAT: added tests. * Fixed lint/type issues & added rel tag to test DS. * Fixed ann id to token issue. * RelCAT: updated test dataset + tests. * RelCAT: updates to requested changes + dataset improvements. * RelCAT: updated docs/logs according to commends. * RelCAT: type fix. * RelCAT: mct export dataset updates. * RelCAT: test updates + requested changes p2. * RelCAT: log for MCT export train. * Updated docs + split train_test & dataset for benchmarks. * type fixes. * RelCAT: Initial Llama integration. * RelCAT: updates to Llama impl. * RelCAT: model typo fix. * RelCAT: label_id /sample no. mixup fix. * Updated cleaned up Relataset, added new ways to create relations via anno types (doc/export only for now). * Added option to predict any text /w annotations via RelCAT. MCT export train fixes. * RelCAT: added sample limiter / class, more logging info. * RelCAT: test/train ds shuffle update. * RelCAT: added option to keep original text when using reldataset class. * Pushing change for stratified batching Implement stratified batching for improved class representation and balanced training * RelCAT: fixed doc processing issue + class weights. * RelCAT: class weights addtions to cfg + param. * RelCAT: added config params for Adam optimizer. * RelCAT updated default config. * RelCAT: config update + optimizer change. * RelCAT: fixed model freeze flags. * RelCAT: model optimizer save/load fix. * RelCAT: added export ent tag check. * Fixed issues when saving/loading model for class weights + inference device cast. * RelCAT: bug fix for ents that are @ EoS. * Rel Dataset updates. * Rel Dataset updates. * Pushing change for ModernBERT * Bumped transformers version. * Updated rel dataset generation from fake Spacy Docs. * ModernBert updates. * Updated RelCAT model-load/save. * Minor relCAT updates, code format. * Type check updates. * Fixed inference issue. * RelCAT: testing updates. * Type fixes. * Type fixes. * Type fixes. * Type fixes IV. * Type fixes python 3.9. * RelCAT: flake8 fixes. * RelCAT: flake8 fixes. * RelCAT: Updates (fixed model loading after save). * Fixed test. * Update RelCAT stuff for improved abstraction * Move separate model implementations to separate packages * Some minor abstraction changes * Remove accidentally copied abstract method decorator * Fix import in test * Fix RelCAT impport in pipe tests * Update base relcat model implementation to include config * Latest RelCAT module updates. * Type fixes + run issues. * Type fixes. * Fixed Llama tokenizer. * Type fixes. * Type fixes: Python3.10 adjustements. * Linting. * Fix base flake8 lint issues * Fix doc string in ConfigRelCAT.load * Fix base component init doc string * Fixed BaseComponent.load method doc string * Fix doc strings in rel_cat ml_utils * Fix doc strings in rel_cat models module * Fix rel-cat test time import * Fix type casting * Align pipe tests with rel cat changes * Fix property paths in rel cat tests * Updates. * Fixed tests. * Fixed relCAT config save. * Latest fixes for model saving/loading. * Lint fix. * RelCAT cfg load test fix. * Remove install requirements from gitignore --------- Co-authored-by: Shubham Agarwal <66172189+shubham-s-agarwal@users.noreply.github.com> Co-authored-by: mart-r <mart.ratas@gmail.com>

vladd-bit added 30 commits August 24, 2021 10:34

Added files.

b20e7c8

Merge branch 'master' of https://github.com/CogStack/MedCAT into rela…

eec6c59

…tion_extraction

More additions to rel extraction.

56220aa

Rel base.

7ad88f5

Update.

233ce36

Updates.

85a7015

Dependency parsing.

5003548

Updates.

541b47d

Added pre-training steps.

c042b0d

Added training & model utils.

87d0c0c

Cleanup & fixes.

4f42696

Update.

018d811

Evaluation updates for pretraining.

f3d3f44

Removed duplicate relation storage.

e5f354e

Merged master.

c69de67

Moved RE model file location.

031d256

Merge branch 'master' of https://github.com/CogStack/MedCAT into rela…

2259a6b

…tion_extraction

Structure revisions.

1c469e9

Added custom config for RE.

423b4e1

Implemented custom dataset loader for RE.

8ae9abb

More changes.

186416c

Small fix.

451e33f

Latest additions to RelCAT (pipe + predictions)

8b36413

Setup.py fix.

2fb8fc9

RE utils update.

930dd11

rel model update.

24b2841

rel dataset + tokenizer improvements.

193ecb1

RelCAT updates.

03111a7

RelCAT saving/loading improvements.

7ab60f4

RelCAT saving/loading improvements.

40875f3

vladd-bit and others added 12 commits April 18, 2025 13:39

Type fixes: Python3.10 adjustements.

64b7602

Linting.

e1b3bb0

Fix base flake8 lint issues

ca7b2b8

Fix doc string in ConfigRelCAT.load

cb68de1

Fix base component init doc string

d19e11d

Fixed BaseComponent.load method doc string

da61a75

Fix doc strings in rel_cat ml_utils

441519c

Fix doc strings in rel_cat models module

9007fdb

Fix rel-cat test time import

fd5f5ed

Fix type casting

2b36753

Align pipe tests with rel cat changes

9a54211

Fix property paths in rel cat tests

f8b2d3e

vladd-bit added 6 commits April 23, 2025 09:40

Updates.

dfda693

Merge branch 'relation_extraction_llama' of https://github.com/CogSta…

2f68169

…ck/MedCAT into relation_extraction_llama

Fixed tests.

3cca0e2

Fixed relCAT config save.

0f5e61b

Latest fixes for model saving/loading.

9cc52b4

Lint fix.

05824f0

mart-r reviewed Apr 24, 2025

View reviewed changes

medcat/config_rel_cat.py

validate_assignment = True

@classmethod

def load(cls, load_path: str = "./") -> "ConfigRelCAT":

This comment was marked as resolved.

Sign in to view

RelCAT cfg load test fix.

a0c78d4

mart-r approved these changes Apr 24, 2025

View reviewed changes

tomolopolis approved these changes Apr 24, 2025

View reviewed changes

Remove install requirements from gitignore

579eb8b

mart-r merged commit 1e346e0 into master Apr 24, 2025
8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Relation extraction llama#522

Relation extraction llama#522
mart-r merged 205 commits intomasterfrom
relation_extraction_llama

vladd-bit commented Mar 10, 2025

Uh oh!

mart-r commented Apr 22, 2025

Uh oh!

mart-r commented Apr 22, 2025

Uh oh!

This comment was marked as resolved.

Uh oh!

mart-r left a comment

Uh oh!

mart-r commented Apr 24, 2025

Uh oh!

tomolopolis left a comment

Uh oh!

tomolopolis Apr 24, 2025

Uh oh!

mart-r Apr 24, 2025

Uh oh!

tomolopolis Apr 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

vladd-bit commented Mar 10, 2025

Uh oh!

mart-r commented Apr 22, 2025

Uh oh!

mart-r commented Apr 22, 2025

Uh oh!

This comment was marked as resolved.

Uh oh!

mart-r left a comment

Choose a reason for hiding this comment

Uh oh!

mart-r commented Apr 24, 2025

Uh oh!

tomolopolis left a comment

Choose a reason for hiding this comment

Uh oh!

tomolopolis Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

mart-r Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

tomolopolis Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants