Skip to content

RAGtruth integration#25

Merged
FrejaThoresen merged 3 commits into
feat/ragtruthfrom
main
May 26, 2026
Merged

RAGtruth integration#25
FrejaThoresen merged 3 commits into
feat/ragtruthfrom
main

Conversation

@FrejaThoresen
Copy link
Copy Markdown
Collaborator

  • Adds full RAGTruth processing pipeline, including new preprocessing and translation scripts for multilingual data preparation.
  • Introduces an end-to-end ground-truth evaluation workflow with new modules and scripts for annotation, schema upgrades, hallucination dataset generation, LLM judging, and evaluation against human labels.
  • Expands core package capabilities with new evaluation and logging components, plus updates to dataset generation, model generation, training, prompt utilities, and hallucination detection.
  • Adds baseline and utility script improvements for generation, detection, and detector training workflows.
  • Updates prompt templates across many languages and adds a new Albanian prompt file.
  • Strengthens testing by adding ground-truth evaluation tests.
  • Refreshes project docs and analysis artifacts, including new research/annotation docs and human evaluation outputs.
    Updates developer and runtime tooling (Dockerfile, docker-compose, pyproject, lockfile, makefile, pre-commit, mise, gitignore).

dependabot Bot and others added 3 commits May 13, 2026 15:14
Bumps [nltk](https://github.com/nltk/nltk) from 3.9.1 to 3.9.4.
- [Changelog](https://github.com/nltk/nltk/blob/develop/ChangeLog)
- [Commits](nltk/nltk@3.9.1...3.9.4)

---
updated-dependencies:
- dependency-name: nltk
  dependency-version: 3.9.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Update docker file

* Scripts from KLabs for ragtruth processing

* Ground truth eval without ragtruth

* merge

* Files from ucloud

* Import fix

* Bugfix in prompt util

* Don't shuffle

* Shuffle after split

* Add switch for ragtruth

* Linting

* Linting

* Update readme

* Add summary of ground truth analysis

* Clean up ground truth scripts

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@FrejaThoresen FrejaThoresen merged commit 18bc098 into feat/ragtruth May 26, 2026
1 check failed
FrejaThoresen added a commit that referenced this pull request May 26, 2026
* Bump nltk from 3.9.1 to 3.9.4 (#17)

Bumps [nltk](https://github.com/nltk/nltk) from 3.9.1 to 3.9.4.
- [Changelog](https://github.com/nltk/nltk/blob/develop/ChangeLog)
- [Commits](nltk/nltk@3.9.1...3.9.4)

---
updated-dependencies:
- dependency-name: nltk
  dependency-version: 3.9.4
  dependency-type: direct:production
...




* Feat/ragtruth (#20)

* Update docker file

* Scripts from KLabs for ragtruth processing

* Ground truth eval without ragtruth

* merge

* Files from ucloud

* Import fix

* Bugfix in prompt util

* Don't shuffle

* Shuffle after split

* Add switch for ragtruth

* Linting

* Linting

* Update readme

* Add summary of ground truth analysis

* Clean up ground truth scripts

* Potential fix for pull request finding



---------



---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant