GitHub - userdarius/llm-reasoning-uncertainty: This repo accompanies a study of uncertainty benchmarking on state of the art llm decoding methods and novel fast inference techniques. We start with multi token prediction, speculative decoding as well as CoT decoding and evaluate llm uncertainty with these added methods compared to the baseline model using semantic entropy as our metric..

Files to watch at the moment are compute_uncertainty_measures and speculative decoding related code.

Name		Name	Last commit message	Last commit date
Latest commit History 20 Commits
config		config
darius/uncertainty/wandb		darius/uncertainty/wandb
data_loader		data_loader
evaluation		evaluation
models		models
techniques		techniques
utils		utils
.gitignore		.gitignore
README.md		README.md
__init__.py		__init__.py
environment.yml		environment.yml
main.py		main.py
requirements.txt		requirements.txt

Provide feedback