Home

💡 Usage guide

See the following wiki pages that that will guide you through various use-cases of factgenie:

Topic	Description
🔧 Setup	How to install factgenie.
🗂️ Data Management	How to manage datasets and model outputs.
🤖 LLM Annotations	How to annotate outputs using LLMs.
👥 Crowdsourcing Annotations	How to annotate outputs using human crowdworkers.
✍️ Generating Outputs	How to generate outputs using LLMs.
📊 Analyzing Annotations	How to obtain statistics on collected annotations.
💻 Command Line Interface	How to use factgenie command line interface.
🌱 Contributing	How to contribute to factgenie.

🔥 Tutorials

We also provide step-by-step walkthroughs showing how to employ factgenie on the the dataset from the Shared Task in Evaluating Semantic Accuracy:

Tutorial	Description
🏀 #1: Importing a custom dataset	Loading the basketball statistics and model-generated basketball reports into the web interface.
💬 #2: Generating outputs	Using Llama 3.1 with Ollama for generating basketball reports.
📊 #3: Customizing data visualization	Manually creating a custom dataset class for better data visualization.
🤖 #4: Annotating outputs with an LLM	Using GPT-4o for annotating errors in the basketball reports.
👨‍💼 #5: Annotating outputs with human annotators	Using human annotators for annotating errors in the basketball reports.