evals

Evaluators code and prompts

• Requirements

• Quickstart

  Evaluator	Code	Prompts
  Grade Level Appropriateness	Jupyter Notebook	Prompts
  Sentence Structure	Jupyter Notebook	Prompts
  Vocabulary	Jupyter Notebook	Prompts
  Subject Matter Knowledge	Jupyter Notebook	Prompts
  Conventionality	Jupyter Notebook	Prompts
  Purpose	Jupyter Notebook	Prompts
  Text complexity (combo)	Jupyter Notebook

Requirements

Set up your environment and install your dependencies:

make install

Next, set your API keys as environment variables in your shell session:

export GOOGLE_API_KEY="..."
export OPENAI_API_KEY="..."

You can also add these environment variables to an .env file.

Quickstart

Use the provided Makefile to quickly set up your environment and install dependencies with make tool.

Install dependencies

make install

Next, start a Jupyter Notebook so you can run the evaluator examples and easily interact with your outputs:

Start Jupyter Notebook:

make jupyter

Jupyter will open in your web browser (usually at http://localhost:8888).

1. In Jupyter file browser, double click on the evaluator you want to try.
2. Copy the text you want to evaluate into the last code cell of the notebook to try the evaluation of your text sample.
3. Use the toolbar to run all cells, or run each cell in sequence until the end.

If you prefer using an IDE with Python and Jupyter notebook support, such as VSCode with Microsoft's Python and Jupyter extensions, please refer to Microsoft's instructions for their installation and configuration.