DeepEval for Typescript

[penguine-ip]

Hi folks - we recently released our plans for Typescript support, and we could borrow some help if some is available :)

The current /typescript folder already contains and acts as a client wrapper to Confident AI, and currently already supports tracing.

Our goal is to make local evals, simulation, and synthetic data generation, available in Typescript by July 1st.

We're leaving out benchmarks on purpose because we don't believe that's what Typescript users want.

Here are the initial TODOs:

• Support local evaluation models
• Support for complete metrics parity, with shared templates
• Support for unit test based testing workflows (similar to evals iterator, evaluate function, and deepeval test run)

Post July 1st:

• Support for complete feature parity with regards to synthetic data generation
• Support for complete feature parity with regards to simulation
• Support for prompt optimization

If you're interested, please comment below.

[A-Vamshi]

Hello! I'll be working on the task Support local evaluation models 😊. We encourage anyone working on the above tasks to please tag us here so we can assign you to the task to avoid any duplicate work. Thank you everyone!

[sahniaditya007]

Hi! I'd love to contribute to the TypeScript support effort.

I'd like to work on complete metrics parity with shared templates for the TypeScript SDK, unless someone is already working on it. I'm happy to coordinate with the maintainers and follow the existing architecture to ensure feature parity with the Python implementation.

Please assign this task to me if it's available.

[KUNDAN1334]

Hey @penguine-ip,

I'd love to contribute to the TypeScript support effort.

I'd like to work on Support for unit test–based testing workflows, specifically implementing TypeScript equivalents of evals_iterator, evaluate, and deepeval test run, while following the existing architecture and maintaining parity with the Python implementation.

If this task is still available, I'd be happy to get started. Please assign it to me if that works for you.

[ahmedalawady]

Hi @A-Vamshi I would like to contribute if it is possible.

[A-Vamshi]

Hey @ahmedalawady, @KUNDAN1334, @sahniaditya007 thanks for expressing your interest in contributing! We really appreciate it, the tasks Support local evaluation models and Support for complete metrics parity, with shared templates have already been completed, and it seems Support for synthetic data, simulation parity, and prompt optimization have been added in PR #2798 by @theanuragg as well.

It would be great if you could help with the Support for unit test based testing workflows (similar to evals iterator, evaluate function, and deepeval test run) task, please let us know if you're interested in this one, along with a list of things you'd like to tackle in a comment below, we'd love your help with these tasks!