Skip to content

Commit e9b1448

Browse files
yzhao244yzhao244
authored
Merge branch 'confident-ai:main' into json-parse-improvement
2 parents 91491b6 + 265bfbd commit e9b1448

1,215 files changed

Lines changed: 127620 additions & 43324 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.cursor-plugin/plugin.json‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"name": "deepeval",
3+
"displayName": "DeepEval",
4+
"version": "1.0.0",
5+
"description": "Skills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications.",
6+
"author": {
7+
"name": "Confident AI",
8+
"email": "founders@confident-ai.com"
9+
},
10+
"homepage": "https://deepeval.com",
11+
"repository": "https://github.com/confident-ai/deepeval",
12+
"license": "Apache-2.0",
13+
"keywords": [
14+
"deepeval",
15+
"llm",
16+
"evaluation",
17+
"tracing",
18+
"datasets",
19+
"confident-ai"
20+
],
21+
"category": "developer-tools",
22+
"skills": "./skills/"
23+
}

‎.github/workflows/changelog.yml‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
name: Generate Changelog
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
mode:
7+
description: "Mode: year or range"
8+
required: true
9+
default: "year"
10+
year:
11+
description: "Year (e.g. 2025)"
12+
required: false
13+
from_tag:
14+
description: "From tag (e.g. v3.7.0)"
15+
required: false
16+
to_tag:
17+
description: "To tag (e.g. v3.9.0)"
18+
required: false
19+
20+
jobs:
21+
changelog:
22+
runs-on: ubuntu-latest
23+
24+
permissions:
25+
contents: write
26+
pull-requests: write
27+
28+
steps:
29+
- name: Checkout repo
30+
uses: actions/checkout@v4
31+
with:
32+
fetch-depth: 0
33+
34+
- name: Fetch tags
35+
run: git fetch --tags --force
36+
37+
- name: Set up Python
38+
uses: actions/setup-python@v5
39+
with:
40+
python-version: "3.11"
41+
42+
- name: Install dependencies
43+
run: |
44+
pip install rich pydantic deepeval
45+
46+
- name: Run changelog generator
47+
env:
48+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
49+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
50+
run: |
51+
if [ "${{ github.event.inputs.mode }}" = "year" ]; then
52+
python .scripts/changelog/generate.py \
53+
--year ${{ github.event.inputs.year }} \
54+
--github --ai
55+
else
56+
python .scripts/changelog/generate.py \
57+
--range ${{ github.event.inputs.from_tag }} ${{ github.event.inputs.to_tag }} \
58+
--github --ai
59+
fi
60+
61+
- name: Create PR
62+
uses: peter-evans/create-pull-request@v6
63+
with:
64+
branch: chore/changelog-update
65+
title: "chore: update changelog"
66+
commit-message: "chore: update changelog"
67+
body: "Auto-generated changelog updates"

‎.github/workflows/full_test_core_for_pr.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,5 +102,5 @@ jobs:
102102
if: ${{ env.OPENAI_API_KEY != '' }}
103103
run: |
104104
poetry run pytest \
105-
tests/test_core/test_synthesizer/ tests/test_core/test_datasets/
105+
tests/test_core/test_synthesizer/ tests/test_core/test_datasets/ tests/test_core/test_simulator/
106106

‎.github/workflows/test_core.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -89,15 +89,15 @@ jobs:
8989
if: ${{ env.OPENAI_API_KEY != '' }}
9090
run: |
9191
poetry run pytest -vv -rA --maxfail=1 --capture=tee-sys -o faulthandler_timeout=300 \
92-
tests/test_core/test_synthesizer/ tests/test_core/test_datasets/
92+
tests/test_core/test_synthesizer/ tests/test_core/test_datasets/ tests/test_core/test_simulator/
9393
9494
# Dev tests (no secrets)
9595
- name: Run dev tests (no secrets)
9696
if: ${{ env.OPENAI_API_KEY == '' }}
9797
run: |
98-
poetry run pytest -vv -rA --maxfail=1 --capture=tee-sys tests/test_core/test_synthesizer/ tests/test_core/test_datasets/ \
98+
poetry run pytest -vv -rA --maxfail=1 --capture=tee-sys tests/test_core/test_synthesizer/ tests/test_core/test_datasets/ tests/test_core/test_simulator/ \
9999
--ignore=tests/test_core/test_tracing/test_dataset_iterator.py \
100100
--ignore=tests/test_core/test_synthesizer/test_context_generator.py \
101-
--ignore=tests/test_core/test_synthesizer/test_conversation_simulator.py \
101+
--ignore=tests/test_core/test_simulator/test_conversation_simulator.py \
102102
--ignore=tests/test_core/test_synthesizer/test_generate_from_goldens.py \
103103
--ignore=tests/test_core/test_synthesizer/test_synthesizer.py

0 commit comments

Comments
 (0)