Skip to content

Commit cd826c2

Browse files
authored
Merge pull request #1 from braintrustdata/ark/update-readme-for-release
update README for release
2 parents 17a66a7 + 8dfd944 commit cd826c2

1 file changed

Lines changed: 90 additions & 29 deletions

File tree

README.md

Lines changed: 90 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,110 @@
1-
# Braintrust Java SDK
1+
# Braintrust Java Tracing & Eval SDK
22

3-
An OpenTelemetry-based Braintrust SDK for Java 17 and up. Contains:
3+
[![javadoc](https://javadoc.io/badge2/dev.braintrust/braintrust-sdk-java/javadoc.svg)](https://javadoc.io/doc/dev.braintrust/braintrust-sdk-java)
4+
[![CI](https://github.com/braintrustdata/braintrust-sdk-java/actions/workflows/ci.yml/badge.svg)](https://github.com/braintrustdata/braintrust-sdk-java/actions/workflows/ci.yml)
45

5-
- Instrumentation for major AI vendors (OpenAI, Anthropic, etc)
6-
- LLM Eval framework for sending Experiments to Braintrust
7-
- Utils for sending Open Telemetry data to Braintrust
6+
## Overview
87

9-
If you're simply looking to call the Braintrust API with Java code, see [https://github.com/braintrustdata/braintrust-java](https://github.com/braintrustdata/braintrust-java)
8+
This library provides tools for **evaluating** and **tracing** AI applications in [Braintrust](https://www.braintrust.dev). Use it to:
109

11-
This SDK is currently is in BETA status and APIs may change
10+
- **Evaluate** your AI models with custom test cases and scoring functions
11+
- **Trace** LLM calls and monitor AI application performance with OpenTelemetry
12+
- **Integrate** seamlessly with OpenAI, Anthropic, and other LLM providers
1213

13-
<!-- # Using the SDK in your code -->
14+
This SDK is currently in BETA status and APIs may change.
1415

15-
<!-- *NOTE: The SDK has not published a release to maven yet. This section will not work until our first release is published* -->
16+
## See Also
1617

17-
<!-- build.gradle -->
18-
<!-- ```gradle -->
19-
<!-- dependencies { -->
20-
<!-- implementation 'dev.braintrust:sdk:0.0.1' -->
21-
<!-- } -->
22-
<!-- ``` -->
18+
If you're looking to call the Braintrust REST API with Java code, see the [Braintrust API Client](https://github.com/braintrustdata/braintrust-java)
2319

24-
# Examples
20+
## Quick Start
2521

26-
All examples can be found here: [./examples/src/main/java/dev/braintrust/examples](./examples/src/main/java/dev/braintrust/examples)
22+
Add the SDK to your package manager. Latest version and full instructions can be found in [Maven Central](https://central.sonatype.com/artifact/dev.braintrust/braintrust-sdk-java/versions)
2723

28-
To run the examples from the command line you need:
29-
- A Braintrust account and a valid BRAINTRUST_API_KEY
30-
- Java 17 (or greater):
31-
- macOS: `brew install openjdk@17`
32-
- Ubuntu: `sudo apt install openjdk-17-jdk`
33-
- (Optional to run the oai example) a valid OPENAI_API_KEY
34-
35-
Then, from the repo root:
24+
build.gradle example:
25+
```gradle
26+
dependencies {
27+
implementation 'dev.braintrust:sdk:<version-goes-here>'
28+
}
3629
```
37-
./gradlew :examples:runSimpleOpenTelemetry
30+
31+
### Evals
32+
33+
```java
34+
var config = BraintrustConfig.fromEnvironment();
35+
var openTelemetry = BraintrustTracing.of(config, true);
36+
var openAIClient = BraintrustOpenAI.wrapOpenAI(openTelemetry, OpenAIOkHttpClient.fromEnv());
37+
38+
Function<String, String> getFoodType =
39+
(String food) -> {
40+
var request =
41+
ChatCompletionCreateParams.builder()
42+
.model(ChatModel.GPT_4O_MINI)
43+
.addSystemMessage("Return a one word answer")
44+
.addUserMessage("What kind of food is " + food + "?")
45+
.maxTokens(50L)
46+
.temperature(0.0)
47+
.build();
48+
var response = openAIClient.chat().completions().create(request);
49+
return response.choices().get(0).message().content().orElse("").toLowerCase();
50+
};
51+
52+
var eval =
53+
Eval.<String, String>builder()
54+
.name("java-eval-x-" + System.currentTimeMillis())
55+
.tracer(BraintrustTracing.getTracer(openTelemetry))
56+
.config(config)
57+
.cases(
58+
EvalCase.of("asparagus", "vegetable"),
59+
EvalCase.of("banana", "fruit"))
60+
.task(getFoodType)
61+
.scorers(
62+
Scorer.of(
63+
"fruit_scorer",
64+
result -> "fruit".equals(result) ? 1.0 : 0.0),
65+
Scorer.of(
66+
"vegetable_scorer",
67+
result -> "vegetable".equals(result) ? 1.0 : 0.0))
68+
.build();
69+
var result = eval.run();
70+
System.out.println("\n\n" + result.createReportString());
3871
```
3972

40-
If you wish to see all examples run:
73+
### OpenAI Tracing
74+
75+
```java
76+
var braintrustConfig = BraintrustConfig.fromEnvironment();
77+
var openTelemetry = BraintrustTracing.of(braintrustConfig, true);
78+
OpenAIClient openAIClient = BraintrustOpenAI.wrapOpenAI(openTelemetry, OpenAIOkHttpClient.fromEnv());
79+
80+
var request =
81+
ChatCompletionCreateParams.builder()
82+
.model(ChatModel.GPT_4O_MINI)
83+
.addSystemMessage("You are a helpful assistant")
84+
.addUserMessage("What is the capital of France?")
85+
.temperature(0.0)
86+
.build();
87+
# openai calls will be automatically traced and reported to braintrust
88+
var response = openAIClient.chat().completions().create(request);
4189
```
90+
91+
## Running Examples
92+
93+
Example source code can be found [here](./examples/src/main/java/dev/braintrust/examples)
94+
95+
```bash
96+
export BRAINTRUST_API_KEY="your-braintrust-api-key"
97+
export OPENAI_API_KEY="your-oai-api-key" # to run oai examples
98+
export ANTHROPIC_API_KEY="your-anthropic-api-key" # to run anthropic examples
99+
# install java 17 or later
100+
brew install openjdk@17 # macOS
101+
sudo apt install openjdk-17-jdk # ubuntu
102+
# to run a specific example
103+
./gradlew :examples:runSimpleOpenTelemetry
104+
# to see all examples
42105
./gradlew :examples:tasks --group="Braintrust SDK Examples"
43106
```
44107

45-
If you wish to hack around with the examples you can modify source then re-run the gradle task.
46-
47108
## Logging
48109

49110
The SDK uses a standard slf4j logger and will use the default log level (or not log at all if slf4j is not installed).

0 commit comments

Comments
 (0)