Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions .agents/skills/apm-integrations/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,16 +82,16 @@ Two ways to fetch the source locally:

1. **Shallow clone** the installed version:

```bash
git clone --depth 1 --branch v<x.y.z> https://github.com/<org>/<repo>.git /tmp/<lib>-versions/v<x.y.z>
```
```bash
git clone --depth 1 --branch v<x.y.z> https://github.com/<org>/<repo>.git /tmp/<lib>-versions/v<x.y.z>
```

2. **`npm pack`** when the published runtime artifact is what matters:

```bash
cd /tmp/<lib>-versions && npm pack <lib>@<x.y.z>
tar -xzf <lib>-<x.y.z>.tgz -C v<x.y.z> --strip-components=1
```
```bash
cd /tmp/<lib>-versions && npm pack <lib>@<x.y.z>
tar -xzf <lib>-<x.y.z>.tgz -C v<x.y.z> --strip-components=1
```

Read the file the wrap hooks, the base classes the hooked methods inherit from, and files the wrap doesn't currently touch — a public method, an internal channel, or a metadata field the current instrumentation skipped often gives a cleaner hook (e.g., kafka `cluster.brokerPool.metadata.clusterId`, couchbase `tracingChannel`).

Expand Down Expand Up @@ -156,20 +156,20 @@ For the complete list by base class, see [Reference Plugins](references/referenc
Follow these steps when creating or modifying an integration:

1. **Investigate** — Read the upstream library's source (see [Read Upstream Source First](#read-upstream-source-first)). Read 1-2 reference integrations of the same type (see table above). Understand the instrumentation and plugin patterns before writing code.
2. **Implement instrumentation** — Create the instrumentation in `packages/datadog-instrumentations/src/`. Use orchestrion for instrumentation.
2. **Implement instrumentation** — Create the instrumentation in `packages/datadog-instrumentations/src/`. Use orchestrion for instrumentation.
3. **Implement plugin** — Create the plugin in `packages/datadog-plugin-<name>/src/`. Extend the correct base class.
4. **Register** — Add entries in `packages/dd-trace/src/plugins/index.js`, `index.d.ts`, `docs/test.ts`, `docs/API.md`, and `.github/workflows/apm-integrations.yml`.
5. **Write tests** — Add unit tests and ESM integration tests. See [Testing](references/testing.md) for templates.
6. **Run tests** — Validate with:
```bash
# Run plugin tests (preferred CI command — handles yarn services automatically)
PLUGINS="<name>" npm run test:plugins:ci
```bash
# Run plugin tests (preferred CI command — handles yarn services automatically)
PLUGINS="<name>" npm run test:plugins:ci

# If the plugin needs external services (databases, message brokers, etc.),
# check docker-compose.yml for available service names, then:
# If the plugin needs external services (databases, message brokers, etc.),
# check docker-compose.yml for available service names, then:
docker compose up -d <service>
PLUGINS="<name>" npm run test:plugins:ci
```
PLUGINS="<name>" npm run test:plugins:ci
```
7. **Verify** — Confirm all tests pass before marking work as complete.

## Reference Files
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ async getStream() { /* returns Promise<ReadableStream> */ }
**When `kind: 'AsyncIterator'` is used, Orchestrion automatically creates TWO channels:**

1. **Base channel**: `tracing:orchestrion:{package}:{channelName}:*`
- Fires when the method is called (before iteration starts)
- Used to create the span
- Fires when the method is called (before iteration starts)
- Used to create the span

2. **Next channel**: `tracing:orchestrion:{package}:{channelName}_next:*`
- Fires on EACH iteration (`next()` call)
- Used to finish the span when `result.done === true`
- Fires on EACH iteration (`next()` call)
- Used to finish the span when `result.done === true`

## Critical Implementation Requirements

Expand Down
6 changes: 3 additions & 3 deletions .agents/skills/apm-integrations/references/orchestrion.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,9 +162,9 @@ This pattern is complex and easy to get wrong. The reference document covers:

1. Install the package: `npm install <package>`
2. Search for the method definition:
```bash
grep -r "methodName" node_modules/<package>/
```
```bash
grep -r "methodName" node_modules/<package>/
```
3. Use the path relative to the package root

**IMPORTANT: Patch both CJS and ESM code paths.** Many libraries duplicate their classes across separate CJS and ESM builds (e.g., `dist/cjs/client.js` and `dist/esm/client.js`). Each file path needs its own entry in the instrumentations array with the same `functionQuery` and `channelName`. If only one is patched, the instrumentation will silently fail for the other module format.
Expand Down
46 changes: 23 additions & 23 deletions .agents/skills/llmobs-integration/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,16 +63,16 @@ See [references/plugin-architecture.md](references/plugin-architecture.md) for t
Answer these questions by reading the code:

1. **Does the package make direct HTTP calls to LLM provider endpoints?**
- YES → Go to question 2
- NO → Go to question 3
- YES → Go to question 2
- NO → Go to question 3

2. **Does it support multiple LLM providers via configuration?**
- YES → **`LlmObsCategory.MULTI_PROVIDER`**
- NO → **`LlmObsCategory.LLM_CLIENT`**
- YES → **`LlmObsCategory.MULTI_PROVIDER`**
- NO → **`LlmObsCategory.LLM_CLIENT`**

3. **Does it implement workflow/graph orchestration with state management?**
- YES → **`LlmObsCategory.ORCHESTRATION`**
- NO → **`LlmObsCategory.INFRASTRUCTURE`**
- YES → **`LlmObsCategory.ORCHESTRATION`**
- NO → **`LlmObsCategory.INFRASTRUCTURE`**

See [references/category-detection.md](references/category-detection.md) for detailed heuristics and examples.

Expand Down Expand Up @@ -108,31 +108,31 @@ See [references/message-extraction.md](references/message-extraction.md) for pro
## Implementation Steps

1. **Detect package category** (REQUIRED FIRST STEP)
- Follow decision tree above
- Output: category, confidence, reasoning
- Follow decision tree above
- Output: category, confidence, reasoning

2. **Create plugin file**
- Location: `packages/dd-trace/src/llmobs/plugins/{integration}/index.js`
- Extend: `LLMObsPlugin` base class
- Implement: Required methods per plugin architecture
- Location: `packages/dd-trace/src/llmobs/plugins/{integration}/index.js`
- Extend: `LLMObsPlugin` base class
- Implement: Required methods per plugin architecture

3. **Implement `getLLMObsSpanRegisterOptions(ctx)`**
- Extract model provider and name from context
- Determine span kind (usually `'llm'`)
- Return registration options object
- Extract model provider and name from context
- Determine span kind (usually `'llm'`)
- Return registration options object

4. **Implement `setLLMObsTags(ctx)`**
- Extract input messages from `ctx.arguments`
- Extract output messages from `ctx.result`
- Extract token metrics (input_tokens, output_tokens, total_tokens)
- Extract metadata (temperature, max_tokens, etc.)
- Tag span using `this._tagger` methods
- Extract input messages from `ctx.arguments`
- Extract output messages from `ctx.result`
- Extract token metrics (input_tokens, output_tokens, total_tokens)
- Extract metadata (temperature, max_tokens, etc.)
- Tag span using `this._tagger` methods

5. **Handle edge cases**
- Streaming responses (if applicable)
- Error cases (empty output messages)
- Non-standard message formats
- Missing metadata
- Streaming responses (if applicable)
- Error cases (empty output messages)
- Non-standard message formats
- Missing metadata

See [references/plugin-architecture.md](references/plugin-architecture.md) for step-by-step implementation guide.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,16 +87,16 @@ Follow this tree to determine category:

```
1. Does the package make direct HTTP calls to LLM provider endpoints?
├─ YES → Go to question 2
└─ NO → Go to question 3
├─ YES → Go to question 2
└─ NO → Go to question 3

2. Does it support multiple LLM providers via configuration?
├─ YES → LlmObsCategory.MULTI_PROVIDER
└─ NO → LlmObsCategory.LLM_CLIENT
├─ YES → LlmObsCategory.MULTI_PROVIDER
└─ NO → LlmObsCategory.LLM_CLIENT

3. Does it implement workflow/graph orchestration with state management?
├─ YES → LlmObsCategory.ORCHESTRATION
└─ NO → LlmObsCategory.INFRASTRUCTURE
├─ YES → LlmObsCategory.ORCHESTRATION
└─ NO → LlmObsCategory.INFRASTRUCTURE
```

## Detection Process
Expand Down
36 changes: 18 additions & 18 deletions .agents/skills/llmobs-testing/references/assertion-helpers.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,34 +99,34 @@ assertLlmObsSpanEvent(events[0], {
## Best Practices

1. **Use MOCK_* for non-deterministic values:**
- Output text: `MOCK_STRING` (real responses vary)
- Token counts: `MOCK_NOT_NULLISH` (counts vary but should exist)
- Error objects: `MOCK_OBJECT` (error details vary)
- Output text: `MOCK_STRING` (real responses vary)
- Token counts: `MOCK_NOT_NULLISH` (counts vary but should exist)
- Error objects: `MOCK_OBJECT` (error details vary)

2. **Use exact values for inputs:**
- Input messages: You control these in tests
- Model parameters: You set these (temperature, max_tokens)
- Model name: You specify this
- Input messages: You control these in tests
- Model parameters: You set these (temperature, max_tokens)
- Model name: You specify this

3. **Always validate core fields:**
- `spanKind` (required for every span)
- `name` (operation identifier)
- `modelName` and `modelProvider` (for LLM spans)
- `spanKind` (required for every span)
- `name` (operation identifier)
- `modelName` and `modelProvider` (for LLM spans)

4. **Validate message format:**
- Ensure `{content: string, role: string}` structure
- Check role values: `'user'`, `'assistant'`, `'system'`, `'tool'`
- Ensure `{content: string, role: string}` structure
- Check role values: `'user'`, `'assistant'`, `'system'`, `'tool'`

5. **Test error paths:**
- Verify empty `outputMessages: [{content: '', role: ''}]` on errors
- Assert `error` field exists with `MOCK_OBJECT`
- Verify empty `outputMessages: [{content: '', role: ''}]` on errors
- Assert `error` field exists with `MOCK_OBJECT`

6. **Match span kind to operation:**
- Chat/completions → `spanKind: 'llm'`
- Workflow execution → `spanKind: 'workflow'`
- Agent runs → `spanKind: 'agent'`
- Tool calls → `spanKind: 'tool'`
- Embeddings → `spanKind: 'embedding'`
- Chat/completions → `spanKind: 'llm'`
- Workflow execution → `spanKind: 'workflow'`
- Agent runs → `spanKind: 'agent'`
- Tool calls → `spanKind: 'tool'`
- Embeddings → `spanKind: 'embedding'`

## Reference Test Implementation

Expand Down
13 changes: 13 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,16 @@ end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[*.avro]
charset = unset
end_of_line = unset
insert_final_newline = unset

# TODO: fix OpenAI tests
[packages/datadog-plugin-openai/test/fine-tune.jsonl]
insert_final_newline = unset

# Streamed verbatim and asserted byte-for-byte by the response-blocking test.
[packages/dd-trace/test/appsec/streamtest.txt]
insert_final_newline = false
17 changes: 17 additions & 0 deletions .editorconfig-checker.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"Verbose": false,
"Debug": false,
"IgnoreDefaults": false,
"SpacesAfterTabs": false,
"NoColor": false,
"Exclude": [
"^\\.nyc_output/",
"^LICENSE",
"^coverage/",
"^packages/dd-trace/test/appsec/bad-formatted-rules\\.json$",
"^packages/dd-trace/test/llmobs/cassettes/",
"^vendor/"
],
"AllowedContentTypes": [],
"PassedFiles": []
}
6 changes: 3 additions & 3 deletions .github/actions/node/setup/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ runs:
maintenance) version=$(node_version 20) ;;
active) version=$(node_version 24) ;;
latest) version=${LATEST_VERSION:-$(node_version 26)}
# When a custom/nightly version is requested via latest,
# force tracer init past the engines upper-bound guard.
[ -n "$LATEST_VERSION" ] && echo "DD_INJECT_FORCE=true" >> "$GITHUB_ENV" || true ;;
# When a custom/nightly version is requested via latest,
# force tracer init past the engines upper-bound guard.
[ -n "$LATEST_VERSION" ] && echo "DD_INJECT_FORCE=true" >> "$GITHUB_ENV" || true ;;
*) version=$VERSION ;;
esac
echo "version=$version" >> "$GITHUB_OUTPUT"
Expand Down
4 changes: 2 additions & 2 deletions .github/selenium/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ RUN apt-get update && apt-get install -y curl git gnupg libatomic1 unzip wget \
RUN wget -q -O - https://dl.google.com/linux/linux_signing_key.pub \
| gpg --dearmor -o /usr/share/keyrings/google-chrome.gpg \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/google-chrome.gpg] http://dl.google.com/linux/chrome/deb/ stable main" \
> /etc/apt/sources.list.d/google-chrome.list \
> /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update \
&& apt-get install -y google-chrome-stable \
&& rm -rf /var/lib/apt/lists/*

# Install ChromeDriver matching the installed Chrome version
RUN CHROME_VER=$(google-chrome --version | awk '{print $3}' | cut -d. -f1-3) \
&& curl -sf https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json \
> /tmp/chrome-versions.json \
> /tmp/chrome-versions.json \
&& DRIVER_URL=$(CHROME_VER="$CHROME_VER" node -e " \
const d = JSON.parse(require('fs').readFileSync('/tmp/chrome-versions.json', 'utf8')); \
const prefix = process.env.CHROME_VER; \
Expand Down
4 changes: 2 additions & 2 deletions .github/vendored-dependencies.csv
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"aws-lambda-nodejs-runtime-interface-client","https://github.com/aws/aws-lambda-nodejs-runtime-interface-client/blob/v2.1.0/src/utils/UserFunction.ts","['Apache-2.0']","['Amazon.com Inc. or its affiliates']"
"is-git-url","https://github.com/jonschlinkert/is-git-url/blob/396965ffabf2f46656c8af4c47bef1d69f09292e/index.js#L9C15-L9C87","['MIT']","['Jon Schlinkert']"
"aws-lambda-nodejs-runtime-interface-client","https://github.com/aws/aws-lambda-nodejs-runtime-interface-client/blob/v2.1.0/src/utils/UserFunction.ts","['Apache-2.0']","['Amazon.com Inc. or its affiliates']"
"is-git-url","https://github.com/jonschlinkert/is-git-url/blob/396965ffabf2f46656c8af4c47bef1d69f09292e/index.js#L9C15-L9C87","['MIT']","['Jon Schlinkert']"
8 changes: 8 additions & 0 deletions .github/workflows/project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,14 @@ jobs:
- uses: ./.github/actions/install
- run: npm run lint

lint-editorconfig:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: ./.github/actions/node/latest
- uses: ./.github/actions/install
- run: npm run lint:editorconfig

release-scripts:
runs-on: ubuntu-latest
steps:
Expand Down
48 changes: 24 additions & 24 deletions .gitlab/benchmarks/container/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -27,35 +27,35 @@ RUN --mount=type=ssh ./install-base-deps.sh
# not fail the build: the bake steps below skip when a file is missing and runall.sh
# installs the per-PR pin at runtime regardless.
RUN raw=https://raw.githubusercontent.com/DataDog/dd-trace-js/master; \
curl -fsSL "$raw/benchmark/sirun/.sirun-version" -o /tmp/.sirun-version || true; \
curl -fsSL "$raw/packages/dd-trace/test/plugins/versions/package.json" -o /tmp/versions.json || true
curl -fsSL "$raw/benchmark/sirun/.sirun-version" -o /tmp/.sirun-version || true; \
curl -fsSL "$raw/packages/dd-trace/test/plugins/versions/package.json" -o /tmp/versions.json || true

RUN if [ -s /tmp/.sirun-version ]; then \
read -r SIRUN_VERSION SIRUN_SHA256 < /tmp/.sirun-version \
&& wget -O sirun.tar.gz "https://github.com/DataDog/sirun/releases/download/v${SIRUN_VERSION}/sirun-v${SIRUN_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
&& echo "${SIRUN_SHA256} sirun.tar.gz" | sha256sum -c - \
&& tar -xzf sirun.tar.gz \
&& rm sirun.tar.gz \
&& mv sirun /usr/bin/sirun \
&& printf '%s' "$SIRUN_VERSION" > /opt/baked-sirun-version; \
fi
read -r SIRUN_VERSION SIRUN_SHA256 < /tmp/.sirun-version \
&& wget -O sirun.tar.gz "https://github.com/DataDog/sirun/releases/download/v${SIRUN_VERSION}/sirun-v${SIRUN_VERSION}-x86_64-unknown-linux-musl.tar.gz" \
&& echo "${SIRUN_SHA256} sirun.tar.gz" | sha256sum -c - \
&& tar -xzf sirun.tar.gz \
&& rm sirun.tar.gz \
&& mv sirun /usr/bin/sirun \
&& printf '%s' "$SIRUN_VERSION" > /opt/baked-sirun-version; \
fi

# Node.js majors the benchmark matrix runs (skip 18/22); exact patch from the manifest.
RUN mkdir -p /usr/local/nvm \
&& wget -q -O /tmp/nvm-install.sh https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh \
&& echo "4b7412c49960c7d31e8df72da90c1fb5b8cccb419ac99537b737028d497aba4f /tmp/nvm-install.sh" | sha256sum -c - \
&& bash /tmp/nvm-install.sh \
&& rm /tmp/nvm-install.sh \
&& . $NVM_DIR/nvm.sh \
&& if [ -s /tmp/versions.json ]; then \
for major in 20 24 26; do \
version=$(sed -n "s/.*\"node-${major}\": *\"npm:node@\([0-9.]*\)\".*/\1/p" /tmp/versions.json); \
[ -n "$version" ] || { echo "No node-${major} pin in versions manifest" >&2; exit 1; }; \
nvm install --no-progress "$version"; \
done \
&& nvm alias default 24 \
&& nvm use 24; \
fi
&& wget -q -O /tmp/nvm-install.sh https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh \
&& echo "4b7412c49960c7d31e8df72da90c1fb5b8cccb419ac99537b737028d497aba4f /tmp/nvm-install.sh" | sha256sum -c - \
&& bash /tmp/nvm-install.sh \
&& rm /tmp/nvm-install.sh \
&& . $NVM_DIR/nvm.sh \
&& if [ -s /tmp/versions.json ]; then \
for major in 20 24 26; do \
version=$(sed -n "s/.*\"node-${major}\": *\"npm:node@\([0-9.]*\)\".*/\1/p" /tmp/versions.json); \
[ -n "$version" ] || { echo "No node-${major} pin in versions manifest" >&2; exit 1; }; \
nvm install --no-progress "$version"; \
done \
&& nvm alias default 24 \
&& nvm use 24; \
fi

RUN mkdir /opt/insecure-bank-js
RUN git clone --depth 1 https://github.com/hdiv/insecure-bank-js.git /opt/insecure-bank-js
Expand Down
6 changes: 3 additions & 3 deletions .gitlab/benchmarks/container/install-base-deps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
set -ex

apt-get update && apt-get install --no-install-recommends -y \
wget curl ca-certificates valgrind \
git openssh-client hwinfo jq procps \
software-properties-common build-essential libnss3-dev
wget curl ca-certificates valgrind \
git openssh-client hwinfo jq procps \
software-properties-common build-essential libnss3-dev

# Prebuilt, relocatable CPython from python-build-standalone (the same builds uv ships).
# Avoids a slow from-source pyenv compile and stays version-pinned + checksum-verified.
Expand Down
Loading
Loading