[pull] master from DataDog:master - #441
Merged
Merged
Conversation
* Add workflow to trigger wheel releases This workflow triggers automated wheel builds in the private agent-integration-wheels-release repo via repository_dispatch. For testing: can be triggered manually with workflow_dispatch Future: will trigger on tag pushes (e.g., postgres-23.2.0) * Fix code injection vulnerability in trigger workflow The integrations input was directly interpolated into JavaScript, allowing code injection via crafted workflow_dispatch inputs. Fix: Pass values via environment variables and JSON.parse() to prevent script injection attacks. Security: Prevents arbitrary code execution with INTEGRATION_RELEASE_TOKEN in scope. * Use dd-octo-sts for secure cross-repo authentication Replace INTEGRATION_RELEASE_TOKEN PAT with OIDC-based authentication. Benefits: - No long-lived secrets stored in GitHub - Short-lived tokens (1 hour) - Audit trail built-in - Access controlled by trust policies in target repo The token is obtained from dd-octo-sts-action and used to trigger repository_dispatch events in agent-integration-wheels-release. * Pin actions/github-script to specific commit SHA Pin actions/github-script@v7 to commit 60a0d83 (v7.0.1) for supply chain security and reproducibility. * Add pull_request trigger for testing Allow workflow to run on pull requests targeting master or the test branch. When triggered by a PR, uses test integrations (postgres, mysql) and builds from the PR's head SHA. * Add id-token permission for OIDC authentication Required for dd-octo-sts-action to generate OIDC tokens. * Update dd-octo-sts policy name Use integrations-core.dispatch-wheel-builds for more explicit naming * Use different dd-octo-sts policies for PR vs production - PR events: integrations-core.dispatch-wheel-builds-test - workflow_dispatch: integrations-core.dispatch-wheel-builds This allows testing in PRs while keeping production restricted. * trigger * Remove master from pull_request trigger branches Only allow PRs targeting dk/test-release-trigger to trigger the workflow. This prevents PRs targeting master from using the test trust policy, maintaining proper security boundaries. Master branch releases will use workflow_dispatch with the production trust policy instead. * Use workflow_dispatch for PR testing For pull_request events, use createWorkflowDispatch to target the release-integrations.yml workflow on the 01-28-add_release_workflow branch. This allows testing before the workflow is merged to main. For workflow_dispatch events (manual triggers), continue using repository_dispatch which will work once the workflow is on main. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Fix pull_request trigger for testing The pull_request.branches config specifies the target branch of PRs, not the source branch. Changed to trigger on PRs targeting master (from the test branch) and added a paths filter to only trigger when the workflow file itself is modified. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Add dispatch-release workflow with chunking support Created a reusable dispatch-release workflow that handles chunking and dispatching release workflow runs to agent-integration-wheels-release. This workflow: - Lives in integrations-core (not agent-integration-wheels-release) - Chunks integrations into batches (default: 200 per batch) - Dispatches multiple workflow runs for >200 integrations - Uses dd-octo-sts for secure cross-repo dispatch - Supports both test and production trust policies - Can be copied to integrations-extras and marketplace Example: 450 integrations → 3 runs of 200, 200, and 50 The trigger workflow now: - Extracts inputs (integrations, source ref, policy name) - Selects correct trust policy based on event type: - pull_request: integrations-core.dispatch-wheel-builds-test - workflow_dispatch: integrations-core.dispatch-wheel-builds - Calls local dispatch workflow with all parameters Benefits: - Scales to unlimited integrations (no 256 limit) - Simpler trigger workflow (removed 64 lines) - Each integration repo controls its own dispatch logic - No cross-repo workflow dependency - Secure: uses appropriate trust policy per event type 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Add chunking test with 4 integrations and batch size of 2 Updated PR testing to use 4 integrations with a chunk size of 2 to test the chunking functionality: - Test integrations: postgres, mysql, redis, nginx - Chunk size: 2 (for PRs only, production uses 200) - Expected behavior: 2 workflow runs with 2 integrations each This will verify that the dispatch workflow correctly: - Splits integrations into multiple chunks - Dispatches separate workflow runs for each chunk - Processes integrations in parallel across runs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * Rename workflow call * Rename redis to redisdb * Use windows_performance_counters and ibm_mq as test integrations Replace mysql and nginx with windows_performance_counters and ibm_mq for more representative E2E testing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Move logic to Python scripts * Improve logic separation * Rename trigger job * Add environment condition for workflow dispatch * Dispatch release even if nothing was tagged (use the HEAD tags) * fix(release): fix integration glob path and validate unknown integrations - Fix glob pattern: remove spurious src/ segment so integrations are found at */datadog_checks/*/__about__.py regardless of layout - Eagerly compute all_integrations once at the start of main() so it can be shared between the 'all' branch and the new validation check - Validate that every integration in a manually-provided JSON list exists in the repo; fail fast with a clear error before dispatching Rationale: The src/ segment in the glob caused get_all_integrations() to return an empty list on repos that don't use that layout, silently building nothing. The validation prevents typos or stale integration names in manual workflow_dispatch invocations from producing a wheels-release run that references non-existent integrations. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * feat(release): batch integration dispatches - Split integrations into configurable batches (default 200 per batch) - Add BATCH_SIZE env var validation (must be positive integer) - Add batch-size workflow input to dispatch-release.yml - Print per-batch progress and full integration list before dispatching Rationale: Large releases can exceed repository_dispatch payload limits. Batching ensures each dispatch stays within GitHub's size constraints while keeping the full release atomic from the operator's perspective. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * Hard code batch size * Use dry-run mode for now * Add dry-run to trigger workflow * Change dry-run description * Replace dry-run with target * Add validation step to dispatch * Use sys.exit instead of raising error * Simplify version checking * feat(release): rename integration→package and add dry-run mode - Rename all references from "integration" to "package" throughout the release workflows and scripts to reflect that the pipeline can release any Python package, not only integrations - Rename detect_integrations.py → detect_packages.py and update all env var names (MANUAL_INTEGRATIONS → MANUAL_PACKAGES, INTEGRATIONS → PACKAGES, has_integrations → has_packages) and output keys throughout - Replace build-integrations repository_dispatch event with build-wheels and update the client_payload key accordingly - Add dry-run boolean input to trigger-release.yml and dispatch-release.yml; when true, tags are created locally but not pushed, and no wheel builds are triggered - Pass DRY_RUN env var to tag_releases.py and dispatch_release.py; both scripts now exit early with a summary when dry_run is true Rationale: The pipeline originally only released integration wheels, but it can also release other Python packages (e.g. datadog_checks_base). Renaming removes the confusing restriction implied by "integration". The dry-run flag lets operators preview what would be released — including which tags would be pushed and which downstream builds would fire — without any side-effects, which is essential for validating the pipeline before promoting to prod. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * fix(release): restore dry-run input to trigger-release.yml - Add missing dry-run boolean input (accidentally dropped in previous commit) - Pass dry-run through to dispatch-release.yml via the with: block This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * Add ddev-version input parameter * TEST trigger dry-run release Remove changelog.d from datadog_checks_base * Fix name detection * Add ddev configuration step * TEST Use test sts policy and delete datadog_checks_base changelog.d * TEST Use dry-run mode for testing * chore(release): improve detect_packages output verbosity - Print detection mode (all / manual / auto-detect) at startup - List each detected package by name after detection - Print explicit message when no packages are found Rationale: CI logs previously showed no indication of which packages were picked up or why, making it hard to verify the detection step worked as expected without digging into downstream steps. * chore(release): improve dispatch output and add run tracking link - Extract TARGET_REPO constant to avoid duplication between DISPATCH_URL and ACTIONS_URL - Print per-package list inside each batch instead of just the count - Print a direct link to the target repo's Actions page after each dispatch Rationale: After dispatching, there was no way to navigate to the triggered runs without manually opening GitHub. The per-package batch list also makes it easier to correlate which packages went into which dispatch call. * chore(release): improve validate_release output verbosity - Separate packages with no version found from pre-release packages - List skipped packages with their reason (no version / pre-release) - List each validated package with its version on success Rationale: The validation step previously only reported failure details. Seeing which packages passed and at what version makes it easy to confirm the right packages were validated without waiting for the dispatch step. * Docs changes * Add job summaries * chore(release): fix formatting issues and f-string inconsistencies - Include manual package value in mode string for consistency with 'all' mode - Store errors as (package, raw) tuples to avoid leaking error message into summary Package column - Move commit_url inline into source_link to eliminate unused variable - Move Track runs print to main() after batch loop so it appears once, not per batch Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Prevent printing released integrations twice * EXPERIMENTAL rework summary format * Revert TEST commits * refactor(release-scripts): modularize release pipeline scripts and add tests - Extract shared logic into _release package (github, packages, validation, summary, dispatch modules) - Fix has_changelog_fragments false positive: .gitkeep and README.md no longer trigger validation failure - Fix DRY_RUN parsing: replace `val != "false"` with explicit parse_bool_env() with safe default - Fix dispatch_release.py silent fallback: warn instead of silently using empty dict when validation file is missing - Eliminate duplicated build_summary() implementations across validate and dispatch scripts - Add 63 unit tests covering all business logic with no subprocess/env dependencies - Add test-release-scripts.yml CI workflow triggered on changes to these files Rationale: scripts had no test coverage, contained bugs not caught in dry runs, and had duplicated logic making them hard to maintain safely This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * fix(release-scripts): address post-review issues - Fix ImportError: _ACTIONS_URL was imported from dispatch.py but defined in summary.py; move it to dispatch.py (where it belongs) and import from there in summary.py - Fix stringly-typed status lookups in validate_release.py: replace raw strings "no_version"/"pre_release"/"ready" with NO_VERSION/PRE_RELEASE/READY constants to match HAS_FRAGMENTS which was already using a constant - Rename HAS_FRAGMENTS value from "error" to "has_fragments" so the constant name and its value are consistent - Fix TOCTOU in _load_validation: replace exists() + read_text() with try/except FileNotFoundError Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(release-scripts): trim redundant tests (63 → 48) Remove tests that cover symmetric or already-proven behavior: - parse_bool_env: collapse missing/empty/whitespace into one case - set_outputs/write_summary: drop trivial no-op-when-unset tests - has_changelog_fragments: drop duplicate fragment-detected and .gitkeep+fragment combo tests - get_version: drop redundant missing-package case (same path as missing-about) - TestBuildSummary: consolidate 8 label tests into one; extract helper to reduce repetition - resolve_packages: drop trivial empty-tags case - build_payload: drop serializable test (structure test already exercises the same) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(release-scripts): address review feedback - Fix trigger path: remove spurious /src/ segment so __about__.py changes actually fire the push trigger - Tag step now always runs; MANUAL_PACKAGES is passed through so ddev release tag is called with the explicit package list when provided, or 'all' when not — removes the conditional skip - Simplify detect step condition to steps.tag.outcome == 'success' now that tag always runs - Propagate dry_run from validation JSON in dispatch_release.py instead of re-reading from env; removes the dead stored field Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * TEST: add dry-run workflow triggered on pull_request Runs dispatch-release with packages=all and dry-run=true on every PR touching the release scripts or workflows. Remove once the pipeline is validated in prod. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert "TEST: add dry-run workflow triggered on pull_request" This reverts commit 4874de3. * fix(release-scripts): address review feedback - Add retry with exponential backoff (up to 5 attempts) on 5xx errors in send_dispatch - Move ACTIONS_URL to summary.py, removing cross-module private import - Fall back to DRY_RUN env var when validation JSON is missing in dispatch_release - Rename _MAX_RETRIES → _MAX_ATTEMPTS to match actual semantics - Improve exit-code-3 comment in tag_releases - Add TestSendDispatch covering 4xx/5xx/transient-5xx behaviour Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Update .github/workflows/test-release-scripts.yml Co-authored-by: NouemanKHAL <noueman.khalikine@datadoghq.com> * Improve ddev-version input parsing * Remove conftest * refactor(release-scripts): decouple dispatch functions from global constants Add batch_size, dispatch_url, and max_attempts as keyword-only parameters to send_dispatch and dispatch_in_batches, using the module-level constants as defaults. This allows callers and tests to inject values without patching globals. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(release-scripts): clean up dispatch_in_batches and build_summary signatures - Remove leftover `*` from build_summary (was artifact from a previous iteration) - Refactor dispatch_in_batches: validate batch_size, early-return on empty, pre-compute total_batches, track batch_num explicitly - Add tests for empty-packages no-op and invalid batch_size Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(release-scripts): add is_stable_release dispatch context - Rename READY → STABLE and add UNRELEASED constant (for 0.0.1 packages) - Change result dict shape from {status} to {type, dispatch} — dispatch is a bool that controls whether the package is actually sent downstream - Add is_stable_release bool parameter to validate_package/validate_packages; pre-release versions dispatch only on pre-release branches and stable versions dispatch only on stable branches - Add IS_STABLE_RELEASE env var parsing in validate_release.py; fail fast when stable versions appear on alpha/beta/rc branches - Add is-stable-release input to dispatch-release.yml so the workflow receives the branch context from the caller - Update summary.py (_TYPE_LABELS) and all tests for the new result shape Rationale: The pipeline needs to support building pre-release (alpha/beta/rc) wheels from feature branches while keeping stable-only dispatch on master/X.X.x. Encoding the branch context as a bool that flows through validate_package lets the same validation code serve both contexts without duplicating logic or adding a separate code path. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * refactor(trigger): replace Python branch detection with shell allowlist - Add alpha/*, beta/*, rc/* to push branch triggers so pre-release branches can trigger wheel builds - Replace checkout + detect_context.py step with a single shell expression using allowlist regex (master or X.X.x → true, else → false) - Remove detect_context.py (was untracked; logic now lives in the workflow) Rationale: The Python script was blocklist-based (alpha/beta/rc → false, everything else → true), which means any arbitrary feature branch would incorrectly be treated as a stable release context. The shell allowlist is simpler, requires no checkout, and correctly defaults to pre-release context for any branch that is not master or X.X.x. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * refactor(dispatch): add _urlopen wrapper for mockable HTTP boundary - Extract _urlopen thin wrapper around urllib.request.urlopen in dispatch.py - Update send_dispatch to call _urlopen instead of the stdlib function directly - Patch _release.dispatch._urlopen in all tests instead of stdlib urlopen - Replace MagicMock context manager with create_autospec(HTTPResponse) for type fidelity - Use call_args.args[0] and list comprehension for cleaner batch assertions Rationale: Tests should mock at the boundary we own, not deep inside stdlib. Patching urllib.request.urlopen directly couples tests to an implementation detail they don't control. The _urlopen wrapper makes the seam explicit and keeps the mock target stable if the underlying HTTP client ever changes. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * test(release-scripts): improve test clarity and parametrize bool env tests - Rewrite TestParseBoolEnv.test_true_values and test_false_values with pytest.mark.parametrize so each value gets its own test case - Rename TestResolvePackages.ALL attribute to all_packages (PEP 8 — not a module-level constant) - Pass all_packages as a keyword argument to resolve_packages for readability - Assert exact mode string in test_all_keyword instead of loose substring checks - Rename test_hyphenated_package_name to test_strips_only_version_suffix and remove inline comment (name is now self-describing) Rationale: PR review feedback flagged these tests as harder to debug than necessary. Parametrized tests produce per-value failure messages instead of stopping at the first failing iteration. The rename and exact assertion changes make failures immediately interpretable without reading the implementation. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * chore(workflows): pin pytest, set prod default, remove validated TODOs - Pin pytest to 8.3.5 in test-release-scripts.yml to prevent surprise breakage from unpinned installs - Change dispatch-release.yml target default from dev to prod and remove the TODO comment - Remove TODO comment from trigger-release.yml target line (pipeline is validated) Rationale: The pipeline has been validated in prod so the TODO comments are no longer accurate. Pinning pytest ensures the CI test job runs against a known-good version rather than whatever the latest release happens to be at run time. This commit made by [/dd:git:commit:atomic](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/atomic.md) * docs(workflows): add cross-repo comments to trigger and dispatch workflows - Note in trigger-release.yml that equivalent workflows exist in integrations-extras and marketplace - Note in dispatch-release.yml that it is a reusable workflow called by trigger-release from integration repos Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(packages): assert exact mode strings; pin ddev to 14.3.2 - Assert full mode string in test_json_array and test_auto_detect - Pin ddev default to 14.3.2 in dispatch-release.yml install step Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(dispatch-release): clarify ddev-version input is pinned by default Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(release-scripts): add tests for entry-point scripts * fix(release-scripts): write summary on stable-on-pre-release failure; minor cleanup - write_summary() now called before sys.exit(1) when stable packages are found on an alpha/beta/rc branch, matching the behaviour of the HAS_FRAGMENTS failure path - add target default comment in dispatch-release.yml to surface that callers typically override it - replace manual batch_num increment with enumerate(..., 1) in dispatch.py - remove stray double blank line in validation.py - update test to assert summary is written on stable-on-pre-release failure Rationale: stable-on-pre-release validation failure left the GitHub Step Summary empty, making it hard to diagnose the failure from the Actions UI This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * refactor(_release/summary): redesign label logic and rename dispatched param - Drop _TYPE_LABELS dict; extract _row_label() that enumerates every case - Rename build_summary param dispatched -> was_dispatched to eliminate naming collision with per-row dispatch field - Rename local will_dispatch -> eligible for clarity - Add explicit labels for STABLE-on-pre-release and PRE_RELEASE-on-stable symmetric cases (previously fell through to a false-positive "✅ Ready") - Add "✅ Validated" label for eligible packages when dispatch was blocked - Update callers (dispatch_release.py, validate_release.py) and tests - Add 3 new label tests covering the previously unhandled cases Rationale: the old catch-all "✅ Ready" in _TYPE_LABELS produced misleading green labels in error summaries; symmetric branch-mismatch cases had no distinct labels, making release summaries hard to interpret at a glance. This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * refactor(_release/summary): replace dict lookup with if-chain in _row_label - Replace .get(typ, fallback) dict literal with explicit if statements - Raise ValueError on unknown type instead of silently returning "⏭️ Skipped" Rationale: dict is allocated on every call for a single lookup; explicit if-chain is clearer, and an unknown type is a programmer error that should surface loudly rather than produce a misleading label. This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * test(_release): strengthen assertions and cover ValueError path - Assert actual env var values in test_successful_validation_writes_json instead of just checking key presence - Add test_unknown_type_raises to cover the ValueError raised by _row_label for unrecognized validation types Rationale: key-presence checks don't catch wrong values; the ValueError branch had no test coverage after the dict-to-if-chain refactor. This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * TEST test(workflows): add dry-run PR trigger for release pipeline smoke tests - New workflow fires on PRs to master that touch release scripts/workflows - Always runs with dry-run=true — no tags pushed, no builds triggered - Uses GITHUB_BASE_REF for stable-release detection (correct for PR events) - Remove environment: release gate from dispatch-release.yml for PR testing - Bump apache 7.3.0→7.3.1, nginx 9.3.0→9.3.1, snmp 12.3.0→12.3.1, kafka 4.3.0→4.3.1 Rationale: verify the end-to-end pipeline works on PRs without risking real dispatches to agent-integration-wheels-release. This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * Revert "TEST test(workflows): add dry-run PR trigger for release pipeline smoke tests" This reverts commit 50e0fb6. * test(_release): use full mock.call assertions instead of index-based call_args_list - Replace index-based call_args_list access with full mock.call comparisons - Extract _GIT_NAME/_GIT_EMAIL class constants to avoid repeating expected calls - Remove test_git_config_called_before_ddev (now redundant: test_push_flag asserts the full call list) Rationale: asserting complete call objects catches regressions in argument values, not just argument position; full call assertions are also easier to read. This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * Rename manual to selected * Use kwargs for resolve_pagages tests * test(_release): replace call_args access with full mock_calls assertions - Replace call_args.args[0] and call_args[0][1] with mock_calls comparisons - All dispatch call assertions now use complete call() objects Rationale: consistent with the full-call assertion style used elsewhere. This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * Fix test * fix(_release): address PR review findings for release trigger workflow - Fix MANUAL_PACKAGES → SELECTED_PACKAGES bug in dispatch-release.yml that silently broke manual package selection - Catch urllib.error.URLError in send_dispatch retry loop; network-level failures were bypassing all retry attempts - Replace sys.exit(1) in send_dispatch with DispatchError; in resolve_packages with ValueError — makes library functions testable and reusable outside CLI context - Fix typo selected_PACKAGES → SELECTED_PACKAGES in error message - Wrap CalledProcessError from get_tags_at_head with clean RuntimeError - Use Path(path).open() instead of bare open() in github.py for consistency with rest of _release/ module - Inline eligible label in build_summary, rename _row_label → _ineligible_label to clarify its contract - Catch json.JSONDecodeError in _load_validation for partial-write resilience - Update misleading is-stable-release comments in workflow files - Add sleep backoff assertions, URLError tests, retry-failure test Rationale: review identified a silent bug (MANUAL_PACKAGES), retry bypass for network errors, and sys.exit anti-patterns in library code This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * chore: remove .agint-review worktrees from index, add to .gitignore Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix .gitignore to include root.json * chore(_release): rename workflow files to release-* prefix - Rename dispatch-release.yml → release-dispatch.yml - Rename trigger-release.yml → release-trigger.yml - Update all internal cross-references in test-release-scripts.yml, release-trigger.yml, and release-dispatch.yml comments Rationale: align with the existing release-* naming convention used by other workflows in this repo (release-base.yml, release-dev.yml, release-hash-check.yml) This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * chore: remove .agint-review worktrees from index again These were accidentally staged by a prior git add -A during the rename commit. .gitignore already covers .agint-review/ but the index had them from a failed commit attempt. * refactor(_release): merge tag/detect/validate into release_prepare.py - Replace three sequential workflow steps (tag_releases, detect_packages, validate_release) with a single release_prepare.py script - Rename dispatch_release.py → release_dispatch.py for naming consistency - Update release-dispatch.yml: single 'prepare' step, all downstream steps.detect refs updated to steps.prepare - Merge corresponding test files into test_script_release_prepare.py and test_script_release_dispatch.py Rationale: eliminates GITHUB_OUTPUT round-trips between steps and centralises all release-prep logic in one place; the "skip validate if no packages" branch is now a plain Python if-statement instead of a YAML conditional reading back from disk This commit made by [/dd:git:commit:quick](https://github.com/DataDog/claude-marketplace/tree/main/dd/commands/git/commit/quick.md) * fix(_release): address PR review findings - Fix SSL errors treated as retriable in dispatch.py (raise immediately) - Normalize SELECTED_PACKAGES="[]" to auto-detect in release_prepare.py - Add diagnostic print before silent sys.exit after ddev tag retry - Make missing release_validation.json a hard error in release_dispatch.py - Add comment explaining dry_run data-provenance asymmetry - Fix parse_bool_env to use strict allowlist (unrecognised → default) - Add PackageValidationResult TypedDict to validation.py - Move TARGET_REPO constant to _release/__init__.py, remove duplicates - Add return type annotation to _urlopen - Rename nbr_packages → num_packages in dispatch_in_batches - Extract _stable_result to module level in test_script_release_prepare.py - Add TestMain.test_dry_run_propagates_to_validation_json - Add TestMain.test_validate_failure_exits_from_main Rationale: Addresses all review findings on PR #22656. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(_release): read dry_run from env var in release_dispatch.py - Remove dry_run from release_validation.json (prepare no longer writes it) - Read DRY_RUN directly from env in dispatch, same as SOURCE_REPO/REF/TARGET - Update tests to set DRY_RUN env var instead of embedding in JSON Rationale: Having dry_run sourced from the JSON was confusing — all other workflow inputs come from env vars injected by the YAML step. Keeping one value in the file while the rest come from env made the contract inconsistent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(_release): add PR-based dry-run trigger for release dispatch - Disable environment: release gate in release-dispatch.yml so the job can run in PR context without requiring environment approval - Add release-test-trigger.yml: fires on pull_request to this branch, invokes release-dispatch with dry-run=true for active_directory, activemq, and airflow (stable packages with no changelog fragments) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert "test(_release): add PR-based dry-run trigger for release dispatch" This reverts commit b91002a. --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: NouemanKHAL <noueman.khalikine@datadoghq.com>
* log source and provider and call pdb debugger * remove pdb call and source field * log both source and provider * log provider only * inject provider as property * add security module to read integration security agent configs * add secure_field property in the spec and the models * update SecurityConfig module, and improve security_field validation logic and model generation * rename secure_field to require_trusted_provider * improve models generation to be more concise * mark http and jmx filepath properties as protected * improve model generation to be more concise * generate new models * improve tests and fix security config default allowlist behavior * add fallback security validation in the base check * ddev validate config and models * add require_trusted_provider to the set of allowed value fields * code cleanup * ddev validate models -s * changelog * Fix path traversal and sibling-directory bypass in is_file_path_allowed Bare startswith allowed bypasses like /allowed-extra/ matching /allowed and /allowed/../etc/passwd traversing outside. Now resolves symlinks with os.path.realpath and enforces os.sep boundary checks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Make DEFAULT_TRUSTED_PROVIDERS immutable Change from mutable list to tuple to prevent accidental mutation of shared module-level state across SecurityConfig instances. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add type hints to core.py and utils.py Per AGENTS.md guidelines, new code should include type hints using modern syntax. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix tests models/spec to include tls_cert param * Catch ValueError in GLOBAL_SECURE_FIELDS fallback ValueError from check_field_trusted_provider was escaping the except ValidationError handler. Add explicit ValueError catch to wrap it as ConfigurationError. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove phantom certificate_path from GLOBAL_SECURE_FIELDS certificate_path doesn't exist in any integration spec. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing JMX fields to GLOBAL_SECURE_FIELDS Add java_bin_path, trust_store_path, key_store_path, and tools_jar_path for fallback coverage of JMX template fields. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Resolve allowlist paths in is_file_path_allowed Apply os.path.realpath() to allowlist entries so symlinks in the allowlist are resolved before comparison. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove dead branch in check_field_trusted_provider security_config cannot be None when the error raises, since validate_require_trusted_provider returns True for None. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Use list[str] in model_info.py Replace List[str] with modern list[str] syntax and remove unused typing import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add test coverage for fallback, allowlist, and excluded_checks Cover three security-critical code paths that had zero test coverage: GLOBAL_SECURE_FIELDS fallback blocking, allowlist bypass, and excluded_checks bypass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Assert ConfigurationError instead of Exception in tests Use specific ConfigurationError in pytest.raises to avoid masking unexpected exceptions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Match ConfigurationError in exception message instead of Exception dd_run_check wraps all errors as Exception, so match on ConfigurationError in the traceback string to verify the correct exception type is raised internally. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix regex patterns to match multiline exception messages Add (?s) dotall flag so ConfigurationError match works across newlines in dd_run_check's traceback output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add auth_token to test model and test for non-string secure field blocking Add an object-typed `auth_token` field with `require_trusted_provider: true` to the test spec/model and a test asserting that non-string secure fields are blocked from untrusted providers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Enforce trusted-provider checks for non-string secure fields Remove the `isinstance(value, str)` early return that let non-string values bypass validation. Non-string values (e.g. object-typed auth_token) from untrusted providers are now blocked; the allowlist escape is applied only to string file paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix validation for non-string fields such as auth_token, add more tests * ddev validate models -s * remove ValueError catch from base.py * fix fallback security validation placement * fix license headers year from 2024 to 2026 * address review: fix a bug where empty trusted_providers list would default to default providers * Revert all changes outside datadog_checks_base/ to origin/master Reset integration model regenerations, changelogs, and unrelated dependency changes so the branch only contains datadog_checks_base/ diffs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert datadog_checks_base models regeneration * address juanpe review --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Return early from Kafka consume_messages when partition end is reached Instead of waiting until the global timeout, break immediately when poll() returns None or when all assigned partitions report EOF. This makes the read_messages action fast when there are no more messages to consume. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add changelog entry for kafka_actions early return fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Simplify early return tests to 2 concise tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Remove enable.partition.eof and EOF tracking — just break on None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fleak pass * map query string * matching OCSF standard * fixing process restraints * align test output * changed pid to unknown * align test output * jump forward timestamp to pass CI test * align test output * more precise activity filtering * format yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )