Open
Conversation
Chore: Add RHOAI demo + reorganize existing demos
[pull] main from trustyai-explainability:main
LLS >=0.4.0 fixes
chore: Upgrade garak to 0.14.0
[pull] main from trustyai-explainability:main
sync: main -> incubation
…ider-trustyai-garak-konflux-replicator-latest-1770650635 Update Tekton files to version latest
Refactor Garak command configuration to use file-based --config
chore: Upgrade llama-stack to 0.5.0
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
fix get_shield for lls-05x + improve results + benchmark override
…ider-trustyai-garak-konflux-replicator-latest-1770880179 Update Tekton files to version latest
…ider-trustyai-garak-konflux-replicator-dev-1770931741 Update Tekton files to version dev
[pull] main from trustyai-explainability:main
sync upstream to midstream
…ax_concurrency) Made-with: Cursor
Add two new user-controllable SDG flow parameters alongside the existing sdg_max_concurrency: - sdg_num_samples: overrides RowMultiplierBlock.num_samples (rows per input row). 0 keeps the flow default. - sdg_max_tokens: overrides LLMChatBlock.max_tokens (token limit per request). 0 keeps the flow default. Flow blocks are found by block_name (not hardcoded index) via _override_flow_block(), so upstream flow reordering won't break overrides. Plumbed through all four execution modes: Llama Stack inline, Llama Stack remote KFP, EvalHub simple, and EvalHub KFP. Also merges main to resolve conflicts and restore disable_cache support. Made-with: Cursor
fixes [RHOAIENG-55875](https://redhat.atlassian.net/browse/RHOAIENG-55875)
…bility/fix-art-report fix: HTML report header on small screen
… constants, add plumbing tests - Move _safe_int to utils.py as safe_int(); remove duplicates from garak_adapter.py and garak_remote_eval.py - Add DEFAULT_SDG_NUM_SAMPLES_BLOCK_NAME and DEFAULT_SDG_MAX_TOKENS_BLOCK_NAME to constants.py so block names live alongside their defaults - Add tests verifying non-default sdg_max_concurrency/num_samples/max_tokens are plumbed through the KFP component to generate_sdg_dataset Made-with: Cursor
…bility/sdg-max-concurrency feat: expose SDG flow overrides (max_concurrency, num_samples, max_tokens) at runtime
Capture the return value of callbacks.mlflow.save() and store it as results.mlflow_run_id when truthy, making the MLflow run ID available to callers. Adds two tests covering the run ID being set and the None case. Made-with: Cursor
Made-with: Cursor
[pull] main from trustyai-explainability:main
Brings latest code changes from main into incubation including: - SDG max_concurrency, num_samples, and max_tokens runtime overrides - HTML report header fix for small screens - Updated eval-hub-sdk and garak versions - Updated SDG defaults - New test_sdg_params tests Preserves incubation's .tekton/ pipeline configuration as each branch has its own separate Konflux image build setup. Made-with: Cursor
Sync main to incubation clean
…bility/save-mlflow-rid
… termination The sidecar container was never shutting down after scan completion because _GarakCallbacks.report_results() never called _signal_termination(). This updates both the callbacks constructor and report_results to match the latest upstream DefaultCallbacks: - Constructor: use adapter.settings instead of raw os.getenv(), add benchmark_index, insecure, oci_proxy_host, termination_file_path, and mlflow_backend fields (mirrors DefaultCallbacks.from_adapter) - report_results: call _signal_termination(error) so the /shared/terminated file is written and the sidecar can detect completion - report_results: forward mlflow_run_id in the status event payload - report_results: track error string and pass to termination signal, split HTTPStatusError from generic Exception (matches upstream) Made-with: Cursor
Made-with: Cursor
…bility/sidecar-termination fix: align _GarakCallbacks with upstream DefaultCallbacks for sidecar termination
[pull] main from trustyai-explainability:main
sync: main to incubation
…-ubi9) Made-with: Cursor
Pre-commit and CI now call `make lock` instead of duplicating the uv pip compile command with the index URL. Future index changes only need to update RHAI_INDEX_URL in the Makefile. Made-with: Cursor
…bility/update-index chore: switch from test to prod RH AI PyPI index (cpu-ubi9-test → cpu-ubi9)
sync upstream -> midstream
sync: main to incubation
Made-with: Cursor
…bility/sidecar-on-error fix: enhance error handling and termination signaling in GarakAdapter
Let garak scans run to natural completion instead of killing them with artificial timeouts. All framework profile timeouts set to 0 (no limit); the subprocess runner treats 0 as None (block forever). Users can still set timeout_seconds in benchmark_config if they need a cap. Also removes soft_probe_prompt_cap=500 from every profile so garak uses its own defaults for prompt counts. Made-with: Cursor
- Guard timeout_seconds > 0 comparison against None to prevent TypeError - Add tests for negative values (fall through to next source) - Add tests for non-numeric values (ignored, fall through) Made-with: Cursor
…bility/remove-scan-timeout fix: remove hardcoded scan timeouts and soft_probe_prompt_cap overrides
sync: main to incubation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sync-branches: New code has just landed in incubation, so let's bring stable up to speed!