Skip to content

Commit d455271

Browse files
oxesoftantonio-amjrrquidutegemini-code-assist[bot]ccruzagralopes
authored
v2.15 CLI release: merge develop into main (#86)
* [Fix] Handling Empty Usage Tag In CLI's PICS Parser (#43) * PICS parser logic update: verifying if the 'usage' tag exists, ignoring if not. * Gemini's suggestion: Improving conditional logic * Removing unnecessary attribution line * Fix _send_prompt_response calls and added timeout for large upload (#44) * Fix _send_prompt_response calls and added timeout for large upload * Update th_cli/test_run/prompt_manager.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review - Gemini * Added constants values --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Show friendly message to unsupported TWO WAY TALK test execution using CLI (#46) * Add Checking for ffmpeg and improve abort-testing command * Webrtc working * Changes related to two way talk * Updated ffmpeg parameters * Deleted unwated config file * changes after black command * Update th_cli/commands/abort_testing.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update th_cli/th_utils/ffmpeg_converter.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update th_cli/test_run/websocket.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review gemini * Code review gemini * Removed extra space in warning message from shared_constant file --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Fixing the CLI Run Test Logic to handle project config and user config (#47) * Fixing the CLI Run Test Logic to handle project config and user config * Gemini Suggestions: Fixing Path creation and adding except print to the user. * Removed Error column from test execution list (#49) * [Feature] Adding the Node ID to CLI Output (#48) * Adding the Node ID to CLI output * Updating ChipServerInfo attribute type Plus minor fixes * Fix cli unit tests not working (#50) * Fix CLI unit tests not working * Removed Error column from test execution list (#49) * Updated conftest and models * Update th_cli/commands/test_run_execution.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [FIX] CLI PICS Parser And Prompt (#51) * Adding Message Prompt Request type and fixing PICS XML parser * Fixing some flake8 issues * Fixing isort issue * Gemini's Suggestion: Remove unnecessary timeout exception logic * Fix test_run_execution command (#52) * Fix test_run_execution command * Update tests/test_test_run_execution.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update tests/test_test_run_execution.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update th_cli/commands/test_run_execution.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Added support to project-id in test_run_execution command * Black formating * Code review * Code review * Code review * code review --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Feature] Printing Manual Pairing Code (#53) * Printing the Manual Pairing Code to CLI's test execution output * Gemini Suggestion: Improving logic and fixing f strings * Fixing comment Adding missing parameter for the method's comment Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> --------- Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> * Adding the test parameters to the models to be used in the CLI's project config (#55) Also, the logic to print the chipserver info was fixed to always show the Node ID, even without the manual pairing code * Enhanced the th-cli available-tests command with new formatting options (#54) * Improve CLI available-tests command * Code formating * Minor code changes * Code review - gemini code assistant * Increase websocket message size (#56) * Increase websocket message size * Create constant for WEBSOCKET_MAX_MESSAGE_SIZE * Improve FFmpeg Error Handling in Video Stream Tests (#57) * Add message for ffmpeg not found * Code review - gemini * Code review * Code review * [Feature] Swap CLI Properties Config With JSON (#58) * Swapping the Properties config file with the JSON format * Fixing lint alerts * New Line At File's End Gemini Suggestion: adding a new line to the default_config.json file as per recommendation. This can prevent som e text processing tools errors. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Implement CLI Support for Push AV Tests (#59) * Initial code * Stable version * Added unit tests * Minor changes * Code formating and isort * Updated unit test * Code review * Add comment regarding disabling SSL verification * Removed unused import * Removed unused imports * Minor changes * Fixed f-string * Removed unecessary init files * Code review - minor changes * [Feature] Double-Dash Extra Arguments (#60) * Implement the double dash extra arguments feature Also including the Unit Tests * Fixing ouput message * Improving parser logic * Fixing lint errors * Improving parser logic for subsequent -- * Fixing the README Notes * Fixing CLI run_tests command conditional for extra arguments * Fixing ChipServer info display when using no project ID or config options (#61) * Fix WebSocket close handshake error for long-running test executions (#63) * Fix WebSocket close handshake error for long-running test executions in the CLI * Update th_cli/test_run/websocket.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [FIX] Updating CLI API Client Generator (#62) * Updating and Fixing CLI API Client Generator Updating tool version, files, scripts to support ARM64 hosts and generate successfully * Re-enable black formatter to fix several files structure * Adding postprocessing script to fix return type of api_response.py file Also, re-enabling the formatters in the end of the generation * Returning the TimeoutException logic but now adding the import accordingly * Updating shell scripts to handle host architecture type and new flag options * Fixing licence header generation with mustache files * Added NFC_WIFI and THREAD_MESHCOP * [Fix] CLI Project Create and Default Config (#67) * Fixing default config file and usage for project creation/update Also, fixing script file extension for run_pytest * Gemini Suggestion: Adding new line at end of file * Update regex pattern for test ID validation (#66) Edited the regex pattern, so that Tests such as TC-I-XXX work with the CLI, which threw an error before. * [Feature] Replace OpenAPI Generator with Datamodel-Code-Generator (#65) * New CLI API Client Generator Exchanging OpenAPI-Generator with Datamodel-Code-Generator * Removing Openapi-generator files and adding README * Fixing parameter placeholder from project command * Fixing the Project Update command to handle empty names * Fixing Project Update Unit Tests * Fixing script's README file * Project update improvements and minor fixes * Implementing some TODOs with missing logic for headers and multipart fields (file or data) * Fixing the README title * [Feature] Refactoring CLI Project Command (#68) * Updating CLI project command to use Click's group feature with subcommads * Simplifying logic using a SyncAPI generator and a contextmanager decorator * Fix test case comparison logic (#69) * Fix test case comparison logic * Update th_cli/utils.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Rever code formating --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Add missing image_handler.py and image_verification.html for snapshot camera verification (#71) * Fixed Screenshot camera verification * Fixed copyright year * Code review Gemini-code-assist * Code review Gemini-code-assist * Fix regex pattern to support additional valid test ID formats (#73) * Fix regex to support single-segment and underscore-category test IDs (TC-I-X, TC-BR-X, TC-MCORE_FS-X.X) * Update tests/test_validation.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Feature/909 Improve CLI unit tests code coverage for camera files (#72) * Fixed Screenshot camera verification * Fixed copyright year * Code review Gemini-code-assist * Code review Gemini-code-assist * Improve CLI test coverage across camera files and minor static analysis violations * Undo cov-fail-under change * black formating * [FIX] Run_Tests Command Configuration Logic (#74) * Fixing run_tests command configuration logic * Gemini review: improving logic * CLI to support two way talk tests (#75) * Initial implementation to support two way talk * Some code improvements * Impreved the user experience by only resume with the test case execution only when the CLI detects that the browser was opened * Improved code comments * Update th_cli/test_run/camera/two_way_talk_handler.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update th_cli/commands/run_tests.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Code review - gemini code assist::Use dependency injection to pass the handler instance through the call stack * Code review --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Feature] Add PICS Parameter To CLI Projects (#78) * Adding pics parameters to CLI's project commands * Fixing project update unit test * Updating run-test command to use the configured pics with none was provided * Fetching default project when no project ID was provided * Improving project manipulation in run-test command * [Fix] Project Create Command With No PICS (#79) * Fixing project create command when PICS is not provided * Gemini Suggestion: Removing unecessary global variable. Using PICS object directly * [FIX] Updating CLI To Handle Temporary PICS (#80) * Updating CLI to handle PICS during execution to be temporary (not persisted to the project in DB) * Fixing else conditional from run_tests.py file * Fixing openapi.json Create Cli Test Run Execution description * [FEATURE] Update CLI with grouped logs download feature (#82) * Update CLI with grouped logs download feature * allow endpoints to return binary data * Fix(cli): unique per-run log filename with timestamp + mode=w (#984) (#83) --------- Co-authored-by: antonio-amjr <116589331+antonio-amjr@users.noreply.github.com> Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com> Co-authored-by: Antonio Melo Jr. <a_junior@apple.com> Co-authored-by: Romulo Quidute Filho <rquidute@apple.com> Co-authored-by: ali gruenhaupt <yesilbas.ma@gmail.com> Co-authored-by: Steven Green <sgreen@csa-iot.org> Co-authored-by: abhisheksingh-esp <abhishek.singh@espressif.com>
1 parent 79f69c3 commit d455271

242 files changed

Lines changed: 14834 additions & 12295 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 69 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<!--
22
*
3-
* Copyright (c) 2023 Project CHIP Authors
3+
* Copyright (c) 2026 Project CHIP Authors
44
*
55
* Licensed under the Apache License, Version 2.0 (the "License");
66
* you may not use this file except in compliance with the License.
@@ -59,17 +59,83 @@ Run `th-cli available-tests` to get a list of tests available in Test Harness, p
5959

6060
### run-tests
6161

62-
Run `th-cli run-tests --tests-list <tests> [--title, -n <title>] [--config, -c <config>] [--pics-config-folder, -p <pics-config-folder>] [--project-id <ID>] [--no-color]` to start a new test execution.
62+
Run `th-cli run-tests --tests-list <tests> [--title, -n <title>] [--config, -c <config>] [--pics-config-folder, -p <pics-config-folder>] [--project-id <ID>] [--no-color] [-- <extra-sdk-args>]` to start a new test execution.
6363

6464
Required:
6565
- `--tests-list`: Comma-separated list of test case identifiers (e.g. --tests-list TC-ACE-1.1,TC_ACE_1_3)
6666

6767
Optional:
6868
- `--title`: Custom title for the test run. If not provided, the current timestamp will be used as the default.
69-
- `--config`: Path to the property config file. If not specified, default_config.properties will be used.
69+
- `--config`: Path to the JSON config file. **Accepts two formats:**
70+
- **Config-only**: `{"network": {...}, "dut_config": {...}}`
71+
- **Full project format**: `{"name": "...", "config": {...}}` (automatically extracts config)
72+
73+
This allows the same JSON file to work with `project create`, `project update`, and `run-tests` commands. If not provided, the project's default configuration will be used.
7074
- `--pics-config-folder`: Path to the folder that contains PICS files. If not specified, no PICS file will be used.
7175
- `--project-id`: Project ID that this test run belongs to. If not provided, uses the default 'CLI Execution Project' in TH.
7276
- `--no-color`: Disable all colors from the CLI's output text of this test run execution
77+
- `-- <extra-sdk-args>`: Pass additional arguments directly to the SDK container Python tests. Use the double dash (`--`) separator followed by any SDK test arguments. These arguments will be added to every Python test execution in the run.
78+
79+
**Example config files:**
80+
81+
Config-only format:
82+
```json
83+
{
84+
"network": {
85+
"wifi": {"ssid": "MyNetwork", "password": "MyPassword"}
86+
},
87+
"dut_config": {
88+
"pairing_mode": "ble-wifi",
89+
"setup_code": "20202021"
90+
}
91+
}
92+
```
93+
94+
Full project format (works with all commands):
95+
```json
96+
{
97+
"name": "My Project",
98+
"config": {
99+
"network": {
100+
"wifi": {"ssid": "MyNetwork", "password": "MyPassword"}
101+
},
102+
"dut_config": {
103+
"pairing_mode": "ble-wifi",
104+
"setup_code": "20202021"
105+
}
106+
}
107+
}
108+
```
109+
110+
**Passing Extra Arguments to SDK Tests:**
111+
112+
You can pass additional arguments directly to the SDK container test execution using the `--` separator. Everything after `--` will be passed as-is to the Python test runner.
113+
114+
Examples:
115+
```bash
116+
# Enable trace logging for all tests in the run
117+
th-cli run-tests -t TC-ACE-1.1 -- --trace-to json:log
118+
119+
# Pass boolean argument
120+
th-cli run-tests -t TC-ACE-1.1,TC-ACE-1.2 -- --bool-arg flag:true
121+
122+
# Pass multiple extra arguments
123+
th-cli run-tests -t TC-ACE-1.1 -- --timeout 60 --int-arg some-arg:1 --bool-arg flag:true
124+
125+
# Combine with other CLI options
126+
th-cli run-tests -t TC-ACE-1.1 --config my-config.json --no-color -- --trace-to json:log
127+
```
128+
129+
Common SDK test arguments you might want to use for example:
130+
- `--endpoint <value>` - Choose the device endpoint
131+
- `--trace-to json:log` - Enable detailed trace logging
132+
- `--timeout <seconds>` - Override default test timeout
133+
- `--int-arg <name>:<value>` - Pass integer argument to test
134+
- `--bool-arg <name>:<true|false>` - Pass boolean argument to test
135+
- `--string-arg <name>:<value>` - Pass string argument to test
136+
- `--hex-arg <name>:<hex-value>` - Pass hex value argument to test
137+
138+
**Note:** These extra arguments are applied to ALL Test Cases in the test run. Invalid arguments could cause test failures, so ensure the arguments are valid for the SDK test framework.
73139

74140
### test-run-execution-history
75141

client_generator/.gitignore

Lines changed: 0 additions & 17 deletions
This file was deleted.

client_generator/Dockerfile

Lines changed: 0 additions & 25 deletions
This file was deleted.

client_generator/Dockerfile.new.new

Whitespace-only changes.

client_generator/LICENSE.TXT

Lines changed: 0 additions & 192 deletions
This file was deleted.

client_generator/LICENSE.TXT.new.new

Whitespace-only changes.

0 commit comments

Comments
 (0)