-
Notifications
You must be signed in to change notification settings - Fork 2
Description
Description
The auto-enable feature for raw_api_responses
in execute_stream_test_read
is not working as documented when zero records are extracted due to incorrect dpath configuration.
Expected Behavior
According to the code comment at line 437:
Toggle to include_raw_responses=True if we had an error or if we are returning no records
When include_raw_responses_data
is set to false
or omitted, it should automatically be enabled when zero records are returned, providing developers with debugging information.
Actual Behavior
The auto-enable only triggers when success=False
, not when zero records are returned. The implementation at line 438 is:
include_raw_responses_data = include_raw_responses_data or not success
This does NOT check for zero records (len(records_data) == 0
).
Impact
When a developer has an incorrect dpath configuration:
- API successfully returns data (HTTP 200 with slices)
- Wrong dpath extracts 0 records from the response
success
remainsTrue
(because slices exist)- Auto-enable does NOT trigger
- Developer gets
"raw_api_responses": null
with no debugging information
This breaks a primary use case for raw responses: debugging dpath issues.
Reproduction
# Create manifest with wrong dpath
poe test-tool execute_stream_test_read '{
"manifest": "@wrong_dpath_manifest",
"stream_name": "users",
"config": {},
"max_records": 2,
"include_raw_responses_data": false
}'
Result: Returns "records_read": 0
but "raw_api_responses": null
Root Cause
The success
flag is only set to False
when:
But when the API call succeeds and returns slices, success
stays True
even if dpath extraction results in zero records.
Proposed Fix
Update line 438 to also check for zero records:
include_raw_responses_data = include_raw_responses_data or not success or len(records_data) == 0
This would:
- Match the documented behavior in the comment
- Provide debugging information when dpath extraction fails
- Enable the primary use case identified in issue TODO: Confirm that the test_read tool with raw_responses enabled is giving necessary info #121
Context
This bug was discovered during verification testing for #121. See detailed analysis in this comment.
Environment
- Repository:
airbytehq/connector-builder-mcp
- File:
connector_builder_mcp/validation_testing.py
- Function:
execute_stream_test_read()
- Lines: 437-438
Requested by: @aaronsteers