Skip to content

Feat tap 10677 optimize js node#3138

Draft
weiliang110100 wants to merge 6 commits intodevelopfrom
feat-TAP-10677-Optimize-JS-node
Draft

Feat tap 10677 optimize js node#3138
weiliang110100 wants to merge 6 commits intodevelopfrom
feat-TAP-10677-Optimize-JS-node

Conversation

@weiliang110100
Copy link
Copy Markdown
Contributor

No description provided.

@augmentcode
Copy link
Copy Markdown

augmentcode bot commented Apr 10, 2026

🤖 Augment PR Summary

Summary: This PR enhances JS node test-run and mock-data capabilities and improves value/date handling.

Changes:

  • Allows providing custom test-run input events as JSON and converts them into TapEvents using table schema.
  • Updates HazelcastSampleSourcePdkDataNode to prefer custom input events (and optionally bypass row limits) when present.
  • Adds utilities to stringify nested TapValue/temporal types for easier JSON output (ClassHandlersV2ToStringUtils).
  • Extends engine-side query service with queryV2/getDataV2 to support stringified results for mock-data.
  • Adds TM-side mock-data RPC flow to fetch sample records via SQL (raw command) or filter query with capability checks.
  • Exposes a new API endpoint for JS mock-data retrieval and wires new DTO fields (sql, testRunInputEventJson, stackTrace).
  • Enhances test-run result reporting to include full stack traces via a new ExceptionUtil.getStackString.
  • Adds object-aware date arithmetic/compare helpers in DateUtil and unit tests for them.
  • Changes JS built-in Date binding to use io.tapdata.entity.schema.value.DateTime.

Technical Notes: Mock-data retrieval is guarded by connector capabilities (run_raw_command_function / query_by_advance_filter_function), and new tests cover JSON conversion and mock-data RPC behavior.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 6 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

return success();
}

@GetMapping("migrate-js/mock-data")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manager/tm/src/main/java/com/tapdata/tm/task/controller/TaskController.java:1268: This is a GET endpoint but it uses @RequestBody, which many clients/proxies won’t send/forward for GET requests (leading to missing dto at runtime). This can make the mock-data API unreliable and hard to call consistently.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

for (Map<String, Object> map : maps) {
codecsFilterManager.transformToTapValueMap(map, tapTable.getNameFieldMap());
originCodecsFilterManager.transformFromTapValueMap(map);
if(limit != null){
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iengine/iengine-app/src/main/java/io/tapdata/services/QueryDataBaseDataService.java:176: limit != null is being used both to control the limit and to toggle stringification + the response shape (sometimes returning only sampleData). This makes the method’s output contract dependent on whether limit is provided, which can easily break callers that pass/omit limit for unrelated reasons.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

private void checkCapability(DataSourceConnectionDto connection, String capabilityId, String errorCode, String connectionId) {
boolean supported = CollectionUtils.isNotEmpty(connection.getCapabilities())
&& connection.getCapabilities().stream()
.anyMatch(capability -> capability.getId().equals(capabilityId));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manager/tm/src/main/java/com/tapdata/tm/task/service/impl/TaskNodeServiceImpl.java:1015: capability.getId().equals(capabilityId) can throw an NPE if any capability has a null id. A null-safe comparison would avoid failing capability checks on partially populated capability entries.

Severity: low

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

return value;
}
if (value instanceof String) {
return JSONUtil.toList(StringUtils.trim((String) value), Map.class);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iengine/iengine-app/src/main/java/io/tapdata/flow/engine/util/TestRunInputEventConvertUtil.java:232: convertArrayValue parses a JSON string using JSONUtil.toList(..., Map.class), which will fail for arrays of primitives/strings (e.g., [1,2]) or mixed element types. If test-run input JSON includes scalar arrays, this will throw during conversion.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

buildInMethod.append("var LinkedHashMap = Java.type(\"java.util.LinkedHashMap\");\n");
buildInMethod.append("var ArrayList = Java.type(\"java.util.ArrayList\");\n");
buildInMethod.append("var Date = Java.type(\"java.util.Date\");\n");
buildInMethod.append("var Date = Java.type(\"io.tapdata.entity.schema.value.DateTime\");\n");
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iengine/iengine-common/src/main/java/com/tapdata/processor/ScriptUtil.java:343: Rebinding the built-in JS Date type from java.util.Date to io.tapdata.entity.schema.value.DateTime is a behavior-breaking change (constructors/methods differ) and can cause existing JS node scripts to fail at runtime. It may need explicit compatibility handling or clear migration guidance.

Severity: high

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

@weiliang110100 weiliang110100 marked this pull request as draft April 10, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant