Skip to content

Issue #48: Support wildcard iterations#49

Open
NassimBtk wants to merge 16 commits intomainfrom
feature/issue-48-support-iterating-over-object-children-using-wildcard
Open

Issue #48: Support wildcard iterations#49
NassimBtk wants to merge 16 commits intomainfrom
feature/issue-48-support-iterating-over-object-children-using-wildcard

Conversation

@NassimBtk
Copy link
Contributor

This pull request adds robust wildcard support to the JFlat utility, enabling expansion of dynamic object keys in JSON structures into CSV rows. The changes include new wildcard handling logic in the code, comprehensive documentation, and thorough tests to ensure correct behavior—including support for escaping wildcards when a key is literally named "*". This greatly improves the flexibility of JFlat when dealing with JSON objects keyed by UIDs or other dynamic names.

Wildcard support for object keys:

  • Added logic to JFlat.java so that a * in a path expands all direct children of the current object, supporting dynamic JSON keys. Escaping with \* allows targeting literal keys named "*". [1] [2]
  • Updated documentation in index.md to explain how to use wildcards and escaping in entry key paths for CSV extraction.

Testing and validation:

  • Added new tests in JFlatTest.java to verify wildcard expansion, property extraction with wildcards, and correct escaping for literal "*" keys.
  • Introduced new test fixtures: object-keys.json and wildcard-key.json to cover various wildcard scenarios, and object-keys-flatMap.txt for expected output validation. [1] [2] [3]
  • Integrated wildcard test cases into the flatMap test suite for broader coverage.

Other:

  • Fixed a typo in the copyright notice ("Metricshb" → "MetricsHub") in JFlat.java.

Added support for wildcard iterations in JFlat,
allowing users to iterate over all children
of an object regardless of their keys.
This enhancement simplifies the process of
flattening JSON structures with dynamic or
unknown keys.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds wildcard (*) support to JFlat.toCSV() entry-key paths so callers can expand dynamic object keys (e.g., UID-keyed maps) into CSV rows, with documentation and tests covering wildcard expansion and literal * escaping.

Changes:

  • Implement wildcard path element handling in JFlat.toCSV() (including \* for literal * keys).
  • Add tests + fixtures validating wildcard expansion, nested property extraction, and escaping behavior.
  • Update site documentation to describe wildcard usage and escaping.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/main/java/org/metricshub/jflat/JFlat.java Implements * expansion in entry-key path parsing and fixes copyright typo.
src/test/java/org/metricshub/jflat/JFlatTest.java Adds unit tests for wildcard expansion and escaped literal * keys; extends flatMap coverage.
src/site/markdown/index.md Documents wildcard entry-key paths and \* escaping.
src/test/resources/object-keys.json New fixture for dynamic object-key scenarios.
src/test/resources/object-keys-flatMap.txt Expected flatMap output for the dynamic object-key fixture.
src/test/resources/wildcard-key.json New fixture containing a literal * key to validate escaping.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +406 to +452
// We iterate over every key in the map and find those that start with the
// current entry's prefix (e.g. "/members/"). From each matching key, we
// extract the immediate child name by looking for the next "/" or "["
// delimiter, which marks a deeper level or an array index.
// Duplicates are skipped (case-insensitive) to ensure each child appears once.
String prefix = existingEntry.equals("/") ? "/" : existingEntry + "/";
for (String key : map.keySet()) {
// Only consider keys that are strictly under the prefix
if (key.length() > prefix.length() && key.regionMatches(true, 0, prefix, 0, prefix.length())) {
// Extract the portion after the prefix, e.g. "abc123/name" from "/members/abc123/name"
String remainder = key.substring(prefix.length());

// Find the boundary of the immediate child name:
// - "/" indicates a deeper nested property
// - "[" indicates an array index
// The child name is everything before the first such delimiter.
int slashPos = remainder.indexOf('/');
int bracketPos = remainder.indexOf('[');
String childName;
if (slashPos == -1 && bracketPos == -1) {
// No delimiter: the remainder itself is the child name (leaf key)
childName = remainder;
} else if (slashPos == -1) {
// Only "[" found: child has array children (e.g. "args[0]")
childName = remainder.substring(0, bracketPos);
} else if (bracketPos == -1) {
// Only "/" found: child has nested properties (e.g. "abc123/name")
childName = remainder.substring(0, slashPos);
} else {
// Both found: take the earlier delimiter
childName = remainder.substring(0, Math.min(slashPos, bracketPos));
}

// Add the child path if it's valid and not already in the list
if (!childName.isEmpty()) {
String childPath = prefix + childName;
// Deduplicate (case-insensitive, consistent with the TreeMap)
boolean alreadyAdded = false;
for (String added : newEntries) {
if (added.equalsIgnoreCase(childPath)) {
alreadyAdded = true;
break;
}
}
if (!alreadyAdded) {
newEntries.add(childPath);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot switch wildcard dedup to a set-based approach.

Comment on lines +392 to +396
// to refer to it without triggering wildcard expansion.
boolean isWildcard = "*".equals(pathElement);
if (pathElement.startsWith("\\")) {
// Strip the escape backslash and treat the rest as a literal property name
pathElement = pathElement.substring(1);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot limit escaping to the wildcard case (only treat * specially).

Copy link

Copilot AI commented Mar 19, 2026

@NassimBtk I've opened a new pull request, #50, to work on those changes. Once the pull request is ready, I'll request review from you.

Copy link

Copilot AI commented Mar 19, 2026

@NassimBtk I've opened a new pull request, #51, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot AI and others added 5 commits March 19, 2026 20:45
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds wildcard (*) support to JFlat’s CSV entry-key path traversal so JSON objects with dynamic keys (e.g., UID-keyed maps) can be expanded into multiple CSV rows, with an escape mechanism (\*) for literal * keys.

Changes:

  • Implemented wildcard expansion logic in JFlat.toCSV() for object-key iteration and \* escaping.
  • Added/extended tests and fixtures to validate wildcard expansion and escaping behavior.
  • Updated site documentation to explain wildcard usage and escaping.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/main/java/org/metricshub/jflat/JFlat.java Adds wildcard expansion and escaped-literal handling in toCSV(); fixes copyright typo.
src/test/java/org/metricshub/jflat/JFlatTest.java Adds CSV wildcard and escape test coverage; extends flatMap test suite with new fixture.
src/site/markdown/index.md Documents wildcard iteration (*) and literal escaping (\*) in entry-key paths.
src/test/resources/object-keys.json New fixture for dynamic object-key maps.
src/test/resources/object-keys-flatMap.txt Expected flat-map output for the new dynamic object-key fixture.
src/test/resources/wildcard-key.json New fixture containing a literal * key to validate escape behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +410 to +418
// We iterate over every key in the map and find those that start with the
// current entry's prefix (e.g. "/members/"). From each matching key, we
// extract the immediate child name by looking for the next "/" or "["
// delimiter, which marks a deeper level or an array index.
// Duplicates are skipped (case-insensitive) to ensure each child appears once.
String prefix = existingEntry.equals("/") ? "/" : existingEntry + "/";
for (String key : map.keySet()) {
// Only consider keys that are strictly under the prefix
if (key.length() > prefix.length() && key.regionMatches(true, 0, prefix, 0, prefix.length())) {
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wildcard expansion currently iterates over the full map.keySet() for each existingEntry. Since map is a TreeMap, this can become expensive on large inputs (and on entry keys that already expanded from arrays). Consider iterating only the relevant key range (e.g., starting from the prefix via tailMap(prefix, false) and stopping once keys no longer match the prefix) to keep the wildcard lookup closer to O(matches) instead of O(totalKeys).

Suggested change
// We iterate over every key in the map and find those that start with the
// current entry's prefix (e.g. "/members/"). From each matching key, we
// extract the immediate child name by looking for the next "/" or "["
// delimiter, which marks a deeper level or an array index.
// Duplicates are skipped (case-insensitive) to ensure each child appears once.
String prefix = existingEntry.equals("/") ? "/" : existingEntry + "/";
for (String key : map.keySet()) {
// Only consider keys that are strictly under the prefix
if (key.length() > prefix.length() && key.regionMatches(true, 0, prefix, 0, prefix.length())) {
// We iterate over keys in the map that are at or after the current entry's
// prefix (e.g. "/members/") and stop once keys no longer match that prefix.
// From each matching key, we extract the immediate child name by looking for
// the next "/" or "[" delimiter, which marks a deeper level or an array index.
// Duplicates are skipped (case-insensitive) to ensure each child appears once.
String prefix = existingEntry.equals("/") ? "/" : existingEntry + "/";
for (String key : map.tailMap(prefix, true).keySet()) {
// Stop once keys are no longer under the prefix (taking case-insensitive match into account)
if (!key.regionMatches(true, 0, prefix, 0, prefix.length())) {
break;
}
// Only consider keys that are strictly under the prefix
if (key.length() > prefix.length()) {

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Consider iterating only the relevant key range (e.g., starting from the prefix via tailMap(prefix, false) and stopping once keys no longer match the prefix) to keep the wildcard lookup closer to O(matches) instead of O(totalKeys).

Comment on lines +445 to +447
String childPath = prefix + childName;
if (seenLowercasePaths.add(childPath.toLowerCase())) {
newEntries.add(childPath);
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

childPath.toLowerCase() uses the default JVM locale, which can lead to incorrect case folding (and therefore incorrect dedup) in some locales (e.g., Turkish). Use locale-independent folding (e.g., toLowerCase(Locale.ROOT)) or a case-insensitive Set implementation to avoid locale-sensitive behavior.

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

Comment on lines +395 to +399
boolean isWildcard = "*".equals(pathElement);
if ("\\*".equals(pathElement)) {
// Strip the escape backslash and treat "*" as a literal property name
pathElement = "*";
isWildcard = false;
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "\\*" path element is now reserved to mean “literal * key”. This makes it impossible to target a JSON property whose name is literally "\\*" (backslash + asterisk), because it will always be interpreted as an escape sequence. If supporting such keys matters, consider defining an escape for the backslash itself (e.g., "\\\\*" in the path syntax) and handling it before the wildcard escape logic.

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI commented Mar 20, 2026

@NassimBtk I've opened a new pull request, #52, to work on those changes. Once the pull request is ready, I'll request review from you.

Copy link

Copilot AI commented Mar 20, 2026

@NassimBtk I've opened a new pull request, #53, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot AI and others added 5 commits March 20, 2026 11:10
Fix locale-sensitive `toLowerCase()` in wildcard path dedup
Use tailMap for O(matches) wildcard key iteration instead of O(totalKeys)
@NassimBtk NassimBtk requested a review from bertysentry March 20, 2026 13:11
Added support for /* iteration over arrays as well as objects.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 77 to 80
assertEquals(
"[0]/arrayB[0]/id;\n[0]/arrayB[1]/id;\n[0]/arrayB[2]/id;\n[1]/arrayB[0]/id;\n[1]/arrayB[1]/id;\n[1]/arrayB[2]/id;\n",
simple.toCSV("/arrayB/id", null, null).toString()
simple.toCSV("/arrayB/*/id", null, null).toString()
);
Copy link

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing non-wildcard array traversal path (/arrayB/id) is no longer covered by tests after switching this assertion to /arrayB/*/id. Since wildcard support is additive, keeping an assertion for /arrayB/id (in addition to the new wildcard one) would help catch regressions and confirm backward compatibility.

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot open a new pull request to apply changes based on this feedback

Copy link

Copilot AI commented Mar 20, 2026

@NassimBtk I've opened a new pull request, #54, to work on those changes. Once the pull request is ready, I'll request review from you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support iterating over object children using wildcard

3 participants