Add JSON extraction and shadow DOM validation steppers

[cedric-chalhoub]

• Add jsonExtractSteps module with extractPropertyFromResponseJson

  • Extracts properties from HTTP JSON responses (arrays or objects)
  • Supports ordinal array indexing (1st, 2nd, 3rd, etc.)

• Add waitForElementInShadowDom to interactionSteps

  • Validates elements inside Shadow DOM with visibility detection

[vid]

I have not had to use the Shadow DOM but if I'm not mistaken (validated with Claude Haiku 4.5) it could be inlined with waitFor.
I think it can also be used with inElement. the end result would look something like this:

	waitFor: {
		gwta: `wait for {target: ${DOMAIN_STRING_OR_PAGE_LOCATOR}}`,
		action: async ({ target }: { target: string }, featureStep: TFeatureStep) => {
			try {
				// Detect shadow DOM selector (contains >>>)
				if (target.includes('>>>')) {
					const selector = target.split('>>>').pop()?.trim() || target;
					await wp.withPage(async (page: Page) => {
						// Wait for the element to appear in shadow DOM and be actually visible
						await page.waitForFunction(
							(innerSel) => {
								// @ts-expect-error - this is the element context when called on a Locator
								const host = (this && this.nodeType) ? this : document.body;
								if (!host?.shadowRoot) return false;
								
								const element = host.shadowRoot.querySelector(innerSel);
								if (!element) return false;
								
								// Use getBoundingClientRect to check if element has dimensions
								// and is actually rendered (not hidden by z-index, display, etc.)
								const rect = element.getBoundingClientRect();
								if (rect.width === 0 || rect.height === 0) return false;
								
								// Check computed styles for common hiding methods
								const computed = window.getComputedStyle(element);
								if (computed.display === 'none' || computed.visibility === 'hidden' || computed.opacity === '0') return false;
								
								// Check if element is behind other layers (negative z-index parent)
								let current = element.parentElement;
								while (current) {
									const style = window.getComputedStyle(current);
									if (style.display === 'none' || style.visibility === 'hidden' || style.opacity === '0') return false;
									// Check for negative z-index which indicates hidden layer
									const zIndex = parseInt(style.zIndex);
									if (!isNaN(zIndex) && zIndex < 0) return false;
									current = current.parentElement;
								}
								
								return true;
							},
							selector,
							{ timeout: 30000 }
						);
					});
					return OK;
				}
				
				// Regular wait
				await wp.withPage(async (page: Page) => await locateByDomain(page, featureStep, 'target').waitFor());
				return OK;
			} catch (e) {
				return actionNotOK(`Did not find ${target}`);
			}
		},
	},

And usage would be in shadowDomRoot, wait for selector.

But we don't have any tests for this (aside from .feature files). Can you try it with your tests and see if it works?

[cedric-chalhoub]

Should be addressed in new commit. Had to make a few changes but it is now inline with waitFor.

[vid]

Should be able to use the existing click function with buttons. We want to minimize the number of stepper steps to avoid cognitive overload for humans, and MCP works better with fewer too. Though at some point, will need to review the steppers and organize them better.

[cedric-chalhoub]

Agreed, was definitely redundant. Addressed in the new commit.

[vid]

LGTM if it is fit to purpose.

[vid]

lgtm!