Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions conformance/testdata/generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@


# What a typed value is written to the trace as when the target reports no
# secure fact for the field. Android reports none for any field, so every
# InputText it records reads this (internal/verifier/redaction.go).
# secure fact for the field, which is what the android fixtures below model
# (internal/verifier/redaction.go).
REDACTED = "[redacted]"


Expand Down
4 changes: 3 additions & 1 deletion docs/manual/runs.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ The trace is written incrementally. An interrupted run is complete up to the ste

A run types into whatever the app puts on screen, login forms included, and the trace and the model call record are both shared. So a typed value is written down as `[redacted]` whenever the target may be a credential entry: the trace action, the recent-action memory the prompt carries, the numbered candidate list, and the `state.lastAction` a spec reads (and can extract into the trace) all render it that way. The app still receives the real keystrokes; only the record is redacted, and the record still names the field that was typed into.

Which values that covers differs by platform, because the platforms differ in what they report. iOS and web state on every editable field whether it masks its input, so only the fields that do are redacted and the rest of the memory keeps its values. Android reports nothing: uiautomator's password attribute is dropped by the native tree mapper before the driver sees it, a password field is indistinguishable from a search box there, and so every typed value on Android is redacted.
Which values that covers is decided per field, from what the platform says about it. iOS and web state on every editable field whether it masks its input. Android states it too, though the tree the sidecar gets from maestro does not: maestro's mapper copies a fixed attribute list off the device's view hierarchy and `password` is not on it, so the sidecar re-reads that hierarchy once per settled snapshot and puts the fact back on the text fields it can match. A field it cannot match is left unstated, and an unstated field is redacted.

On iOS and web the fact comes from the platform's own widget type, and a Compose Multiplatform app has none: iOS exposes a password field as a `TextArea` rather than a `SecureTextField` (`internal/driver/ioscompanion/hierarchymap.go`), and web renders it as a `contenteditable` div rather than an `<input type="password">` (`internal/driver/chrome/driver.go`). Both checks then state `secure: false` from a test that cannot say yes for such an app, and the value is written to the record in the clear. Measured on 2026-08-19 against `examples/folio` on both targets: the login password reaches `trace.jsonl` as `ledger123`, in the step's `next_action.text` and again in the `state.lastAction` the following step reports. Android is not affected; the fact it reads is the device's own `password` attribute. Until this is closed, treat an iOS or web trace of a Compose app as holding whatever the run typed.

## App state across runs

Expand Down
4 changes: 2 additions & 2 deletions docs/manual/spec-language.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Every key-value pair must match. A key means the same thing here as in the strin

Known attribute names are typed; you get autocomplete on `testTag`, `text`, `content-desc`, the boolean states (`clickable`, `enabled`, `focused`, `checked`, `selected`, `editable`, `secure`), and the cross-platform aliases (`identifier`, `accessibilityIdentifier`, `accessibilityText`, `accessibilityLabel`, `ariaLabel`, `contentDescription`, `label`, `testID`, `resource-id`, `class`, `className`, `elementType`, `package`, `placeholderValue`, `hintText`). Boolean state attributes accept a native `true` / `false`. Other attribute keys still type-check as a string-valued fallback so raw driver attributes remain reachable.

A boolean state matches only where the platform reports it. `{secure: true}` names the password entry and `{secure: false}` names an editable field that is not one, so neither value names an element that is no field at all, and neither matches anything on Android, which reports the fact for nothing.
A boolean state matches only where the platform reports it. `{secure: true}` names the password entry and `{secure: false}` names an editable field that is not one, so neither value names an element that is no field at all. All three platforms report it; on Android a text field the sidecar cannot match against the device's own view hierarchy is left unstated and answers to neither value.

`clickable`, `enabled`, `focused`, `checked`, `selected` and `editable` are reported for every element, so both values of each match: `{clickable: false}` names every element that is not a tap target. They are read off the element as it stands, never off a markup attribute of the same name, so a box the user ticked answers to `{checked: true}` on a page whose markup never wrote `checked` anywhere.

Expand Down Expand Up @@ -145,7 +145,7 @@ Fields available on every element returned by `find` / `findAll`:
| `checked` | `boolean` | Checkbox or toggle state |
| `focused` | `boolean` | Element has input focus |
| `selected` | `boolean` | Selection state |
| `secure` | `boolean \| null` | Field masks what is typed into it; `null` where the platform does not report it (Android never does) |
| `secure` | `boolean \| null` | Field masks what is typed into it; `null` where the platform does not report it |
| `bounds` | `{ left, top, right, bottom }` | Bounding box in device pixels |
| `x` | `number` | Center X (derived from bounds) |
| `y` | `number` | Center Y (derived from bounds) |
Expand Down
25 changes: 25 additions & 0 deletions internal/hierarchy/hierarchy.go
Original file line number Diff line number Diff line change
Expand Up @@ -667,6 +667,31 @@ func (t *Tree) Transitional() bool {
return false
}

// ScreenName names the route this tree shows: the driver-set screen when the
// platform reports one (web), otherwise the resource id ending in "Screen" that
// marks the route composable. A tree carrying two different route ids is a
// cross-fade in flight and names no screen, which is the same reading
// Transitional gives it.
func (t *Tree) ScreenName() string {
if t == nil || len(t.Elements) == 0 {
return ""
}
if screen := t.Elements[0].Screen; screen != "" {
return screen
}
name := ""
for _, element := range t.Elements {
if !strings.HasSuffix(element.ResourceID, "Screen") {
continue
}
if name != "" && name != element.ResourceID {
return ""
}
name = element.ResourceID
}
return name
}

// Find returns the first element matching the selector, or nil.
func (t *Tree) Find(selector string) *Element {
node := t.FindNode(selector)
Expand Down
54 changes: 54 additions & 0 deletions internal/hierarchy/hierarchy_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1821,3 +1821,57 @@ func TestTagMatchesTheElementItNames(t *testing.T) {
})
}
}

func TestScreenNameNamesTheRouteTheTreeShows(t *testing.T) {
cases := []struct {
name string
tree string
want string
}{
{
"driver-set screen",
`{"attributes": {"bounds": "[0,0,10,10]", "sanderling-screen": "/ledger"}, "children": []}`,
"/ledger",
},
{
"route marker",
`{"attributes": {"bounds": "[0,0,10,10]"}, "children": [
{"attributes": {"resource-id": "HomeScreen", "bounds": "[0,0,10,10]"}, "children": []}
]}`,
"HomeScreen",
},
{
"route marker repeated by a nested node",
`{"attributes": {"bounds": "[0,0,10,10]"}, "children": [
{"attributes": {"resource-id": "HomeScreen", "bounds": "[0,0,10,10]"}, "children": [
{"attributes": {"resource-id": "HomeScreen", "bounds": "[0,0,10,5]"}, "children": []}
]}
]}`,
"HomeScreen",
},
{
"cross-fade names no screen",
`{"attributes": {"bounds": "[0,0,10,10]"}, "children": [
{"attributes": {"resource-id": "HomeScreen", "bounds": "[0,0,10,10]"}, "children": []},
{"attributes": {"resource-id": "LedgerScreen", "bounds": "[0,0,10,10]"}, "children": []}
]}`,
"",
},
{
"no marker at all",
`{"attributes": {"bounds": "[0,0,10,10]"}, "children": []}`,
"",
},
}
for _, testCase := range cases {
t.Run(testCase.name, func(t *testing.T) {
tree, err := Parse(testCase.tree)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if got := tree.ScreenName(); got != testCase.want {
t.Errorf("ScreenName = %q, want %q", got, testCase.want)
}
})
}
}
59 changes: 59 additions & 0 deletions internal/runner/log_step_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package runner

import (
"bytes"
"log/slog"
"strings"
"testing"

"github.com/priyanshujain/sanderling/internal/verifier"
)

func logStepLine(t *testing.T, action verifier.Action, treeJSON string) string {
t.Helper()
var buffer bytes.Buffer
logger := slog.New(slog.NewTextHandler(&buffer, nil))
logStep(logger, 7, "LoginScreen", 42, action, nil, "", mustParseTree(t, treeJSON))
return buffer.String()
}

func TestLogStepNamesTheActionItsTargetAndTheTypedValue(t *testing.T) {
line := logStepLine(t, typeInto("id:LoginEmail"), iosLoginTreeJSON)

for _, want := range []string{
"index=7", "screen=LoginScreen", "nodes=42",
"action=InputText", "target=id:LoginEmail", typedCredential,
} {
if !strings.Contains(line, want) {
t.Errorf("step log = %q, want it to carry %q", line, want)
}
}
}

func TestLogStepRedactsTheTypedValueOfASecureField(t *testing.T) {
line := logStepLine(t, typeInto("id:LoginPassword"), iosLoginTreeJSON)

if strings.Contains(line, typedCredential) {
t.Errorf("step log = %q, want the typed credential withheld", line)
}
if !strings.Contains(line, verifier.RedactedInputText) {
t.Errorf("step log = %q, want %q", line, verifier.RedactedInputText)
}
if !strings.Contains(line, "target=id:LoginPassword") {
t.Errorf("step log = %q, want it to still name the field typed into", line)
}
}

func TestLogStepReportsAStepThatActedOnNothing(t *testing.T) {
var buffer bytes.Buffer
logger := slog.New(slog.NewTextHandler(&buffer, nil))
logStep(logger, 3, "", 0, verifier.Action{}, verifier.ErrNoAction, actionSkippedNoActionProduced, nil)

line := buffer.String()
if !strings.Contains(line, "action=none") {
t.Errorf("step log = %q, want action=none", line)
}
if !strings.Contains(line, "skipped="+string(actionSkippedNoActionProduced)) {
t.Errorf("step log = %q, want the skip reason", line)
}
}
10 changes: 5 additions & 5 deletions internal/runner/redaction_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ const webLoginTreeJSON = `{
]
}`

// androidLoginTreeJSON is the same form on Android, where the native tree
// mapper drops uiautomator's password attribute and neither field carries the
// fact.
// androidLoginTreeJSON is the same form with the fact missing from both fields,
// which is what an Android tree looks like when the sidecar could not match its
// text fields against the device's own view hierarchy.
const androidLoginTreeJSON = `{
"attributes": {"bounds": "[0,0,1080,2340]"},
"children": [
Expand All @@ -65,8 +65,8 @@ func TestTraceActionForRedactsTypedValuesTheTargetCannotClear(t *testing.T) {
}{
{"ios secure field", iosLoginTreeJSON, "id:LoginPassword"},
{"web secure field", webLoginTreeJSON, "id:login-password"},
{"android field reported as neither", androidLoginTreeJSON, "id:login_email"},
{"android password field", androidLoginTreeJSON, "id:login_password"},
{"field reported as neither", androidLoginTreeJSON, "id:login_email"},
{"password field reported as neither", androidLoginTreeJSON, "id:login_password"},
} {
t.Run(testCase.name, func(t *testing.T) {
traceAction := traceActionFor(typeInto(testCase.selector), mustParseTree(t, testCase.treeJSON))
Expand Down
38 changes: 32 additions & 6 deletions internal/runner/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,7 @@ func Run(ctx context.Context, options Options) (Summary, error) {
}
lastLogTime = stepStart

screen := ""
if tree != nil && len(tree.Elements) > 0 {
screen = tree.Elements[0].Screen
}
screen := tree.ScreenName()

// A transitional tree is one nothing can vouch for: a NavHost mid
// cross-fade, a screen that changed shape between two reads, or a
Expand Down Expand Up @@ -332,8 +329,6 @@ func Run(ctx context.Context, options Options) (Summary, error) {
logger.Warn("unsettled tree; skipping verifier",
"step", stepIndex, "screen", screen, "nodes", treeSize)
}
logger.Info("step", "index", stepIndex, "screen", screen, "nodes", treeSize)

// A frame the verifier would not look at is not one to act on either.
// #75 is the fuzzer tapping into a screen that is still filling in, and
// holding the action back is also what keeps the spec's view of the run
Expand Down Expand Up @@ -474,6 +469,8 @@ func Run(ctx context.Context, options Options) (Summary, error) {
// the action it points at is still the one the next verified step has to
// be told about.

logStep(logger, stepIndex, screen, treeSize, nextAction, nextErr, actionSkipped, tree)

step := trace.Step{
Index: stepIndex,
Timestamp: stepStart,
Expand Down Expand Up @@ -1578,6 +1575,35 @@ func driverIsAndroid(ctx context.Context, options Options, logger *slog.Logger)
return health.Platform == "android"
}

// logStep prints the one line a run emits per step: what screen it saw and what
// it did there. The typed value goes through the same redaction the trace and
// the prompt use, so the console cannot publish a credential the records
// withhold.
func logStep(
logger *slog.Logger,
stepIndex int,
screen string,
treeSize int,
action verifier.Action,
actionErr error,
skipped actionSkipReason,
tree *hierarchy.Tree,
) {
attrs := []any{"index", stepIndex, "screen", screen, "nodes", treeSize}
if actionErr != nil {
attrs = append(attrs, "action", "none")
} else {
attrs = append(attrs, "action", string(action.Kind), "target", actionTarget(action))
if action.Kind == verifier.ActionKindInputText {
attrs = append(attrs, "text", verifier.RecordedActionText(action, tree))
}
}
if skipped != "" {
attrs = append(attrs, "skipped", string(skipped))
}
logger.Info("step", attrs...)
}

func traceActionFor(action verifier.Action, tree *hierarchy.Tree) *trace.Action {
traceAction := &trace.Action{
Kind: string(action.Kind),
Expand Down
8 changes: 4 additions & 4 deletions internal/runner/testdata/trace.jsonl
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{"extractor_changes":{"extractor_0":{"prev":null,"curr":0}},"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"step":1,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"step":2,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"step":3,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"step":4,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"extractor_changes":{"extractor_0":{"prev":null,"curr":0}},"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"screen":"HomeScreen","step":1,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"screen":"HomeScreen","step":2,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"screen":"HomeScreen","step":3,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
{"hierarchy":{"elements":[{"resourceId":"HomeScreen","bounds":{"left":0,"top":0,"right":0,"bottom":0},"attrs":{"editable":"false","resource-id":"HomeScreen"}},{"resourceId":"next","clickable":true,"enabled":true,"bounds":{"left":40,"top":80,"right":240,"bottom":160},"attrs":{"bounds":"[40,80,240,160]","clickable":"true","editable":"false","enabled":"true","resource-id":"next"}}],"depths":[0,1]},"next_action":{"kind":"Tap","selector":"id:next","resolved_bounds":{"x":40,"y":80,"width":200,"height":80},"tap_point":{"x":140,"y":120},"source":"seeded"},"residuals":{"balanceNonNegative":{"op":"true"}},"screen":"HomeScreen","step":4,"timestamp":"0001-01-01T00:00:00Z","trace_version":1}
10 changes: 5 additions & 5 deletions internal/verifier/redaction.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,11 @@ func RecordedAction(action Action, tree *hierarchy.Tree) Action {
// have produced.
//
// A target the platform reports as a secure entry is redacted, and so is a
// target carrying no report at all. iOS and web state the fact on every
// editable element, so a missing one means Android, whose native tree mapper
// drops uiautomator's password attribute before the sidecar sees it. There a
// password field cannot be told from a search box, and the target that cannot
// be told apart is treated as the credential.
// target carrying no report at all. All three platforms state the fact on their
// editable elements, so a missing one is a field none of them could speak for:
// an action that named no target, or an Android text field the sidecar could
// not match against the device's own view hierarchy. The target that cannot be
// told apart is treated as the credential.
func recordedInputText(text string, target secureFact) string {
if target.reported && !target.secure {
return text
Expand Down
Loading
Loading