Skip to content

Commit cd08d26

Browse files
committed
docs(ptc-lisp): add language design guidelines
Document when PTC-Lisp should follow Clojure and when sandbox safety or recoverable signal values justify intentional divergence. Link the policy from contributor instructions, README, spec, conformance gaps, and generated reference docs. Verified with mix ptc.gen_docs, mix ptc.validate_spec, and mix test test/readme_test.exs test/ptc_runner/lisp/spec_validator_test.exs.
1 parent 4d827fb commit cd08d26

14 files changed

Lines changed: 188 additions & 20 deletions

AGENTS.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,14 @@ Use a concise Conventional Commit subject, e.g.
88
For non-trivial commits, add a short body covering what changed and how
99
it was verified.
1010

11+
## PTC-Lisp Changes
12+
13+
When changing PTC-Lisp syntax, builtins, Java interop, or Clojure
14+
conformance behavior, read `docs/ptc-lisp-design-guidelines.md` first.
15+
Clojure compatibility is the default, but sandbox safety and recoverable
16+
signal values take precedence for Clojure-named functions where Clojure
17+
would raise; Java-named dot methods keep Java semantics.
18+
1119
<!-- usage-rules-start -->
1220
<!-- usage-rules-header -->
1321
# Usage Rules

README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,10 +262,13 @@ llm = PtcRunner.LLM.callback("bedrock:haiku", cache: true)
262262

263263
- **[Signature Syntax](docs/signature-syntax.md)** - Input/output type contracts
264264
- **[PTC-Lisp Specification](docs/ptc-lisp-specification.md)** - The language SubAgents write (a Clojure subset: 211 of 534 `clojure.core` vars, plus `clojure.string`, `clojure.set`, and `java.lang.Math`)
265-
- **[Function Reference](docs/function-reference.md)** - All 272 built-in functions with signatures
265+
- **[PTC-Lisp Design Guidelines](docs/ptc-lisp-design-guidelines.md)** - What belongs in the language and when it intentionally diverges from Clojure
266+
- **[Function Reference](docs/function-reference.md)** - All built-in functions with signatures
266267
- **Clojure Conformance** - [Core](docs/clojure-core-audit.md) | [String](docs/clojure-string-audit.md) | [Set](docs/clojure-set-audit.md) | [Math](docs/java-math-audit.md) | [Java Interop](docs/java-interop.md)
267268
- **[Benchmark Evaluation](docs/benchmark-eval.md)** - LLM accuracy by model
268269

270+
PTC-Lisp follows Clojure where that is safe and bounded. Intentional divergences favor sandbox safety and recoverable signal values for Clojure-named helpers; Java-named dot methods keep Java semantics.
271+
269272
### Interactive
270273

271274
- **`mix ptc.repl`** - Interactive REPL for testing PTC-Lisp expressions

docs/clojure-conformance-gaps.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ Tracked differences between PTC-Lisp and Clojure semantics, discovered via confo
44

55
**Test file:** `test/ptc_runner/lisp/sci_conformance_test.exs`
66
**Related issue:** [#832](https://github.com/andreasronge/ptc_runner/issues/832)
7+
**Design policy:** `docs/ptc-lisp-design-guidelines.md`
78
**Audit (function coverage):** `docs/clojure-core-audit.md`
89
**Function reference:** `docs/function-reference.md`
910

@@ -616,3 +617,4 @@ When conformance testing reveals a new gap:
616617
3. Set priority: P0 if it causes silent wrong results, P1 if it errors where Clojure succeeds, P2 if edge case
617618
4. Include a minimal reproducer with both Clojure and PTC-Lisp output
618619
5. Note the source (SCI test name + line, Joker test, manual, etc.)
620+
6. For `DIV-*` entries, apply the rules in `docs/ptc-lisp-design-guidelines.md`: state why Clojure conformance loses to sandbox safety, bounded execution, or recoverable signal values

docs/clojure-core-audit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
77
Comparison of `clojure.core` vars against PTC-Lisp builtins.
88

9-
See also: [Function Reference](function-reference.md) | [Clojure String Audit](clojure-string-audit.md) | [Clojure Set Audit](clojure-set-audit.md) | [Java Math Audit](java-math-audit.md)
9+
See also: [Function Reference](function-reference.md) | [Design Guidelines](ptc-lisp-design-guidelines.md) | [Clojure String Audit](clojure-string-audit.md) | [Clojure Set Audit](clojure-set-audit.md) | [Java Math Audit](java-math-audit.md)
1010

1111
## Summary
1212

docs/clojure-set-audit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
77
Comparison of `clojure.set` vars against PTC-Lisp builtins.
88

9-
See also: [Function Reference](function-reference.md) | [Clojure Core Audit](clojure-core-audit.md) | [Clojure String Audit](clojure-string-audit.md) | [Java Math Audit](java-math-audit.md)
9+
See also: [Function Reference](function-reference.md) | [Design Guidelines](ptc-lisp-design-guidelines.md) | [Clojure Core Audit](clojure-core-audit.md) | [Clojure String Audit](clojure-string-audit.md) | [Java Math Audit](java-math-audit.md)
1010

1111
## Summary
1212

docs/clojure-string-audit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
77
Comparison of `clojure.string` vars against PTC-Lisp builtins.
88

9-
See also: [Function Reference](function-reference.md) | [Clojure Core Audit](clojure-core-audit.md) | [Clojure Set Audit](clojure-set-audit.md) | [Java Math Audit](java-math-audit.md)
9+
See also: [Function Reference](function-reference.md) | [Design Guidelines](ptc-lisp-design-guidelines.md) | [Clojure Core Audit](clojure-core-audit.md) | [Clojure Set Audit](clojure-set-audit.md) | [Java Math Audit](java-math-audit.md)
1010

1111
## Summary
1212

docs/function-reference.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
77
292 functions and special forms.
88

9-
See also: [PTC-Lisp Specification](ptc-lisp-specification.md) | [Clojure Core](clojure-core-audit.md) | [Clojure String](clojure-string-audit.md) | [Clojure Set](clojure-set-audit.md) | [Java Math](java-math-audit.md)
9+
See also: [PTC-Lisp Specification](ptc-lisp-specification.md) | [Design Guidelines](ptc-lisp-design-guidelines.md) | [Clojure Core](clojure-core-audit.md) | [Clojure String](clojure-string-audit.md) | [Clojure Set](clojure-set-audit.md) | [Java Math](java-math-audit.md)
1010

1111
## Table of Contents
1212

docs/java-interop.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ PTC-Lisp emulates a subset of Java interop for LLM compatibility. These are **no
88

99
17 interop entries across 7 classes.
1010

11-
See also: [Function Reference](function-reference.md) | [PTC-Lisp Specification](ptc-lisp-specification.md)
11+
See also: [Function Reference](function-reference.md) | [PTC-Lisp Specification](ptc-lisp-specification.md) | [Design Guidelines](ptc-lisp-design-guidelines.md)
1212

1313
### java.lang.Double
1414

docs/java-math-audit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
77
Comparison of `java.lang.Math` methods against PTC-Lisp builtins.
88

9-
See also: [Function Reference](function-reference.md) | [Clojure Core Audit](clojure-core-audit.md) | [Clojure String Audit](clojure-string-audit.md) | [Clojure Set Audit](clojure-set-audit.md)
9+
See also: [Function Reference](function-reference.md) | [Design Guidelines](ptc-lisp-design-guidelines.md) | [Clojure Core Audit](clojure-core-audit.md) | [Clojure String Audit](clojure-string-audit.md) | [Clojure Set Audit](clojure-set-audit.md)
1010

1111
## Summary
1212

docs/ptc-lisp-design-guidelines.md

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# PTC-Lisp Design Guidelines
2+
3+
Rules for deciding what belongs in PTC-Lisp and when it should diverge from Clojure.
4+
5+
PTC-Lisp is a Clojure-shaped language for LLM-generated, sandboxed programs. Clojure conformance is valuable because models already know Clojure idioms and conformance tests catch subtle behavior bugs. It is not the top-level goal. The top-level goal is deterministic, bounded, recoverable data transformation inside an agent loop.
6+
7+
See also: [PTC-Lisp Specification](ptc-lisp-specification.md), [Function Reference](function-reference.md), and [Clojure Conformance Gaps](clojure-conformance-gaps.md).
8+
9+
## Design Priorities
10+
11+
Apply these in order when adding syntax, functions, or interop:
12+
13+
1. **Sandbox safety** - programs must be bounded in time, memory, and host access.
14+
2. **Recoverability for LLM code** - common bad inputs should produce guardable signal values when recovery is useful.
15+
3. **Clojure familiarity** - use Clojure names, arities, truthiness, collection behavior, and data idioms unless a higher priority overrides them.
16+
4. **Determinism** - avoid ambient state, uncontrolled time, random behavior, filesystem access, and network access except through explicit tools.
17+
5. **Small surface area** - prefer a compact set of predictable primitives over full language completeness.
18+
6. **Boundary clarity** - distinguish PTC-Lisp data functions, tool calls, and Java-named compatibility methods by name and behavior.
19+
20+
## What To Include
21+
22+
Include a feature when it:
23+
24+
- Helps with data transformation, filtering, aggregation, validation, string processing, JSON, or tool-result shaping.
25+
- Runs eagerly within sandbox limits.
26+
- Is deterministic and has no hidden global state.
27+
- Does not expose host capabilities, filesystem I/O, arbitrary class access, or runtime code loading.
28+
- Can be documented with a few examples that LLMs are likely to generate correctly.
29+
- Preserves the Clojure contract or has an explicit `DIV-*` rationale.
30+
31+
Good candidates: pure collection functions, predicates, threading forms, destructuring patterns, bounded regex helpers, JSON helpers, and small Java compatibility shims that models commonly generate.
32+
33+
Poor candidates: lazy or infinite sequences, macros, `eval`, `read-string`, mutable references, arbitrary host interop, dynamic vars, filesystem I/O, exception machinery, protocols, multimethods, and large abstraction systems.
34+
35+
## Clojure Conformance Rules
36+
37+
Use Clojure behavior as the default for Clojure-named functions and forms:
38+
39+
- Preserve truthiness: only `nil` and `false` are falsey.
40+
- Preserve return-value idioms: `and`/`or` return actual values, `seq` returns `nil` for empty collections, `some` returns the first truthy result.
41+
- Preserve nil-friendly data access: `get`, keyword lookup, `get-in`, `first`, `last`, and `nth` should remain easy to compose with `some->` and `when`.
42+
- Preserve names and arities when the Clojure contract is safe and bounded.
43+
- Test supported Clojure-compatible behavior against the conformance suite when practical.
44+
45+
If a feature is marked supported in the audits but behaves differently from Clojure, either fix it or move it to an intentional `DIV-*` entry with rationale.
46+
47+
## Intentional Divergence Rules
48+
49+
Diverge from Clojure when matching Clojure would make LLM-generated sandbox code less safe, less bounded, or less recoverable.
50+
51+
Prefer an intentional divergence when one of these applies:
52+
53+
- **Clojure raises for bad input data.** PTC-Lisp has no `try`/`catch`; raising terminates the program. For Clojure-named helpers, prefer signal values such as `nil`, `""`, `false`, or an empty collection when the caller can reasonably continue.
54+
- **Clojure relies on laziness or infinity.** PTC-Lisp is eager and bounded. Require finite inputs and explicit limits.
55+
- **Clojure relies on mutable or global runtime state.** Omit it unless there is a narrow deterministic substitute.
56+
- **Clojure exposes host power.** Omit it or provide a tiny whitelisted compatibility surface.
57+
- **Clojure's exact behavior would create plausible wrong output.** Prefer a clean signal over a value that looks valid but means "miss".
58+
59+
Do not signal when collapsing distinguishable failures would hide a likely code bug. The practical line is: properties of input data may signal; properties of the program should raise. For example, `(parse-long "abc")` returns `nil` because external text failed to parse, but `(+ 1 nil)`, invalid arity, or an unknown symbol raises because the generated program is wrong.
60+
61+
Every intentional divergence must be documented in [Clojure Conformance Gaps](clojure-conformance-gaps.md#intentional-divergences--by-design-not-bugs) with:
62+
63+
- the Clojure behavior,
64+
- the PTC-Lisp behavior,
65+
- the reason for diverging,
66+
- the expected caller idiom,
67+
- links from the function reference or spec when the function is user-visible,
68+
- a regression test that fails if the divergence is accidentally "fixed" back to Clojure behavior.
69+
70+
## Signal Values
71+
72+
Signal values are ordinary return values that let code keep running and branch explicitly:
73+
74+
| Signal | Use when |
75+
|--------|----------|
76+
| `nil` | Missing value, parse failure, absent match, invalid non-critical input |
77+
| `""` | String extraction miss where the result is naturally string-shaped |
78+
| `false` | Predicate cannot prove the property or receives unsupported input |
79+
| `[]` / `{}` / `#{}` | Collection result is naturally empty and the operation succeeded |
80+
81+
Use a signal value only when it is unlikely to hide a serious programmer fault. Arithmetic with `nil`, invalid arity, unknown symbols, and malformed tool calls should still raise because continuing would hide a bug in the generated program.
82+
83+
When choosing between signals, keep the output type stable. A string function should usually return a string signal such as `""`; a lookup or parser should usually return `nil`; a predicate should return `false`.
84+
85+
## Error Rules
86+
87+
Raise an execution error for programmer faults:
88+
89+
- syntax and parse errors,
90+
- invalid arity,
91+
- unknown symbols or tools,
92+
- non-callable values in call position,
93+
- type errors where no useful recovery signal exists,
94+
- invalid tool or catalog arguments,
95+
- sandbox limits such as timeout, memory, recursion, or iteration caps.
96+
97+
Return signal values for world faults and expected data misses:
98+
99+
- missing keys,
100+
- no regex match,
101+
- failed parse of external text,
102+
- absent JSON or malformed JSON in data supplied by a tool,
103+
- upstream failures that are explicitly modeled as recoverable.
104+
105+
This split is part of the language contract: program bugs should be loud; messy data should be composable.
106+
107+
Callers should convert a signal into `(fail ...)` when the missing or invalid value means the agent cannot complete the requested task. Otherwise, guard or filter the signal locally. See [Getting Started](guides/subagent-getting-started.md) for the multi-turn `return` / `fail` flow.
108+
109+
## Java-Named Methods
110+
111+
Java-named methods keep Java semantics.
112+
113+
Dot-prefixed forms such as `.substring`, `.indexOf`, `.length`, and date/time methods exist because LLMs often generate Java-shaped code. The dot prefix signals that the caller opted into Java compatibility. These methods should follow Java's arity, index, sentinel, and error behavior unless a specific method is documented otherwise.
114+
115+
This is intentionally different from Clojure-named helpers. For example:
116+
117+
- `.indexOf` returns `-1` when not found, matching Java.
118+
- `index-of` returns `nil` when not found, matching the safer Clojure-shaped PTC-Lisp idiom.
119+
- `.substring` raises on invalid indices, matching Java.
120+
- `subs` returns string-shaped signal values for out-of-range cases, per `DIV-22`.
121+
122+
Do not silently soften Java-named methods unless the method's name or docs make the new contract obvious. If safer behavior is needed, prefer an existing Clojure/PTC-named wrapper or choose a plain descriptive name that states the operation, not a `safe-*` prefix.
123+
124+
## Feature Review Checklist
125+
126+
Before adding a function or form, answer:
127+
128+
- What LLM-generated task does this make easier?
129+
- Is the operation pure and deterministic?
130+
- Can it be bounded without lazy evaluation?
131+
- Does it expose host capabilities or new side effects?
132+
- If Clojure would raise, should PTC-Lisp raise or return a signal value?
133+
- What is the smallest useful arity set?
134+
- Does it fit the existing data model, especially vector-first sequential data and string-keyed tool boundaries?
135+
- Should it appear in `priv/functions.exs`, `priv/function_audit.exs`, the function reference, the spec, or the conformance gaps doc?
136+
- What regression test pins the intended divergence?
137+
138+
If the answer depends on "models might generate it", prefer a narrow compatibility shim over a broad subsystem.
139+
140+
## Conformance Workflow
141+
142+
When checking Clojure conformance:
143+
144+
1. Run the existing conformance tests or add a minimal reproducer against SCI, Babashka, Joker, or direct Clojure output.
145+
2. Classify the result as a bug, missing candidate, not relevant, or intentional divergence.
146+
3. Fix bugs that violate the design priorities or common Clojure idioms.
147+
4. Document intentional divergences as `DIV-*` entries.
148+
5. Add a regression test that encodes the PTC-Lisp contract, not only the Clojure comparison.
149+
6. Link user-visible divergences from the spec and generated registry metadata.
150+
151+
Conformance should keep the language familiar. It should not force PTC-Lisp to inherit features that make sandboxed LLM programs harder to execute safely.

0 commit comments

Comments
 (0)