Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Spec 1.2.1: each constraint must become MORE constrained in template processing order.
# The job template (processed last) widens the environment template's INT range, so the
# merge must be rejected. The default satisfies both ranges, isolating the widening rule.
template:
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: Count
type: INT
minValue: 0
maxValue: 1000
default: 50
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- -c
- print(r'Count={{Param.Count}}')
environments:
- specificationVersion: environment-2023-09
parameterDefinitions:
- name: Count
type: INT
minValue: 10
maxValue: 100
environment:
name: TestEnv
variables:
COUNT: '{{Param.Count}}'
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Spec 1.2.1: each constraint must become MORE constrained in template processing order.
# The job template (processed last) relaxes the environment template's minLength, so the
# merge must be rejected. The default satisfies both, isolating the widening rule.
template:
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: Value
type: STRING
minLength: 2
default: test-value
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- -c
- print(r'Value={{Param.Value}}')
environments:
- specificationVersion: environment-2023-09
parameterDefinitions:
- name: Value
type: STRING
minLength: 5
environment:
name: TestEnv
variables:
VAL: '{{Param.Value}}'
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Parameter type names are case-sensitive in base (case-insensitivity is EXPR-only); 'string' must be rejected.
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: MyParam
type: string
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- "-c"
- "print()"
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# An INT parameter allowedValues entry of 2^63 exceeds the signed 64-bit integer range and must be rejected.
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: MyParam
type: INT
allowedValues:
- 9223372036854775808
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- "-c"
- "print('{{Param.MyParam}}')"
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# An INT parameter default of 2^63 exceeds the signed 64-bit integer range and must be rejected.
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: MyParam
type: INT
default: 9223372036854775808
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- "-c"
- "print('{{Param.MyParam}}')"
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# An INT parameter default of -2^63-1 is below the signed 64-bit integer range and must be rejected.
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: MyParam
type: INT
default: -9223372036854775809
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- "-c"
- "print('{{Param.MyParam}}')"
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# An INT parameter maxValue of 2^63 exceeds the signed 64-bit integer range and must be rejected.
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: MyParam
type: INT
maxValue: 9223372036854775808
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- "-c"
- "print('{{Param.MyParam}}')"
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# An INT parameter minValue of 2^63 exceeds the signed 64-bit integer range and must be rejected.
specificationVersion: jobtemplate-2023-09
name: TestJob
parameterDefinitions:
- name: MyParam
type: INT
minValue: 9223372036854775808
steps:
- name: Step1
script:
actions:
onRun:
command: python
args:
- "-c"
- "print('{{Param.MyParam}}')"
64 changes: 64 additions & 0 deletions conformance-tests/2023-09/base/proposed/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Proposed conformance fixtures (expected failures)

Every fixture in this directory is believed to be **spec-correct but fails against the
current reference implementations**. They are parked here so the main suite stays green
while the implementation (or, where noted, the spec) is fixed. Each entry records the
observed behavior, the spec citation, and a classification. Once the underlying issue is
resolved, move the fixture into the appropriate sibling directory
(`job_templates/`, `jobs/`).

Verification environment: `openjd` CLI = openjd-rs debug build; cross-checks with
openjd-model-for-python via `decode_job_template` / `merge_job_parameter_definitions`.

## INT 64-bit boundary rejects (known live bug)

Spec §2.3: an INT parameter takes `<integer>` values. Both reference implementations
model INT as a signed 64-bit integer, so values outside [-2^63, 2^63-1] cannot round-trip
and must be rejected. Both implementations currently **accept** all five constructs below
(`openjd check` reports "passes validation checks"). The matching accept-twins at 2^63-1
and -2^63 live in `job_templates/` and pass.

| Fixture | Construct | Observed |
|---|---|---|
| `2.3--int-default-above-int64-max.invalid.yaml` | `default: 9223372036854775808` (2^63) | accepted by both |
| `2.3--int-minvalue-above-int64-max.invalid.yaml` | `minValue: 9223372036854775808` | accepted by both |
| `2.3--int-maxvalue-above-int64-max.invalid.yaml` | `maxValue: 9223372036854775808` | accepted by both |
| `2.3--int-allowedvalues-above-int64-max.invalid.yaml` | `allowedValues: [9223372036854775808]` | accepted by both |
| `2.3--int-default-below-int64-min.invalid.yaml` | `default: -9223372036854775809` (-2^63-1) | accepted by both |

Classification: **implementation bug** (validation-time bounds handling), one branch per
field. Note the *supplied value* branch is fixed/working: `jobs/2.3--int-value-above-int64-max.invalid.test.yaml`
is rejected correctly at job creation and lives in the main suite.

## Type-name case sensitivity

| Fixture | Construct | Observed |
|---|---|---|
| `2--type-lowercase.invalid.yaml` | `type: string` in base (no `extensions:`) | openjd-rs: **accepted**; python: rejected ("Input tag 'string' ... does not match any of the expected tags") |

Spec §2: `type: "STRING"` is a literal, and the spec states type names become
case-insensitive only "When the `EXPR` extension is enabled" (see RFC 0007). Base is
case-sensitive.

Classification: **openjd-rs implementation bug** (unconditional case-insensitive type
parsing); implementations diverge, python is spec-conformant.

## Merge widening (§1.2.1)

Spec §1.2.1: "each constraint must become more constrained as a subsequent definition is
merged with the previous ones in processing order" (Job Template last). These fixtures
have a Job Template that *widens* the Environment Template's constraint while the default
satisfies both ranges, isolating the direction rule from the merged-consistency rule.

| Fixture | Construct | Observed |
|---|---|---|
| `1.2.1--constraint-widening-int-range.invalid.test.yaml` | env `[10,100]`, job template `[0,1000]` | accepted by both; job runs |
| `1.2.1--constraint-widening-minlength.invalid.test.yaml` | env `minLength: 5`, job template `minLength: 2` | accepted by both; job runs |

Cross-check: `openjd.model.merge_job_parameter_definitions` returns the *intersection*
(`minValue=10, maxValue=100`) rather than rejecting — both implementations merge by
intersection and never enforce the narrowing direction.

Classification: **implementation gap in both implementations** (or the spec sentence
should be revised to define intersection semantics; until the spec changes, the stated
rule is normative and these fixtures encode it).
Loading