Breaking schema changes go unnoticed due broken `$id` references

[apuig]

Jenkins and plugins versions report

updating from 2.504 to 2.543

What Operating System are you using (both controller, and any agents involved in the problem)?

Linux

Reproduction steps

Schema validation fails to detect obsolete or invalid attributes in YAML configurations due to broken $id references in the generated JSON schema. This allows configurations with deprecated attributes to pass validation, only to fail at runtime after Jenkins upgrades.

For example, jenkinsci/jenkins#25918 removed client IP from CSRF crumb calculation.

1. Use Jenkins 2.542 or earlier with JCasC plugin
2. Create YAML configuration:

jenkins:
  crumbIssuer:
    standard:
      excludeClientIPFromCrumb: false

3. Configuration loads successfully
4. Upgrade to Jenkins 2.543+
5. Validate with the new schema

Expected Results

• The validation can catch this attribute is not longer expected.

• A test like this should fail to indicate the breaking change.

Actual Results

• Schema validation fails to spot the error, and then during load the expected error is thrown

io.jenkins.plugins.casc.UnknownAttributesException: standard: Invalid configuration elements for type: 
class hudson.security.csrf.DefaultCrumbIssuer : excludeClientIPFromCrumb.

Anything else?

The generated schema contains broken (no definitions section) $id references for (at least) HeteroDescribableCconfigurators:

"crumbIssuer": {
    "oneOf": [
        {
            "additionalProperties": false,
            "properties": {"standard": {"$id": "#/definitions/hudson.security.csrf.DefaultCrumbIssuer"}}
        },
        {
            "additionalProperties": false,
            "properties": {"test": {"$id": "#/definitions/org.jvnet.hudson.test.TestCrumbIssuer"}}
        }
    ],
    "type": "object"
},

This non-existent #/definitions/..., causes the validator to accept any properties, such as obsolete attributes (like excludeClientIPFromCrumb), typos or completely made-up properties.

An schema like this will more accurately describe the current crumb standar implementation:

"crumbIssuer": {"oneOf": [
    {
        "additionalProperties": false,
        "properties": {"standard": {
            "additionalProperties": false,
            "type": "object",
            "properties": {"excludeClientIPFromCrumb": {"type": "boolean"}}
        }}
    },
    {
        "additionalProperties": false,
        "properties": {"test": {
            "additionalProperties": false,
            "type": "object",
            "properties": {}
        }}
    }
]},

After the Jenkins upgrade, the attribute will be removed (leaving "properties": {}), which will effectively prevent any attributes there. As a result, the validJenkinsBaseConfigWithSymbol.yml test will fail, highlighting the breaking change

---

please note, oneOf should not uses "type": "object"

[timja]

To be clear the issue is that export still contains them but they are not usable.

AND the expected behaviour is that export does not contain them?

[apuig]

The exported schema is correct

Thanks @timja for pushing on this, and sorry I initially overlooked the actual problem.
I was confused by SchemaGenerationTest passing including the excludeClientIPFromCrumb attribute. However, I now understand the test only validates YAML structure and its accepting this obsolete attibute.

I'm closing this issue since the core functionality is working as expected.

I'll consider opening a separate issue to improve SchemaGenerationTest so it enforces additionalProperties: false at all schema levels, not just the top level. This would help catch obsolete attributes during schema validation rather than only at configuration load time.

Sorry for the confusion! 🙏

[apuig]

After taking a closer look at the current schemas, I have completely rewritten the issue. The expectation remains similar (these changes should be easy to anticipate before loading) and improving the schemas would benefit not only the tests (which was my initial focus)