Resolved AsyncAPI 3.0 specification becomes invalid when channel.servers is dereferenced

[MichakrawSB]

According to the AsyncAPI 3.0 specification, the Channel Object includes a servers property, which must be a list of references. The documentation also states:

RECOMMENDED that parsers (or other software) dereference this property for a better development experience.

https://www.asyncapi.com/docs/reference/specification/v3.0.0#channelObject

However, in practice, when the specification is fully resolved (all $ref replaced with actual objects), the resulting document is no longer a valid AsyncAPI specification. This leads to issues such as:

The resolved spec cannot be rendered by asyncapi-react.
Tools expecting a valid AsyncAPI document fail because the servers property now contains objects instead of references.

Questions:

• why was this change introduced in AsyncAPI 3.0?
• is this intended behavior - that a fully dereferenced specification is not valid according to the AsyncAPI schema?
• if yes, what is the recommended approach for tools that need a resolved spec for rendering or processing?

Expected Behavior

• a resolved specification should remain valid and renderable by official AsyncAPI tools.

[github-actions]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[derberg]

please share example AsyncAPI document with $ref and after dereferencing + specify what tool did you use for dereferencing.

as you can see, servers is an array of $ref, so ultimate document passed to docs generation, should not contain real server object - I'm guessing that now you are getting simply an error that document you have is not valid, right?

[MichakrawSB]

@derberg thanks for quick response.
Tool that we are using, or rather trying implement resolve functionality is apidom.

example of unresolved file

asyncapi: 3.0.0
info:
  title: Streetlights Kafka API
  version: 1.0.0
  description: "The Smartylighting Streetlights API allows you to remotely manage the city lights.\n\n### Check out its awesome features:\n\n* Turn a specific streetlight on/off \U0001F303\n* Dim a specific streetlight \U0001F60E\n* Receive real-time information about environmental lighting conditions \U0001F4C8\n"
  license:
    name: Apache 2.0
    url: 'https://www.apache.org/licenses/LICENSE-2.0'
defaultContentType: application/json
servers:
  scram-connections:
    host: 'test.mykafkacluster.org:18092'
    protocol: kafka-secure
    description: Test broker secured with scramSha256
    security:
      - $ref: '#/components/securitySchemes/saslScram'
    tags:
      - name: 'env:test-scram'
        description: >-
          This environment is meant for running internal tests through
          scramSha256
      - name: 'kind:remote'
        description: This server is a remote server. Not exposed by the application
      - name: 'visibility:private'
        description: This resource is private and only available to certain users
  mtls-connections:
    host: 'test.mykafkacluster.org:28092'
    protocol: kafka-secure
    description: Test broker secured with X509
    security:
      - $ref: '#/components/securitySchemes/certs'
    tags:
      - name: 'env:test-mtls'
        description: This environment is meant for running internal tests through mtls
      - name: 'kind:remote'
        description: This server is a remote server. Not exposed by the application
      - name: 'visibility:private'
        description: This resource is private and only available to certain users
channels:
  lightingMeasured:
    address: 'smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured'
    messages:
      lightMeasured:
        $ref: '#/components/messages/lightMeasured'
    description: The topic on which measured values may be produced and consumed.
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
  lightTurnOn:
    address: 'smartylighting.streetlights.1.0.action.{streetlightId}.turn.on'
    messages:
      turnOn:
        $ref: '#/components/messages/turnOnOff'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
  lightTurnOff:
    address: 'smartylighting.streetlights.1.0.action.{streetlightId}.turn.off'
    messages:
      turnOff:
        $ref: '#/components/messages/turnOnOff'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
  lightsDim:
    address: 'smartylighting.streetlights.1.0.action.{streetlightId}.dim'
    messages:
      dimLight:
        $ref: '#/components/messages/dimLight'
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
operations:
  receiveLightMeasurement:
    action: receive
    channel:
      $ref: '#/channels/lightingMeasured'
    summary: >-
      Inform about environmental lighting conditions of a particular
      streetlight.
    traits:
      - $ref: '#/components/operationTraits/kafka'
    messages:
      - $ref: '#/channels/lightingMeasured/messages/lightMeasured'
  turnOn:
    action: send
    channel:
      $ref: '#/channels/lightTurnOn'
    traits:
      - $ref: '#/components/operationTraits/kafka'
    messages:
      - $ref: '#/channels/lightTurnOn/messages/turnOn'
  turnOff:
    action: send
    channel:
      $ref: '#/channels/lightTurnOff'
    traits:
      - $ref: '#/components/operationTraits/kafka'
    messages:
      - $ref: '#/channels/lightTurnOff/messages/turnOff'
  dimLight:
    action: send
    channel:
      $ref: '#/channels/lightsDim'
    traits:
      - $ref: '#/components/operationTraits/kafka'
    messages:
      - $ref: '#/channels/lightsDim/messages/dimLight'
components:
  messages:
    lightMeasured:
      name: lightMeasured
      title: Light measured
      summary: >-
        Inform about environmental lighting conditions of a particular
        streetlight.
      contentType: application/json
      traits:
        - $ref: '#/components/messageTraits/commonHeaders'
      payload:
        $ref: '#/components/schemas/lightMeasuredPayload'
    turnOnOff:
      name: turnOnOff
      title: Turn on/off
      summary: Command a particular streetlight to turn the lights on or off.
      traits:
        - $ref: '#/components/messageTraits/commonHeaders'
      payload:
        $ref: '#/components/schemas/turnOnOffPayload'
    dimLight:
      name: dimLight
      title: Dim light
      summary: Command a particular streetlight to dim the lights.
      traits:
        - $ref: '#/components/messageTraits/commonHeaders'
      payload:
        $ref: '#/components/schemas/dimLightPayload'
  schemas:
    lightMeasuredPayload:
      type: object
      properties:
        lumens:
          type: integer
          minimum: 0
          description: Light intensity measured in lumens.
        sentAt:
          $ref: '#/components/schemas/sentAt'
    turnOnOffPayload:
      type: object
      properties:
        command:
          type: string
          enum:
            - 'on'
            - 'off'
          description: Whether to turn on or off the light.
        sentAt:
          $ref: '#/components/schemas/sentAt'
    dimLightPayload:
      type: object
      properties:
        percentage:
          type: integer
          description: Percentage to which the light should be dimmed to.
          minimum: 0
          maximum: 100
        sentAt:
          $ref: '#/components/schemas/sentAt'
    sentAt:
      type: string
      format: date-time
      description: Date and time when the message was sent.
  securitySchemes:
    saslScram:
      type: scramSha256
      description: Provide your username and password for SASL/SCRAM authentication
    certs:
      type: X509
      description: Download the certificate files from service provider
  parameters:
    streetlightId:
      description: The ID of the streetlight.
  messageTraits:
    commonHeaders:
      headers:
        type: object
        properties:
          my-app-header:
            type: integer
            minimum: 0
            maximum: 100
  operationTraits:
    kafka:
      bindings:
        kafka:
          clientId:
            type: string
            enum:
              - my-app-id

and

resolved file

asyncapi: 3.0.0
info:
  title: Streetlights Kafka API
  version: 1.0.0
  description: |
    The Smartylighting Streetlights API allows you to remotely manage the city lights.

    ### Check out its awesome features:

    * Turn a specific streetlight on/off U0001F303
    * Dim a specific streetlight U0001F60E
    * Receive real-time information about environmental lighting conditions U0001F4C8
  license:
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0
defaultContentType: application/json
servers:
  scram-connections:
    host: test.mykafkacluster.org:18092
    protocol: kafka-secure
    description: Test broker secured with scramSha256
    security:
    - type: scramSha256
      description: Provide your username and password for SASL/SCRAM authentication
    tags:
    - name: env:test-scram
      description: This environment is meant for running internal tests through scramSha256
    - name: kind:remote
      description: This server is a remote server. Not exposed by the application
    - name: visibility:private
      description: This resource is private and only available to certain users
  mtls-connections:
    host: test.mykafkacluster.org:28092
    protocol: kafka-secure
    description: Test broker secured with X509
    security:
    - type: X509
      description: Download the certificate files from service provider
    tags:
    - name: env:test-mtls
      description: This environment is meant for running internal tests through mtls
    - name: kind:remote
      description: This server is a remote server. Not exposed by the application
    - name: visibility:private
      description: This resource is private and only available to certain users
channels:
  lightingMeasured:
    address: "smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured"
    messages:
      lightMeasured:
        name: lightMeasured
        title: Light measured
        summary: Inform about environmental lighting conditions of a particular streetlight.
        contentType: application/json
        traits:
        - headers:
            type: object
            properties:
              my-app-header:
                type: integer
                minimum: 0
                maximum: 100
        payload:
          type: object
          properties:
            lumens:
              type: integer
              minimum: 0
              description: Light intensity measured in lumens.
            sentAt:
              type: string
              format: date-time
              description: Date and time when the message was sent.
    description: The topic on which measured values may be produced and consumed.
    parameters:
      streetlightId:
        description: The ID of the streetlight.
  lightTurnOn:
    address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.on"
    messages:
      turnOn:
        name: turnOnOff
        title: Turn on/off
        summary: Command a particular streetlight to turn the lights on or off.
        traits:
        - headers:
            type: object
            properties:
              my-app-header:
                type: integer
                minimum: 0
                maximum: 100
        payload:
          type: object
          properties:
            command:
              type: string
              enum:
              - "on"
              - "off"
              description: Whether to turn on or off the light.
            sentAt:
              type: string
              format: date-time
              description: Date and time when the message was sent.
    parameters:
      streetlightId:
        description: The ID of the streetlight.
  lightTurnOff:
    address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.off"
    messages:
      turnOff:
        name: turnOnOff
        title: Turn on/off
        summary: Command a particular streetlight to turn the lights on or off.
        traits:
        - headers:
            type: object
            properties:
              my-app-header:
                type: integer
                minimum: 0
                maximum: 100
        payload:
          type: object
          properties:
            command:
              type: string
              enum:
              - "on"
              - "off"
              description: Whether to turn on or off the light.
            sentAt:
              type: string
              format: date-time
              description: Date and time when the message was sent.
    parameters:
      streetlightId:
        description: The ID of the streetlight.
  lightsDim:
    address: "smartylighting.streetlights.1.0.action.{streetlightId}.dim"
    messages:
      dimLight:
        name: dimLight
        title: Dim light
        summary: Command a particular streetlight to dim the lights.
        traits:
        - headers:
            type: object
            properties:
              my-app-header:
                type: integer
                minimum: 0
                maximum: 100
        payload:
          type: object
          properties:
            percentage:
              type: integer
              description: Percentage to which the light should be dimmed to.
              minimum: 0
              maximum: 100
            sentAt:
              type: string
              format: date-time
              description: Date and time when the message was sent.
    parameters:
      streetlightId:
        description: The ID of the streetlight.
operations:
  receiveLightMeasurement:
    action: receive
    channel:
      address: "smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured"
      messages:
        lightMeasured:
          name: lightMeasured
          title: Light measured
          summary: Inform about environmental lighting conditions of a particular streetlight.
          contentType: application/json
          traits:
          - headers:
              type: object
              properties:
                my-app-header:
                  type: integer
                  minimum: 0
                  maximum: 100
          payload:
            type: object
            properties:
              lumens:
                type: integer
                minimum: 0
                description: Light intensity measured in lumens.
              sentAt:
                type: string
                format: date-time
                description: Date and time when the message was sent.
      description: The topic on which measured values may be produced and consumed.
      parameters:
        streetlightId:
          description: The ID of the streetlight.
    summary: Inform about environmental lighting conditions of a particular streetlight.
    traits:
    - bindings:
        kafka:
          clientId:
            type: string
            enum:
            - my-app-id
    messages:
    - name: lightMeasured
      title: Light measured
      summary: Inform about environmental lighting conditions of a particular streetlight.
      contentType: application/json
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          lumens:
            type: integer
            minimum: 0
            description: Light intensity measured in lumens.
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
  turnOn:
    action: send
    channel:
      address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.on"
      messages:
        turnOn:
          name: turnOnOff
          title: Turn on/off
          summary: Command a particular streetlight to turn the lights on or off.
          traits:
          - headers:
              type: object
              properties:
                my-app-header:
                  type: integer
                  minimum: 0
                  maximum: 100
          payload:
            type: object
            properties:
              command:
                type: string
                enum:
                - "on"
                - "off"
                description: Whether to turn on or off the light.
              sentAt:
                type: string
                format: date-time
                description: Date and time when the message was sent.
      parameters:
        streetlightId:
          description: The ID of the streetlight.
    traits:
    - bindings:
        kafka:
          clientId:
            type: string
            enum:
            - my-app-id
    messages:
    - name: turnOnOff
      title: Turn on/off
      summary: Command a particular streetlight to turn the lights on or off.
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          command:
            type: string
            enum:
            - "on"
            - "off"
            description: Whether to turn on or off the light.
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
  turnOff:
    action: send
    channel:
      address: "smartylighting.streetlights.1.0.action.{streetlightId}.turn.off"
      messages:
        turnOff:
          name: turnOnOff
          title: Turn on/off
          summary: Command a particular streetlight to turn the lights on or off.
          traits:
          - headers:
              type: object
              properties:
                my-app-header:
                  type: integer
                  minimum: 0
                  maximum: 100
          payload:
            type: object
            properties:
              command:
                type: string
                enum:
                - "on"
                - "off"
                description: Whether to turn on or off the light.
              sentAt:
                type: string
                format: date-time
                description: Date and time when the message was sent.
      parameters:
        streetlightId:
          description: The ID of the streetlight.
    traits:
    - bindings:
        kafka:
          clientId:
            type: string
            enum:
            - my-app-id
    messages:
    - name: turnOnOff
      title: Turn on/off
      summary: Command a particular streetlight to turn the lights on or off.
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          command:
            type: string
            enum:
            - "on"
            - "off"
            description: Whether to turn on or off the light.
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
  dimLight:
    action: send
    channel:
      address: "smartylighting.streetlights.1.0.action.{streetlightId}.dim"
      messages:
        dimLight:
          name: dimLight
          title: Dim light
          summary: Command a particular streetlight to dim the lights.
          traits:
          - headers:
              type: object
              properties:
                my-app-header:
                  type: integer
                  minimum: 0
                  maximum: 100
          payload:
            type: object
            properties:
              percentage:
                type: integer
                description: Percentage to which the light should be dimmed to.
                minimum: 0
                maximum: 100
              sentAt:
                type: string
                format: date-time
                description: Date and time when the message was sent.
      parameters:
        streetlightId:
          description: The ID of the streetlight.
    traits:
    - bindings:
        kafka:
          clientId:
            type: string
            enum:
            - my-app-id
    messages:
    - name: dimLight
      title: Dim light
      summary: Command a particular streetlight to dim the lights.
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          percentage:
            type: integer
            description: Percentage to which the light should be dimmed to.
            minimum: 0
            maximum: 100
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
components:
  messages:
    lightMeasured:
      name: lightMeasured
      title: Light measured
      summary: Inform about environmental lighting conditions of a particular streetlight.
      contentType: application/json
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          lumens:
            type: integer
            minimum: 0
            description: Light intensity measured in lumens.
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
    turnOnOff:
      name: turnOnOff
      title: Turn on/off
      summary: Command a particular streetlight to turn the lights on or off.
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          command:
            type: string
            enum:
            - "on"
            - "off"
            description: Whether to turn on or off the light.
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
    dimLight:
      name: dimLight
      title: Dim light
      summary: Command a particular streetlight to dim the lights.
      traits:
      - headers:
          type: object
          properties:
            my-app-header:
              type: integer
              minimum: 0
              maximum: 100
      payload:
        type: object
        properties:
          percentage:
            type: integer
            description: Percentage to which the light should be dimmed to.
            minimum: 0
            maximum: 100
          sentAt:
            type: string
            format: date-time
            description: Date and time when the message was sent.
  schemas:
    lightMeasuredPayload:
      type: object
      properties:
        lumens:
          type: integer
          minimum: 0
          description: Light intensity measured in lumens.
        sentAt:
          type: string
          format: date-time
          description: Date and time when the message was sent.
    turnOnOffPayload:
      type: object
      properties:
        command:
          type: string
          enum:
          - "on"
          - "off"
          description: Whether to turn on or off the light.
        sentAt:
          type: string
          format: date-time
          description: Date and time when the message was sent.
    dimLightPayload:
      type: object
      properties:
        percentage:
          type: integer
          description: Percentage to which the light should be dimmed to.
          minimum: 0
          maximum: 100
        sentAt:
          type: string
          format: date-time
          description: Date and time when the message was sent.
    sentAt:
      type: string
      format: date-time
      description: Date and time when the message was sent.
  securitySchemes:
    saslScram:
      type: scramSha256
      description: Provide your username and password for SASL/SCRAM authentication
    certs:
      type: X509
      description: Download the certificate files from service provider
  parameters:
    streetlightId:
      description: The ID of the streetlight.
  messageTraits:
    commonHeaders:
      headers:
        type: object
        properties:
          my-app-header:
            type: integer
            minimum: 0
            maximum: 100
  operationTraits:
    kafka:
      bindings:
        kafka:
          clientId:
            type: string
            enum:
            - my-app-id

Below you can see how it look in AsyncApi Studio.

[robert-hebel-sb]

Hey @derberg 👋
The same issue can be reproduced using @asyncapi/parser library

Steps to reproduce:

1. Create a valid AsyncAPI 3 specs which has defined messages for channel. For example
   https://gist.github.com/robert-hebel-sb/5432fbc43624c20f54210261d2bdf969
2. Use @asyncapi/parser to parse definition

import { Parser } from '@asyncapi/parser';

const parser = new Parser();

.
.
.

const res = await parser.parse(content);

return res.document.json()

3. Paste parsed result in https://studio.asyncapi.com/
   https://gist.github.com/robert-hebel-sb/3f8a7817421f6e9a1bf542059a0dbc63

Expected behavior

• parsed spec is validated and preview is available

Actual behavior

• Error occurs in validation, no preview is displayed

I see for resolved specs altered async3 schema should be used. I'm not sure against which schema validation happens for the case described here
https://github.com/asyncapi/parser-js/blob/master/packages/parser/src/ruleset/functions/documentStructure.ts#L55

[maishivamhoo123]

@team i want to work on this isuue could you please assign this to me ?

[MichakrawSB]

@maishivamhoo123 what will be the solution? Does it mean that this is bug in @asyncapi/parser? After fix fully resolved specification should be validated and preview should be available?

[theAnuragMishra]

The problem is that the parser turns $refs into full objects, but the validator expects them to be references (according to the spec). This isn't an issue when validating for yourself, you should just validated unresolved schemas instead.
Now for use by other tools like asyncapi-react, I think we need to think of a long term idea for this.
My first instincts say we should let these tools accept fully resolved schemas instead of unresolved ones. Probably we need to have two different specs for 1. unresolved schemas, and 2. resolved ones. And let the tools accept resolved schemas instead.
Or if my understanding is right, even better if we could embed parsers in these tools and have a pre-validation step of parsing? And then we don't need to have two specs.

[theAnuragMishra]

@MichakrawSB why was the issue closed?

[MichakrawSB]

Sorry, my mistake

[SujalTripathi]

hello @MichakrawSB i am new to these organization and sir can i work on these issue and can you help me how i will contribute in these organization. Thanks

[theAnuragMishra]

You can find better places to ask such questions. Please don't add unnecessary distractions to ongoing issues.