Skip to content

fix(cli): surface actionable errors when OpenAPI spec generation fails#7592

Open
DogInfantry wants to merge 2 commits into
OpenBB-finance:v5from
DogInfantry:fix/7585-actionable-spec-errors
Open

fix(cli): surface actionable errors when OpenAPI spec generation fails#7592
DogInfantry wants to merge 2 commits into
OpenBB-finance:v5from
DogInfantry:fix/7585-actionable-spec-errors

Conversation

@DogInfantry

Copy link
Copy Markdown

Follow-up to #7585, as invited by @deeleeramone in #7585 (comment). #7586 and #7589 fixed the modular spec resolution and added zero-command guards to --generate-spec and --generate-extension. This PR covers the remaining places where a bad document or an unreachable server still produced a raw traceback or a silent empty session.

What changed

  1. Interactive openbb --server URL startup (cli/openbb_cli/cli.py)

    • A fetch failure (connection refused, timeout, HTTP error status, unparseable document) now prints a one-line actionable message to stderr and exits 2 instead of dumping a traceback through the top-level exception handler.
    • A document that resolves to zero GET/POST commands now aborts with the same guard message used by --generate-spec, instead of silently launching an empty REPL.
  2. --generate-spec fetch errors (cli/openbb_cli/cli.py)

    • httpx transport and status errors, and ValueError from parsing or bundling, are caught and reported as failed to fetch the OpenAPI document from <url>: <cause>, exit 2.
  3. Dangling JSON pointers in external refs (cli/openbb_cli/dispatchers/openapi_schema.py)

    • _resolve_json_pointer previously raised a bare KeyError: 'foo' or IndexError when a modular spec pointed at a missing member. It now raises ValueError naming the pointer and the failing part, for example: OpenAPI reference pointer #/paths/~1x not found: no member '/x'.
  4. Malformed YAML (cli/openbb_cli/dispatchers/openapi_schema.py)

    • yaml.YAMLError escaped the existing except (json.JSONDecodeError, ValueError) handling in fetch_openapi, so a broken YAML endpoint produced a raw scanner traceback. _yaml_load now converts it to ValueError("Document is not valid YAML: <first line>"), which integrates with the existing fallback logic.

Tests

  • cli/tests/test_cli.py: fetch failure and HTTP 500 during --generate-spec; fetch failure and zero-command guard on interactive --server startup.
  • cli/tests/test_dispatchers_openapi_schema.py: missing member and bad array index in _resolve_json_pointer; dangling pointer through _bundle_external_refs; malformed YAML through _parse_spec_text.

All of cli/tests/test_cli.py, cli/tests/test_dispatchers_openapi_schema.py, and cli/tests/test_dispatchers_spec.py pass locally (309 tests).

Follow-up to OpenBB-finance#7585. OpenBB-finance#7586 and OpenBB-finance#7589 fixed modular spec resolution and
added zero-command guards to --generate-spec and --generate-extension.
This covers the remaining places a bad document or unreachable server
still produced a raw traceback or a silent empty session:

- Interactive `openbb --server URL` startup: fetch failures now print a
  one-line message and exit 2 instead of a traceback, and a zero-command
  document aborts with the same guard used by --generate-spec instead of
  launching an empty REPL.
- --generate-spec fetch errors (httpx transport/status, parse/bundle
  ValueError) are caught and reported with the source URL.
- _resolve_json_pointer raises ValueError naming the pointer and the
  failing member instead of a bare KeyError/IndexError when an external
  $ref points at a missing node.
- _yaml_load converts yaml.YAMLError into ValueError so malformed YAML
  integrates with the existing fetch_openapi fallback instead of raising
  a raw scanner traceback.
@CLAassistant

CLAassistant commented Jul 15, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@deeleeramone

Copy link
Copy Markdown
Contributor

There's likely some middle ground between suppressing the Exception and traceback, and a just raw dumping the stack.

For this A/B, I found that on Windows, GitHub URLs needed to have the "raw=true" query parameter in order to fetch the actual file. What's notable about the raw error is that it also doesn't actually identify the problem with the file type. What got returned was a HTML content-type which could have an error message embedded in the text or body, yet still carried a 200 status.

From this PR branch, it's a little too sparse. So, maybe something like raise exc from None, and then if OPENBB_DEBUG_MODE is set, show the full stack? Ideally, this error would get caught at the request stage, before sending it to the yaml parser.

(obb5) PS C:\Users\dangl\github\OpenBB> openbb --generate-spec -o .\eodhd.spec --server 'https://eodhd.com' --openapi-path 'https://github.com/EodHistoricalData/EODHD-openapi/blob/main/openapi.yaml?raw=true'
wrote 82 commands to .\eodhd.spec
(obb5) PS C:\Users\dangl\github\OpenBB> openbb --generate-spec -o .\eodhd.spec --server 'https://eodhd.com' --openapi-path 'https://github.com/EodHistoricalData/EODHD-openapi/blob/main/openapi.yaml'         
failed to fetch the OpenAPI document from https://eodhd.com: Document is not valid YAML: mapping values are not allowed here

The full error:

(obb5) PS C:\Users\dangl\github\OpenBB> openbb --generate-spec -o .\eodhd.spec --server 'https://eodhd.com' --openapi-path 'https://github.com/EodHistoricalData/EODHD-openapi/blob/main/openapi.yaml'
Traceback (most recent call last):
  File "C:\Users\dangl\github\OpenBB\cli\openbb_cli\dispatchers\openapi_schema.py", line 588, in _parse_spec_text
    return json.loads(text)
           ~~~~~~~~~~^^^^^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\json\__init__.py", line 352, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\json\decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\json\decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 1 (char 6)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\dangl\miniconda3\envs\obb5\Scripts\openbb.exe\__main__.py", line 5, in <module>
    sys.exit(main())
             ~~~~^^
  File "C:\Users\dangl\github\OpenBB\cli\openbb_cli\cli.py", line 795, in main
    return _generate_spec(
        args.server,
    ...<4 lines>...
        socrata_story=getattr(args, "socrata_story", None),
    )
  File "C:\Users\dangl\github\OpenBB\cli\openbb_cli\cli.py", line 446, in _generate_spec
    openapi = fetch_openapi(
        server_url, path=openapi_path, headers=headers, query_params=query_params
    )
  File "C:\Users\dangl\github\OpenBB\cli\openbb_cli\dispatchers\openapi_schema.py", line 872, in fetch_openapi
    _parse_spec_text(
    ~~~~~~~~~~~~~~~~^
        response.text,
        ^^^^^^^^^^^^^^
        content_type=response.headers.get("content-type", ""),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ),
    ^
  File "C:\Users\dangl\github\OpenBB\cli\openbb_cli\dispatchers\openapi_schema.py", line 590, in _parse_spec_text
    return _yaml_load(text)
  File "C:\Users\dangl\github\OpenBB\cli\openbb_cli\dispatchers\openapi_schema.py", line 597, in _yaml_load
    return yaml.safe_load(text)
           ~~~~~~~~~~~~~~^^^^^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\__init__.py", line 81, in load
    return loader.get_single_data()
           ~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\composer.py", line 58, in compose_document
    self.get_event()
    ~~~~~~~~~~~~~~^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\parser.py", line 118, in get_event
    self.current_event = self.state()
                         ~~~~~~~~~~^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\parser.py", line 193, in parse_document_end
    token = self.peek_token()
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\scanner.py", line 129, in peek_token
    self.fetch_more_tokens()
    ~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\scanner.py", line 223, in fetch_more_tokens
    return self.fetch_value()
           ~~~~~~~~~~~~~~~~^^
  File "C:\Users\dangl\miniconda3\envs\obb5\Lib\site-packages\yaml\scanner.py", line 577, in fetch_value
    raise ScannerError(None, None,
            "mapping values are not allowed here",
            self.get_mark())
yaml.scanner.ScannerError: mapping values are not allowed here
  in "<unicode string>", line 35, column 28:
          --tab-size-preference: 4;
                               ^

… mode

Addresses review feedback on OpenBB-finance#7592.

- The fetch stage now detects an HTML page (content-type text/html or an
  <!doctype html>/<html> body) before handing it to the YAML parser and
  raises an actionable error pointing at the raw file URL. This replaces the
  confusing 'Document is not valid YAML' scanner error for GitHub blob URLs,
  while still extracting embedded specs from Swagger UI pages.
- Spec fetch failures print the one-line message by default and emit the full
  traceback only when OPENBB_DEBUG_MODE is set.
@DogInfantry

Copy link
Copy Markdown
Author

Thanks for the review. Pushed a follow-up addressing both points.

HTML at the fetch stage. _parse_spec_text now detects an HTML response (content-type text/html, or a body starting with <!doctype html>/<html>) before it reaches the YAML parser. If the page embeds a spec (Swagger UI) it is still extracted; otherwise it raises a clear error:

failed to fetch the OpenAPI document from https://eodhd.com: the URL returned an HTML page, not an OpenAPI document. If this is a GitHub 'blob' link, use the raw file URL (or append '?raw=true').

So the GitHub blob URL gives an actionable message instead of Document is not valid YAML: mapping values are not allowed here, and the ?raw=true form still writes 82 commands. The check lives in the shared parser, so every fetch path (explicit --openapi-path, landing page, and bundled external $refs) benefits.

Traceback visibility. The failure keeps the one-line message and exit code 2 by default, and now prints the full traceback when OPENBB_DEBUG_MODE is set. I kept the exit-2 path over raise ... from None so scripts get a deterministic exit code plus a clean message, with the debug env var as the escape hatch for the full stack.

New tests cover the HTML rejection, the embedded-spec regression, and the debug-gated traceback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants