Skip to content

Send console errors to stderr instead of stdout - #1792

Merged
pierrecamilleri merged 6 commits into
frictionlessdata:mainfrom
DMZ22:console-errors-to-stderr
Jul 27, 2026
Merged

Send console errors to stderr instead of stdout#1792
pierrecamilleri merged 6 commits into
frictionlessdata:mainfrom
DMZ22:console-errors-to-stderr

Conversation

@DMZ22

@DMZ22 DMZ22 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

print_error and print_exception rendered onto whatever console the command passed in, and every command builds a plain stdout Console(). So the reporter's case does this:

$ frictionless describe doesnotexist.csv --json > out.txt 2> err.txt
before:  stdout 324 bytes (the red error panel), stderr 0 bytes
after:   stdout 0 bytes,                          stderr 324 bytes

Both helpers now print to a module-level Console(stderr=True), including the debug=True path, which called console.print_exception() and would otherwise have kept leaking tracebacks to stdout.

@pierrecamilleri — your comment named helpers.py as the fix locus, which is right, but there is one call site in there that must not move:

extract.py:233 calls print_error(console, note="No rows found", title="Empty") from inside the table-rendering loop, after console.rule("[bold]Tables") and followed by continue. That is report content on a successful run (exit 0) using print_error purely for the red panel styling. Sending it to stderr would tear it out of the sequence of tables it sits between. So it moves to a new print_panel helper that renders on the passed console. Verified: frictionless extract empty.csv still prints "No rows found" to stdout, exit 0, with stderr empty.

I kept the console parameter on both helpers rather than dropping it and touching all 31 call sites across 12 command files. It is unused in print_error/print_exception now, which is a little untidy — happy to remove it and update the call sites if you'd prefer that diff.

Tests. Five console tests asserted error text on result.stdout; they now assert on result.stderr, which is precisely the behaviour under change:

  • test_describe.py::test_console_describe_error_not_found
  • test_transform.py::test_console_transform_error_not_found_source_issue_814
  • test_validate.py::test_console_validate_single_invalid_resource_221
  • test_extract.py::test_console_extract_single_invalid_resource
  • test_extract.py::test_console_extract_single_valid_resource_invalid_package

Two new tests pin the separation directly: an error run must leave stdout completely empty while stderr carries the message, and a successful --json run must still produce parseable JSON on stdout with stderr empty. Reverting the source while keeping the tests fails the first of those; the second passes either way, deliberately, so it catches over-correction.

pytest frictionless/console goes from 4 failed / 85 passed to 4 failed / 92 passed — the four are pre-existing dialect_sheet_option failures unrelated to this change and present on unmodified main. ruff check reports the same counts on both files I touched before and after (182 and 33), so this adds no new lint findings, though note those counts are from a newer ruff than the pinned one.

I did not add a CHANGELOG entry since there is no unreleased section, but this is a user-visible behaviour change for anyone parsing stdout, so say the word if you want one.

DMZ22 and others added 6 commits July 24, 2026 20:15
print_error and print_exception rendered onto the caller's stdout console,
so redirecting stdout captured the error panel: `frictionless describe
data.csv --json > out.json` wrote the red panel into out.json whenever the
source could not be read. Both now print to a module-level stderr console,
including the debug traceback path.

extract's "No rows found" panel used print_error purely for its styling,
but it is part of the report body on a successful run and sits between the
rendered tables, so it moves to a new print_panel helper and stays on
stdout.

The five console tests that asserted error text on stdout now assert on
stderr, which is the behaviour being fixed.
@pierrecamilleri
pierrecamilleri merged commit 3cf7b2c into frictionlessdata:main Jul 27, 2026
10 checks passed
@pierrecamilleri

Copy link
Copy Markdown
Collaborator

Thanks, I additionnaly removed the wordy LLM comments, and adapted the tests for python 3.9 support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Command line tools should send errors to stderr, not stdout

2 participants