Displayer for QueryRowsResult #1138

Michu1596 · 2024-12-04T09:06:25Z

This PR adds Displayer for QueryRowsResult trying to mimic cqlsh way of displaying tables, but there are some differences:

I implement the display for the duration type differently, because I think cqlsh displays this type in an unclear way,for more details look in the comments.
When displaying collections (list, map, set, tuple, udt), I use quotes only for text as I think putting other types in quotes might be confusing.

Fixes: #999

Pre-review checklist

I have split my patch into logically separate commits.
All commit messages clearly explain what they change and why.
I added relevant tests for new features and bug fixes.
All commits compile, pass static checks and pass test.
PR description sums up the changes and reasons why they should be introduced.
I have provided docstrings for the public items that I want to introduce.
I have adjusted the documentation in ./docs/source/.
I added appropriate Fixes: annotations to PR description.

scylla/src/transport/query_result.rs

Michu1596 · 2024-12-08T20:51:12Z

screenshot shows results of command SCYLLA_URI="127.0.0.1:9042" cargo run --example displayer

note the difference between the way cqlsh is displaying duration

github-actions · 2025-01-13T13:53:10Z

cargo semver-checks found no API-breaking changes in this PR.
Checked commit: 30b30a3

wprzytula

💯 This is a really great job done! In the aspect of usability, I'm very satisfied.

🔧 There are a few items to be addressed, though, mainly about:

efficiency: I believe we can get rid of String allocations on the way to display,
more idiomatic code,
division into modules.

scylla/Cargo.toml

scylla/src/response/query_result.rs

examples/displayer.rs

wprzytula · 2025-01-15T11:38:17Z

scylla/src/response/query_result.rs

+    }
+}
+
+/// A utility struct for displaying rows received from the database in a [`QueryRowsResult`].


📌 As I noted elsewhere, we should consider making this work both with QueryRowsResult and with TypedRowStream.

scylla/src/response/query_result.rs

examples/Cargo.toml

scylla/Cargo.toml

scylla/src/response/query_result.rs

scylla/src/response/rows_displayer.rs

wprzytula · 2025-01-23T17:35:26Z

scylla/src/response/rows_displayer.rs

+impl fmt::Display for WrapperDisplay<'_, CqlDate> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        // example of formating date 2021-12-31
+        let magic_constant = 2055453495; // it is number of days from -5877641-06-23 to -250000-01-01
+                                         // around -250000-01-01  is the limit of naive date
+


❓ Please explain in detail what the "magic constant" is and why it is correct in your opinion.

I believe it is correct because the dates printed by the displayer are the same as those inserted into the database, this constant allows shifting the date calculation to a valid range, thus preventing from_ymd_opt() from returning None. The date -5877641-06-23 is the reference point for CqlDate.

scylla/src/response/rows_displayer.rs

Michu1596 · 2025-01-26T17:14:50Z

rebased on main

Michu1596 · 2025-01-26T19:08:26Z

addressed comments

wprzytula

💯 Looks good!

Could you please squash the 3 latter commits into the respective former 2? So that nothing is introduced in a bad way in one commit only to be fixed in another.

examples/displayer.rs

scylla/Cargo.toml

wprzytula · 2025-01-28T14:09:02Z

🔧 There are new merge conflicts; when squashing the commits I asked you for, please remember to rebase on main.

wprzytula

💯 🎉 Looks good to me! Thank you for the contribution!

Lorak-mmk · 2025-03-13T13:02:54Z

@Michu1596 Thanks for the screenshots comparing the display of cqlsh and your displayer.
Could you provide additional ones for collections? In the PR description you mentioned that you display them a bit differently, but I don't see any examples of that on existing screenshots.

Lorak-mmk

This code desperately needs:

Tests
Proper commit messages and cover letter

I am sure writing this needed a lot of design decisions. Those design decisions and limitations should be all described in PR and commits. I managed to decipher some of the approaches, but not all, and I have no idea how correct I am. I also should not have to guess why the code is written the way it is - it should be explained.

I managed to get through some of the changes. I'll go through the rest after proper descriptions and tests are added.

Lorak-mmk · 2025-03-13T13:04:47Z

examples/Cargo.toml

+
+
+[[example]]
+name = "displayer"
+path = "displayer.rs"


⛏ Missing newline

Lorak-mmk · 2025-03-13T13:09:19Z

examples/displayer.rs

It's nice that you added a new example for the displayer.
Do you think you could modify our cqlsh-rs example to utilize the new displayer, instead of using its simplistic print_result function?

Lorak-mmk · 2025-03-13T13:12:13Z

scylla/src/response/query_result.rs

+        // use of QueryRowsResult after use of displayer
+        #[cfg(feature = "result-displayer")]
+        {
+            let rr = sample_raw_rows(2, 1);
+            let rqr = QueryResult::new(Some(rr), None, Vec::new());
+            let qr: QueryRowsResult = rqr.into_rows_result().unwrap();
+            let mut displayer = qr.rows_displayer();
+            displayer.set_terminal_width(80);
+            displayer.set_blob_displaying(ByteDisplaying::Hex);
+            displayer.use_color(true);
+            let _ = format!("{}", displayer);
+            let rows = qr.rows::<(&str, bool)>();
+
+            let mut rows_data = rows.unwrap();
+            let row = rows_data.next().unwrap().unwrap();
+
+            assert_eq!(row, ("MOCK", true));
+        }


This test is already gigantic, can you put it in a separate test?
If you put the tests in the rows_displayer module, you won't need to guard it with cfg.

(@wprzytula is there a reason for this being a 300 lines test instead of few smaller ones? test cases in it seem independent at the first glance).

Hmm.... is this the only test for this feature? We definitely need more.
Check out history tests (history.rs file) - it is also a module that produces text output, and has tests for its correctness.

(btw @wprzytula is there a reason for this being a 300 lines test instead of few smaller ones? test cases in it seem independent at the first glance).

It was easier to be written as a single test, just that.

Lorak-mmk · 2025-03-13T13:58:13Z

scylla/src/response/rows_displayer.rs

+    /// Sets the terminal width for wrapping the table output.
+    /// The table will be wrapped to the given width, if possible.
+    /// If the width is set to 0, the table will not be wrapped.
+    /// The default value is 0.
+    pub fn set_terminal_width(&mut self, terminal_width: usize) {
+        self.display_settings.terminal_width = terminal_width;
+    }


Instead of special value (0) this method could accept Option, with None indicating that wrapping should be disabled. It's a more Rusty way.

Lorak-mmk · 2025-03-13T13:59:55Z

scylla/src/response/rows_displayer.rs

+    /// Sets the exponent display for integers.
+    /// If set to true, integers will be displayed in scientific notation.
+    /// The default value is false.
+    pub fn set_exponent_displaying_integers(&mut self, exponent_displaying_integers: bool) {
+        self.display_settings.exponent_displaying_integers = exponent_displaying_integers;
+    }
+
+    /// Sets the exponent display for floats and doubles.
+    pub fn set_floating_point_precision(&mut self, precision: usize) {
+        self.display_settings.double_precision = precision;
+    }
+}


Comment on the second function seems to not describe what the function really does. copypaste error?
Also, please add the function that actually sets exponent displaying for floating point numbers - I see there is such a field in RowsDisplayerSettings.

This is a great example of why tests are needed.

Lorak-mmk · 2025-03-13T15:02:06Z

scylla/src/response/rows_displayer.rs

+        for byte in self.value {
+            match self.settings.byte_displaying {
+                ByteDisplaying::Ascii => {
+                    write_colored!(
+                        f,
+                        self.settings.print_in_color,
+                        self.settings.colors.blob,
+                        "{}",
+                        *byte as char
+                    )?;
+                }
+                ByteDisplaying::Hex => {
+                    write_colored!(
+                        f,
+                        self.settings.print_in_color,
+                        self.settings.colors.blob,
+                        "{:02x}",
+                        byte
+                    )?;
+                }
+                ByteDisplaying::Dec => {
+                    write_colored!(
+                        f,
+                        self.settings.print_in_color,
+                        self.settings.colors.blob,
+                        "{}",
+                        byte
+                    )?;
+                }
+            }
+        }


Oh wow. Here you call write_colored! for every single byte.
This macro will write, apart from the byte itself, 19 other bytes of ansi escape codes.
Please fix this. Write ascii escape codes once, then write the buffer, and then the closing ascii codes.

Lorak-mmk · 2025-03-13T15:03:23Z

scylla/src/response/rows_displayer.rs

+                ByteDisplaying::Dec => {
+                    write_colored!(
+                        f,
+                        self.settings.print_in_color,
+                        self.settings.colors.blob,
+                        "{}",
+                        byte
+                    )?;
+                }


This will write each byte as decimal, and concatenate this.
This will result in a very unreadable output. Is that what cqlsh does?
Even if it is, we should do it in some actually usable way.

Lorak-mmk · 2025-03-13T15:06:35Z

docs/source/statements/displaying-results.md

+Options:
+- `Hex`: Display as hexadecimal values (e.g., "0A FF 42")
+- `Ascii`: Display as ASCII characters where possible
+- `Dec`: Display as decimal values (e.g., "213 7 66")
+


Is that correct? In the implementation I did not see spaces between values. In your screenshot it also looked differently than here.

Lorak-mmk · 2025-03-13T15:08:24Z

docs/source/statements/displaying-results.md

+## Best Practices
+
+1. **Terminal Width**
+   - Set appropriate terminal width for better readability
+   - Consider using terminal width detection if available
+   - Use 0 width for untruncated output
+
+2. **Color Usage**
+   - Enable colors for better type distinction
+   - Disable colors when outputting to files or non-terminal destinations
+   - Consider user accessibility settings
+
+3. **Binary Data**
+   - Choose appropriate blob display format based on data content
+   - Use Hex for general binary data
+   - Use ASCII when data is known to be text
+   - Use Dec for byte-oriented analysis
+
+4. **Number Formatting**
+   - Adjust floating point precision based on data requirements
+   - Enable scientific notation for very large/small numbers
+   - Consider locale-specific formatting needs
+


Is this section LLM-generated? It contains a lot of text, yet not a single useful information.

Lorak-mmk · 2025-03-13T15:09:43Z

docs/source/statements/displaying-results.md

+## Implementation Details
+
+The displayer uses the following traits internally:
+- `Display` for converting values to strings
+- Custom formatting traits for specific types
+
+Output is generated using Rust's formatting system (`fmt::Display`), ensuring efficient memory usage and proper error handling.


Why would a user care about implementation details?
Why are we talking about efficient memory usage if the formatter uses Row and CqlValue, allocated all the rows at once, and boxes all the values again?

…qlsh` and examples of usage

wprzytula changed the title ~~Displayer for QueryRowsResult #999~~ Displayer for QueryRowsResult Dec 4, 2024

Michu1596 force-pushed the query_rows_result_displayer branch from c8ad86d to ddf3d5d Compare December 6, 2024 16:20

wprzytula reviewed Dec 8, 2024

View reviewed changes

scylla/src/transport/query_result.rs Outdated Show resolved Hide resolved

Michu1596 force-pushed the query_rows_result_displayer branch from 3a3f682 to e15634e Compare January 10, 2025 16:57

Michu1596 marked this pull request as ready for review January 10, 2025 17:00

Michu1596 force-pushed the query_rows_result_displayer branch from e15634e to e822850 Compare January 13, 2025 13:47

wprzytula requested review from wprzytula and muzarski January 13, 2025 13:51

Michu1596 force-pushed the query_rows_result_displayer branch 4 times, most recently from 54ad40d to 7b10b8b Compare January 15, 2025 11:04

wprzytula requested changes Jan 15, 2025

View reviewed changes

Michu1596 force-pushed the query_rows_result_displayer branch from 7b10b8b to 7687302 Compare January 15, 2025 11:43

wprzytula added the QoL Quality of Life enhancements label Jan 15, 2025

wprzytula assigned Michu1596 Jan 15, 2025

wprzytula added this to the 1.x.0 milestone Jan 15, 2025

github-actions bot added the semver-checks-breaking cargo-semver-checks reports that this PR introduces breaking API changes label Jan 21, 2025

Michu1596 force-pushed the query_rows_result_displayer branch from 222febc to 549fd82 Compare January 22, 2025 10:26

github-actions bot removed the semver-checks-breaking cargo-semver-checks reports that this PR introduces breaking API changes label Jan 22, 2025

wprzytula requested review from wprzytula and Lorak-mmk January 22, 2025 11:36

wprzytula requested changes Jan 23, 2025

View reviewed changes

Michu1596 force-pushed the query_rows_result_displayer branch from 549fd82 to 9b05d0e Compare January 26, 2025 17:12

wprzytula self-requested a review January 26, 2025 21:10

wprzytula reviewed Jan 27, 2025

View reviewed changes

examples/displayer.rs Outdated Show resolved Hide resolved

scylla/Cargo.toml Outdated Show resolved Hide resolved

Michu1596 force-pushed the query_rows_result_displayer branch 3 times, most recently from 369b40c to 20d1534 Compare January 29, 2025 08:52

wprzytula previously approved these changes Jan 29, 2025

View reviewed changes

wprzytula modified the milestones: 1.x.0, 1.1.0 Mar 7, 2025

Michu1596 dismissed wprzytula’s stale review via cbad25f March 8, 2025 19:45

Michu1596 force-pushed the query_rows_result_displayer branch 2 times, most recently from 57d9a2f to 475ab0d Compare March 8, 2025 22:05

Lorak-mmk requested changes Mar 13, 2025

View reviewed changes

Michu1596 added 2 commits March 13, 2025 21:46

Addad RowsDisplayer for displaying results of QueryRowsResult like `c…

54296d6

…qlsh` and examples of usage

added docs for displaying query results

30b30a3

Michu1596 force-pushed the query_rows_result_displayer branch from 475ab0d to 30b30a3 Compare March 13, 2025 20:47

Lorak-mmk modified the milestones: 1.1.0, 1.2.0 Apr 3, 2025

wprzytula assigned wprzytula and unassigned Michu1596 Apr 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Displayer for QueryRowsResult #1138

Displayer for QueryRowsResult #1138

Michu1596 commented Dec 4, 2024 •

edited

Loading

Michu1596 commented Dec 8, 2024

github-actions bot commented Jan 13, 2025 •

edited

Loading

wprzytula left a comment

wprzytula Jan 15, 2025

wprzytula Jan 23, 2025

Michu1596 Jan 26, 2025

Michu1596 commented Jan 26, 2025

Michu1596 commented Jan 26, 2025

wprzytula left a comment

wprzytula commented Jan 28, 2025

wprzytula left a comment

Lorak-mmk commented Mar 13, 2025

Lorak-mmk left a comment

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025 •

edited

Loading

Lorak-mmk Mar 13, 2025

wprzytula Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Lorak-mmk Mar 13, 2025

Displayer for QueryRowsResult #1138

Are you sure you want to change the base?

Displayer for QueryRowsResult #1138

Conversation

Michu1596 commented Dec 4, 2024 • edited Loading

Pre-review checklist

Michu1596 commented Dec 8, 2024

github-actions bot commented Jan 13, 2025 • edited Loading

wprzytula left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Michu1596 commented Jan 26, 2025

Michu1596 commented Jan 26, 2025

wprzytula left a comment

Choose a reason for hiding this comment

wprzytula commented Jan 28, 2025

wprzytula left a comment

Choose a reason for hiding this comment

Lorak-mmk commented Mar 13, 2025

Lorak-mmk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lorak-mmk Mar 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Michu1596 commented Dec 4, 2024 •

edited

Loading

github-actions bot commented Jan 13, 2025 •

edited

Loading

Lorak-mmk Mar 13, 2025 •

edited

Loading