proper vertical cursor advance

[jerch]

echo -e '\x1bPq$-$-$-$-\x1b\\' should move the text cursor downwards, even if no pixels were modified (given sixel scrolling is on).

@hackerb9 If you find some time, can you confirm, that an "empty" sixel sequence (no pixel notion) with just GCR/GLFs would still advance the text cursor downwards on a vt340? Thats at least how I read the sixel docs, but kinda none of the terminals except xterm does it atm, so I am not sure if I misinterpret things here.

[j4james]

Prediction: The "image" will take up three rows, and the following prompt will be on the fourth row.

You've got four graphic linefeeds, making five sixel rows, and the default aspect ratio is 2:1. So that gives you an image height of 5 x 6 x 2 = 60 pixels. The VT340 has 20 pixels per row, so that's three rows, with the final text cursor position being on the third row. The echo adds another linefeed taking you to the fourth row.

[jerch]

@j4james Yep, thats what I would expect, but seriously - no TE except xterm does it atm, thus I wonder if thats actually true. (Or if all TEs minus xterm fall into the same trap as I did from a zero image --> no dimensions --> no advance, lol.)

[j4james]

Well it works in Reflection Desktop and in my Windows console sixel fork, but I couldn't find any others. Even XTerm didn't work for me so I'm not sure why it's working for you. I'm fairly sure I've checked out the latest code.

It's possible my interpretation is wrong, but bear in mind that sixel is meant to be output as it's received, so if you've got a stream of sixel pumping out graphic linefeeds, you should see the viewport scrolling, even if no actual pixel data has been received yet. I know there is a certain amount of buffering, but it's highly unlikely the terminal would buffer linefeeds indefinitely while waiting for pixels to arrive. What would be the point?

[jerch]

@j4james Oh well it is broken in recent xterm (just tested, thx for the headsup), but works in older version (still broken kinda there, as it advances too much, not the same as a sixel with pixels would have, lol).

I now just go with whatever the same sixel with at least one pixel would have advanced the cursor vertically. Really strange that kinda all xterm-alikes get it wrong, hmm.

[jerch]

It's possible my interpretation is wrong, but bear in mind that sixel is meant to be output as it's received, so if you've got a stream of sixel pumping out graphic linefeeds, you should see the viewport scrolling, even if no actual pixel data has been received yet. I know there is a certain amount of buffering, but it's highly unlikely the terminal would buffer linefeeds indefinitely while waiting for pixels to arrive. What would be the point?

Since sixel is coming from a printer I think a GLF should be respected in the terminal (resp. GLFs accounting for x text lines). The problem which prolly most TE impls have is the "empty image returned" trap - no dimensions, so no cursor advance either (just a guess). But thats looks certainly wrong to me, if one thinks sixel as a "pixel-printer-on-top, with vertical stop at last active printed sixel line".

[j4james]

The problem which prolly most TE impls have is the "empty image returned" trap - no dimensions, so no cursor advance either

Yeah, but what I'm saying is that's just a symptom of a bigger problem, which is that most TEs are treating sixel as an image format, when it's really a drawing mode. When you think of it as a drawing mode, you don't wait for the ST to find out how big the "image" is. The concept of "image size" doesn't have any meaning.

The raster attributes just specify an area of the display that is cleared in advance, but that doesn't limit where you can draw. You've got a kind of "pen" which you can move around, and which will scroll the viewport when moved offscreen. You can plot pixels at the pen's current location, and when the mode ends, the text cursor is simply positioned wherever the pen was.

[jerch]

So in theory it is possible to stay in sixel mode forever, doing all outputs from sixel notation (just like normal PRINT state), and moving things forward with GLFs. I think really none of the OSS TEs can handle that - well my addon certainly cannot, as I cannot create an infinite canvas to draw on. Here the scrollbuffer idea somewhat contradicts the graphics-on-top sixel idea.

While my sixel decoder supports partial output as it is basically operating at sixel band granularity (e.g. you can watch it filling the bands with colors, pretty much like printer head movements), my back integration in xterm.js cannot do that atm, as the overhead for more fine-grained graphics linking to text cell is really expensive, also I cannot directly draw&render atm, as this is even more expensive in JS (Well this would change with webgl, but support is still too flaky across browsers).

Edit: I might be able to change into an "endless" mode by separating final bands (not reachable by the text cursor anymore) from intermediate bands, that need more turnovers. Not sure yet where to set the buffering limit, as I cannot do that for every single band, that got altered by one sixel (in fact the very first impl worked that way, and it was slow as hell). Will see what I can do here...

[j4james]

So in theory it is possible to stay in sixel mode forever, doing all outputs from sixel notation (just like normal PRINT state), and moving things forward with GLFs.

Yep. A typical use case would be something like the output from a seismograph. And it doesn't necessarily require an infinitely growing canvas, because if you're drawing within margins, the top of the image will be erased as new output is generated on the bottom.

It could always be implemented as a series of image slices on terminals that don't support this concept, but it's a lot simpler just remaining in sixel mode the whole time. Especially back in days of the original hardware terminals where bandwidth would have been more of a constraint.

I think really none of the OSS TEs can handle that

Yeah, the only TEs I know of that can handle this are the commercial ones: Reflection Desktop, and IBM Personal Communications. My Windows fork does too, but that's essentially vaporware so I don't think that counts. 😉

[hackerb9]

Prediction: The "image" will take up three rows, and the following prompt will be on the fourth row.

@j4james is, as usual, correct. I added a test program that people can use to test their terminals: https://github.com/hackerb9/vt340test/blob/main/jerch/gcrglf.sh

[EDIT: Original script presumed the terminal's font size is 20 pixels high, but that has been fixed.]

Output should look something like this:

[jerch]

@hackerb9 Thx for this awsome test script. I still have an offset by 1 line error, but kinda on both sides:

I mainly have an issue in my decoder, which tries very hard to find the last pixel touched by sixels, which also cuts off your '$-' at the end of the 2nd sixel (thus cursor stops in last line overprinting lower part). In the sense of images it kinda makes sense to find the real dimensions this way, but it makes absolutely no sense in the graphics layer metaphor. Gonna change that.
My second issue is linked to this - as the addon does not respect custom aspect ratios my left side is also off (always doing 1:1). Manually fixing both issues I get this:

which quite nicely mimicks vt340's line progression (even resembles the tiny offset at the arrows).

Last but not least, all of these fail the test:

• xterm v369 (not newest release though)
• mlterm, 3.8.4 (default from distro)
• contour (latest release)
• WezTerm (quite recent version)

Basically all of them just output a single empty line, no matter how many GCRGLFs were provided.