[feature] Terminal Graphics Protocol support

[SlyCedix]

I am currently working on a plugin which provides a native previewer for all kinds of files, including images. My plan was to utilize Kitty's Terminal Graphics Protocol and to just pass the rendering string directly through the body of a Dynamic ui element.

Despite the text being correctly formatted, as could be verified by saving it to a file and then using cat, when rendered in xplr it appears as text and runs off the screen.

By running xplr -c repro.lua you can see the issue. The body of the ui element displays unescaped text, but upon exiting, you can see that the image which was printed using io.write works. I'm testing this in kitty 0.45.0

repro.lua:

version = "1.1.0"

local tgptext = "\x1B_Ga=T,q=2,f=100,s=32,v=32;iVBORw0KGgoAAAANSUhEUgAAACAAAAAgAQMAAABJtOi3AAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAAGUExURf8AAP///0EdNBEAAAABYktHRAH/Ai3eAAAAB3RJTUUH6gIBBRoi2vHflQAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyNi0wMi0wMVQwNToyNjozNCswMDowMJYs8ncAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjYtMDItMDFUMDU6MjY6MzQrMDA6MDDncUrLAAAAKHRFWHRkYXRlOnRpbWVzdGFtcAAyMDI2LTAyLTAxVDA1OjI2OjM0KzAwOjAwsGRrFAAAAAxJREFUCNdjYBjcAAAAoAABYSV9RwAAAABJRU5ErkJggg==\x1B"
io.write(tgptext)
xplr.config.layouts.builtin.default = {
	Static = {
		CustomParagraph = {
			ui = { title = { format = "This should be an image!" } },
			body = tgptext,
		},
	},
}

[sayanarijit]

Rendering images in xplr might make the interaction very slow, since it doesn’t support async rendering yet.

Since previewing files is a rather complex issue, I have kept it out of scope to keep xplr minimal. But you can integrate xplr with external previewers.

Checkout the plugins: https://xplr.dev/en/awesome-plugins

Specially https://github.com/sayanarijit/preview-tabbed.xplr

[SlyCedix]

To be clear, this would not be rendering images with xplr directly, rather, just formatting escape sequences in lua, which is fairly trivial and fast, and printing it out to stdout. The rendering of the image is "async" automatically by virtue of the rendering being offloaded to the terminal emulator itself.

This could be made even more efficient by not directly encoding the image data in the output at all but instead use TGP Unicode Placeholders. sxyazi/yazi does this and it's extremely fast.

I assume what's actually happening here is that some sanitization is occurring, either in xplr or in ratatui which is preventing the escape code from printing appropriately.

[sayanarijit]

Ah interesting. Will look in it then.

[SlyCedix]

Thanks! The documentation from kitty is a little unclear on the exact formatting , so I recommend downloading kitty and using kitty +kitten icat --transport-mode=file <filename> to generate test image strings. That command can be piped out to a file, and it will contain the full image data, encoded in base64 split amongst 4096 character payloads.

If you have image-magick installed you can also generate simple test images and pipe them directly into the icat kitten:
magick -size 32x32 xc:red PNG:- | kitty +kitten icat --transport-mode=file > red_square_tmp.txt

Then using cat red_square_tmp.txt shows the image. I used a similar command to generate the string that I was using for testing in repro.lua, though it needed to be cleaned up, since the kitty command adds some extra junk for centering and newlines, plus the escape characters need to be fixed.

As for using the unicode placeholder, I'm not 100% certain how exactly it works, when I tried, it seemed like calling icat waited for the placeholder to be put onto the screen before exiting, and times out after 10 second because the shell_execute call was blocking updating the body of the element.