allow --shell-escape equivalent

[3continents]

Hi. Thanks for this product. It is truly great. It has allowed me to deploy latex generation to our internal web applications without having to download and manage a tex distribution locally on the app-server with each update.

I have, unfortunately, ran into a specific. In my latex generation, I need to download images to include them in the documents. This is helpful locally but truly required when using your docker container with is separate from the rest of the containers. To achieve this locally, I have to run lualatex using the --shell-escape directive or I get image not found errors.

I am getting this error when posting to your web interface. Would it be possible to add the --shell-escape option in the URL parameters? That would solve everything for me.

[dmke]

Hi @3continents,

I'm glad you find texd useful.

Adding -shell-escape support to texd, is a bit dangerous, because it allows anyone with access to that server to run arbitrary commands. Some danger might be mitigated when running the compile process in a Docker container, but still, I wouldn't deploy this even to an internal-only system.

Do you require the lualatex compiler to fetch the images, or would it also helpful, if texd accepts URLs in addition to file attachments? Something akin to:

POST /render HTTP/1.1
Content-Type: multipart/form-data; boundary=boundary

--boundary
Content-Disposition: form-data; name=input.tex; filename=input.tex
Content-Type: application/octet-stream

[content of input.tex omitted]
--boundary
Content-Disposition: form-data; name=image.tiff; filename=image.tiff
Content-Type: application/x.texd; url=1

https://somewhere.example.com/satimg/export.tiff
--boundary--

This could instruct texd to fetch https://somewhere.example.com/satimg/export.tiff and store it as ./image.tiff in the project directory - before lualatex even starts.

(I'd prefer this approach, because then texd could also try to fetch multiple images in parallel).

[3continents]

I am not really worried about the security issue -- although I understand the concern. texd is running in a sandboxed container within a private network of other containers and only the app server has access to it. The app server makes http call which I control and everything sits behind a nginx reverse proxy and a firewall where only ports 80 and 443 are open. In any case, I can construct the request to download the images through the API if this is a preferred option for you. I appreciate your quick response and your willingness to add this functionality.

…

On Tue, Nov 5, 2024 at 5:08 PM Dominik Menke ***@***.***> wrote: Hi @3continents <https://github.com/3continents>, I'm glad you find texd useful. Adding -shell-escape support to texd, is a bit dangerous, because it allows anyone with access to that server to run arbitrary commands. Some danger might be mitigated when running the compile process in a Docker container, but still, *I* wouldn't deploy this even to an internal-only system. Do you require the lualatex compiler to fetch the images, or would it also helpful, if texd accepts URLs in addition to file attachments? Something akin to: POST /render HTTP/1.1Content-Type: multipart/form-data; boundary=boundary --boundaryContent-Disposition: form-data; name=input.tex; filename=input.texContent-Type: application/octet-stream [content of input.tex omitted]--boundaryContent-Disposition: form-data; name=image.tiff; filename=image.tiffContent-Type: application/x.texd; url=1 https://somewhere.example.com/satimg/export.tiff--boundary-- This could instruct texd to fetch https://somewhere.example.com/satimg/export.tiff and store it as ./image.tiff in the project directory - before lualatex even starts. (I'd prefer this approach, because then texd could also try to fetch multiple images in parallel). — Reply to this email directly, view it on GitHub <#152 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BERXO2G4SYOOSYUEDBZRXBDZ7DUPHAVCNFSM6AAAAABRGSCKWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJXGU4DSNZTGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dmke]

I'm just worried that some users will simply ignore the red flags, enable this feature and place a server instance on the internet (and then complain about the consequences). I guess, the typical TeX user has a bit more understanding of the security implications.

I might prioritize adding a --shell-escape CLI argument to texd over a custom pipeline for fetching and storing external files (this is waaaay easier to implement).

[dmke]

(I accidentally fat-fingered the git command while rebasing. The implementation is almost done, but not quite yet :))

[3continents]

Hi. I hate to be a bother. I was just wondering if you were close to completing this. I build an ad-hoc / temporary solution with my own python based micro-service but it is very clunky. (I didn't get the --shell-escape in the docker containers -- no idea why as it worked locally -- and so I finally had to implement something with a two step rendering of my jinja templates so the microservice could use python and jinja to download the files. It's really ugly and I am about to implement a number of different use cases.

If you are close, then I will wait. If not, I think I will test installing tex-live in the docker container for my application server so that I can simpy bypass the microservice and can call the command directly. (I have wanted to avoid this because I have many instances of application server and tex-live is so quite bloated.

Please let me know.

[dmke]

I'm relatively close, but didn't have the time to finish. There's one thing I wanted to verify before I merge the changes: Does latexmk actually accept a -shell-escape flag?

The man-page does not mention it explicitly, but the source code has a list of flags it passes to the TeX compiler (with -shell-escape being one of them). If latexmk -shell-escape does not work as expected, I would need to come up with something else...

If you're willing to test, you could checkout the feat/shell-escape branch, and build a preview Docker image:

$ git clone https://github.com/digineo/texd -b feat/shell-escape ./texd
$ cd texd
$ docker build -t texd:experimental -f .github/Dockerfile.release .

This would produce a texd:experimental image, which accepts --shell-escape (note the double-dash on the option for texd). You would run it as:

$ docker run --rm --name ... -v ... -p ... texd:experimental --shell-escape ...

[3continents]

I tried to test but I got an error at the docker build stage. This command -- docker build -t texd:experimental -f .github/Dockerfile.release . Gave the following error -- Status: Downloaded newer image for ghcr.io/digineo/texd:base

…

---> c56f8b2a3f9b Step 8/11 : COPY --from=builder /work/texd /bin/ COPY failed: stat work/texd: file does not exist THanks! Monte

On Mon, Nov 25, 2024 at 10:58 AM Dominik Menke ***@***.***> wrote: I'm relatively close, but didn't have the time to finish. There's one thing I wanted to verify before I merge the changes: Does latexmk actually accept a -shell-escape flag? The man-page does not mention it explicitly, but the source code has a list of flags it passes to the TeX compiler (with -shell-escape being one of them). If latexmk -shell-escape does not work as expected, I would need to come up with something else... If you're willing to test, you could checkout the feat/shell-escape branch, and build a preview Docker image: $ git clone https://github.com/digineo/texd -b feat/shell-escape ./texd $ cd texd $ docker build -t texd:experimental -f .github/Dockerfile.release . This would produce a texd:experimental image, which accepts --shell-escape (note the double-dash on the option for texd). You would run it as: $ docker run --rm --name texd texd:experimental --shell-escape <other options...> — Reply to this email directly, view it on GitHub <#152 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BERXO2FSKUUEG63GIML2KXT2CLYF5AVCNFSM6AAAAABRGSCKWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJXGQ4DAOJQGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>