Proxy calls don't seem to work on a Docker Compose setup running Jam and Joinmarket separately

[pedromtcosta]

I am trying (actually managed, but I will get there) to run Joinmarket and Jam separately under a Docker Compose setup.

It seems the instructions for connecting to a local joinmarket instance are outdated, I think none of the env vars mentioned in the final docker run ... command for running Jam are actually used in code, at least I couldn't find them in the repo:

docker run --rm  -it \
        --add-host=host.docker.internal:host-gateway \
        --env JAM_JMWALLETD_HOST="host.docker.internal" \
        --env JAM_JMWALLETD_API_PORT="28183" \
        --env JAM_JMWALLETD_WEBSOCKET_PORT="28283" \
        --env JAM_JMOBWATCH_PORT="62601" \
        --publish "3000:80" \
        ghcr.io/joinmarket-webui/jam-ui-only:${jam_version}

I found this setupProxy.js file which should setup the proxy calls to the Joinmarket services, but I think it is problematic because 1) it assumes the services are running on localhost, so I have to expose them on the host machine even if all the services are running under Docker and 2) I believe it only works on dev mode (running with npm run dev).

I think theoretically I should be able to make it work by treating Docker as a remote machine and setting up the SSH tunnel as mentioned in here, but I only thought about this after getting a working setup using nginx, and it also seems to not be needed when running under Docker.

The way I managed to get it to work was by:

1. Build Jam with a Dockerfile and run it with nginx

Dockerfile:

FROM node:20-alpine AS builder

WORKDIR /app
COPY ./jam .

RUN npm install && npm run build

FROM nginx:alpine

RUN rm -rf /usr/share/nginx/html/*

COPY --from=builder /app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf

EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

nginx.conf:

worker_processes 1;

events {
  worker_connections 1024;
}

http {
  include       mime.types;
  default_type  application/octet-stream;
  sendfile        on;
  keepalive_timeout  65;

  server {
    listen 80;

    # Serve static frontend
    root /usr/share/nginx/html;
    index index.html;

    location / {
      try_files $uri $uri/ /index.html;
    }

    # Proxy API calls to jmwalletd
    location /api/ {
      proxy_pass https://jmwalletd:28183/api/;
      proxy_set_header Authorization $http_x_jm_authorization;
    }

    # Proxy WebSocket connections to jmwalletd
    location /jmws {
      proxy_pass https://jmwalletd:28283;
      proxy_http_version 1.1;
      proxy_set_header Authorization $http_x_jm_authorization;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      proxy_ssl_verify off;
    }

    # Proxy Orderbook calls to obwatch
    location /obwatch/ {
      proxy_pass http://obwatch:62601/;
      proxy_set_header Authorization $http_x_jm_authorization;
    }
  }
}

2. Build Joinmarket using its own Dockerfile and run jmwalletd and obwatch under Docker Compose

jmwalletd:
  image: joinmarket
  container_name: jmwalletd
  command: >
    bash -c "python3 scripts/jmwalletd.py"
  volumes:
    - /Some_Place_On_Host:/root/.joinmarket
  expose:
    - "28183"
    - "28283"
  networks:
    - jam_net

obwatch:
  image: joinmarket
  container_name: obwatch
  command: >
    bash -c "python3 scripts/obwatch/ob-watcher.py --host=0.0.0.0"
  volumes:
    - /Some_Place_On_Host:/root/.joinmarket
  expose:
    - "62601"
  networks:
    - jam_net


jam:
  image: jam
  container_name: jam
  restart: always
  ports:
    - "31313:80"
  volumes:
    - jam_data:/jam
  networks:
    - jam_net

This way I managed to get it working running everything under Docker exposing only Jam to the host machine, but I am wondering if there was an easier way to get this working. If not, I believe it would be a good idea to have this example on the docs for the people who want to build both Joinmarket and Jam from source and run them under Docker.

[theborakompanioni]

Hey @pedromtcosta!

Thanks for posting the issue. You must have spent quite some time on this, so I want to clarify and hope to make things more clear.

The first thing that is imporant: How are you running Jam?
The jam-ui-only docker image is exactly what you want, if you want to connect to a remote/existing JM instance.
However, it expects obwatch and jmwalletd running on the same host - the env vars (JAM_JMWALLETD_HOST, JAM_JMWALLETD_API_PORT, etc.) are actually used: https://github.com/joinmarket-webui/jam-docker/blob/master/ui-only/jam-entrypoint.sh#L4-L20

They are subsequently used in the nginx config: https://github.com/joinmarket-webui/jam-docker/blob/master/ui-only/nginx/templates/default.conf.template#L3

So, having said this, if you are using the ui-only image in your docker compose setup, you have to link the containers and set the env vars accordingly, something like the follow should work:

  jmwalletd:
    image: joinmarket
    container_name: jmwalletd
    command: >
      bash -c "python3 scripts/jmwalletd.py"
    volumes:
      - /Some_Place_On_Host:/root/.joinmarket
    expose:
      - "28183"
      - "28283"

  jam:
    image: ghcr.io/joinmarket-webui/jam-ui-only:v0.4.0-clientserver-v0.9.11
    container_name: jam
    environment:
      JAM_JMWALLETD_HOST: "jmwalletd"
      JAM_JMWALLETD_API_PORT: 28183
      JAM_JMWALLETD_WEBSOCKET_PORT: 28283
      JAM_JMOBWATCH_PORT: 62601 
    ports:
      - "8080:80"
    depends_on:
      jmwalletd:
        condition: service_started

If you want, you can create an ticket on the jam-docker repo, in order for the ui-only version to support running obwatch on a different host than the one specified in JAM_JMWALLETD_HOST (this line is the culprit).

I found this setupProxy.js file which should setup the proxy calls to the Joinmarket services

You are right, the setupProxy is only meant for development purposes. Jam should be delivered as a static web application from a webserver, without needing nodejs, npm and so on.

I think this will not answer all of your questions, but feel free to ask if you need more information.
Thanks for using Jam and for the detailed report 🙏

[pedromtcosta]

Hey @theborakompanioni , thank you for the response!

I am building Jam from source using the Dockerfile I mentioned when opening the issue and then running it with Docker Compose. From what I'm seeing, I imagine the mentioned environment only works when using your Docker image. I just saw on the jam-docker repo that you only use these vars there to set the proxies URLs on the entrypoint of the Dockerimage, and then I saw they are used on the Nginx config to configure the actual proxies. So I think basically what I did in my setup was to sorta replicate how you build the Docker image for jam-ui-only.

But I think in my setup even using your Docker image would not work because I am running jmwalletd and obwatch on different containers, so the image would need a different env var for the obwatch host.

[theborakompanioni]

But I think in my setup even using your Docker image would not work because I am running jmwalletd and obwatch on different containers, so the image would need a different env var for the obwatch host.

Yeah, let's make it work then!

Tracked here: joinmarket-webui/jam-docker#161

[theborakompanioni]

But I think in my setup even using your Docker image would not work because I am running jmwalletd and obwatch on different containers, so the image would need a different env var for the obwatch host.

Hey @pedromtcosta! Do you mind reviewing joinmarket-webui/jam-docker#162 and verify it is working with your setup? That'd be awesome 🙏

[pedromtcosta]

Hey @theborakompanioni sorry for the delay on this, I will try to get back to you on the weekend. Thank you for implementing this 🙏

[theborakompanioni]

Hey @theborakompanioni sorry for the delay on this, I will try to get back to you on the weekend. Thank you for implementing this 🙏

Hey, thanks for the reply. Please.. not urgency involved! We merge once we hear back from you. Anytime, brother 🙏

[theborakompanioni]

Hey @pedromtcosta - have you had the chance to test joinmarket-webui/jam-docker#162?