How to get the Docker Compose setup working with Docker host networking?

[Hammit]

My current setup follows the guidelines for compose and I also utilize Duck DNS in my Caddyfile. It works great except for the fail2ban banning. Note: fail2ban is installed on the host, not running in a container.

The fail2ban only sees the IP of the router. Note: I have a port forward on my router to my local Vaultwarden compose. I have added the header_up X-Real-IP {remote_host} but in the logs I see only my router IP when someone fails to login.

Originally I thought this might be a SNAT (Source NAT) but I'm not an iptables whiz, so I'd like to try Docker host networking to see if it is a Docker networking issue.

I have added network_mode: host to both of my caddy and vaultwarden services but now my Vaultwarden can't be reached from the public IP of my router with my port forward.

Here is my a relevant part of .env

ROCKET_PORT=8000

Here is my docker-compose.yml

services:
  vaultwarden:
    #image: vaultwarden/server:latest
    image: vaultwarden/server:1.34.3
    container_name: vaultwarden
    restart: always
    env_file:
      - ./.env
    environment:
      #DOMAIN: "https://vaultwarden.example.com"  # Your domain; vaultwarden needs to know it's https to work properly with attachments
      DOMAIN: "<REDACTED>"  # Your domain; vaultwarden needs to know it's https to work properly with attachments
    network_mode: host
    volumes:
      - ./vw-data:/data

  caddy:
    image: caddy:2
    container_name: caddy
    restart: always
#    ports:
#      - 80:8000  # Needed for the ACME HTTP-01 challenge.
#      - 443:443
#      - 443:443/udp # Needed for HTTP/3.
    network_mode: host
    volumes:
      - ./caddy:/usr/bin/caddy
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - ./caddy-config:/config
      - ./caddy-data:/data
    environment:
      #DOMAIN: "https://vaultwarden.example.com"  # Your domain.
      DOMAIN: "<REDACTED"  # Your domain; vaultwarden needs to know it's https to work properly with attachments
      EMAIL: "<REDACTED>"                 # The email address to use for ACME registration.
      DUCKDNS_TOKEN: <REDACTED>
      LOG_FILE: "/data/access.log"

Here is my Caddyfile

{$DOMAIN}:443 {
  log {
    level INFO
    output file {$LOG_FILE} {
      roll_size 10MB
      roll_keep 10
    }
  }

  # Use the ACME HTTP-01 challenge to get a cert for the configured domain.
  tls {
    dns duckdns {$DUCKDNS_TOKEN}
  }

  # This setting may have compatibility issues with some browsers
  # (e.g., attachment downloading on Firefox). Try disabling this
  # if you encounter issues.
  encode zstd gzip

  # Proxy everything Rocket
  reverse_proxy 127.0.0.1:8000 {
       # Send the true remote IP to Rocket, so that vaultwarden can put this in the
       # log, so that fail2ban can ban the correct IP.
       header_up X-Real-IP {remote_host}
  }
}

When I connect to the router port forward, I get no logs from caddy or vaultwarden and connection refused in browser