Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix URL Prefix Support #1655

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Conversation

manuth
Copy link

@manuth manuth commented Mar 12, 2025

Description

Currently, hosting the SyncStorage service under any root URL other than / like, say, /firefox-sync, causes 401 HTTP error codes caused by mismatching Message Authentication Codes (or MACs for short) as pointed out by @ethowitz here.

Changes made in this PR add a new option public_url allowing users to specify the public facing URL to the root of the syncservers services.

This public_url option is used for determining the original request uri and perform the MAC authentication properly.

Things to Note

As explained by @kyz here, the host and port for performing the MAC authentication are taken from the Forwarded or the X-Forwarded-For and X-Forwarded-Scheme etc. headers:

let host_port: Vec<_> = ci.host().splitn(2, ':').collect();
let host = host_port[0];
let port = if host_port.len() == 2 {
host_port[1].parse().map_err(|_| {
ValidationErrorKind::FromDetails(
"Invalid port (hostname:port) specified".to_owned(),
RequestErrorLocation::Header,
None,
label!("request.validate.hawk.invalid_port"),
)
})?
} else if ci.scheme() == "https" {
443
} else {
80
};
let path = uri.path_and_query().ok_or(HawkErrorKind::MissingPath)?;

It might be a good idea to swap this to perform the authentication based on public_url if specified, instead. However, I did not include this in this PR and I would love to hear what other people think about this.

Testing

  1. Spin up a syncserver which is hosted under a root other than /, for example: http://localhost:8080/firefox-sync:
    services:
      web:
        image: nginxproxy/nginx-proxy
        restart: unless-stopped
        ports:
          - 127.0.0.1:8080:80
        environment:
          DEFAULT_HOST: sync.example.com
        volumes:
          - /var/run/docker.sock:/tmp/docker.sock:ro
      sync-server:
        build:
          context: .
          dockerfile_inline: |
            FROM rust AS build
            ARG SYNC_STORAGE_VERSION=0.18.2
            RUN git clone https://github.com/mozilla-services/syncstorage-rs -b $${SYNC_STORAGE_VERSION} /app
            WORKDIR /app
    
            RUN \
              apt-get update \
              && apt-get install -y libpython3-dev \
              && cargo install --path ./syncserver --features mysql --locked \
              && cargo clean \
              && apt-get remove -y libpython3-dev \
              && rm -rf /var/lib/apt/lists \
              && bash -O extglob -c 'rm -rf /usr/local/cargo/!(bin)' \
              && bash -O extglob -c 'rm -rf /usr/local/cargo/bin/!(syncserver)'
    
            FROM python:3.11 AS sync
            COPY --from=build /usr/local/cargo/bin/syncserver /usr/local/bin
            COPY --from=build /app/requirements.txt .
            RUN pip install -r requirements.txt
            CMD [ "/usr/local/bin/syncserver" ]
          target: sync
        restart: unless-stopped
        environment:
          VIRTUAL_PORT: 80
          VIRTUAL_PATH: "/firefox-sync/"
          VIRTUAL_DEST: "/"
          VIRTUAL_HOST: sync.example.com
          RUST_LOG: warn
          SYNC_HUMAN_LOGS: 1
          SYNC_HOST: "0.0.0.0"
          SYNC_PORT: 80
          SYNC_MASTER_SECRET: secret
          SYNC_SYNCSTORAGE__ENABLED: "true"
          SYNC_SYNCSTORAGE__DATABASE_URL: mysql://sync:password@sync-db/SyncStorage
          SYNC_SYNCSTORAGE__ENABLE_QUOTA: 0
          SYNC_TOKENSERVER__ENABLED: "true"
          SYNC_TOKENSERVER__DATABASE_URL: mysql://token:password@token-db/TokenServer
          SYNC_TOKENSERVER__RUN_MIGRATIONS: "true"
          SYNC_TOKENSERVER__FXA_METRICS_HASH_SECRET: secret
          SYNC_TOKENSERVER__FXA_EMAIL_DOMAIN: api.accounts.firefox.com
          SYNC_TOKENSERVER__FXA_OAUTH_SERVER_URL: https://oauth.accounts.firefox.com
          SYNC_TOKENSERVER__FXA_BROWSERID_AUDIENCE: https://token.services.mozilla.com
          SYNC_TOKENSERVER__FXA_BROWSERID_ISSUER: https://api.accounts.firefox.com
          SYNC_TOKENSERVER__FXA_BROWSERID_SERVER_URL: https://verifier.accounts.firefox.com/v2
        expose:
          - 80
      sync-db:
        image: mariadb
        restart: unless-stopped
        environment:
          MARIADB_RANDOM_ROOT_PASSWORD: "yes"
          MARIADB_USER: sync
          MARIADB_PASSWORD: password
          MARIADB_DATABASE: SyncStorage
      token-db:
        build:
          context: .
          dockerfile_inline: |
            FROM rust AS build
            ARG SYNC_STORAGE_VERSION=0.18.2
            RUN git clone https://github.com/mozilla-services/syncstorage-rs -b $${SYNC_STORAGE_VERSION} /app
    
            RUN \
              cargo install diesel_cli --no-default-features --features mysql --locked \
              && bash -O extglob -c 'rm -rf /usr/local/cargo/!(bin)' \
              && bash -O extglob -c 'rm -rf /usr/local/cargo/bin/!(diesel)'
    
            FROM mariadb AS db
            RUN mkdir -p /app/tokenserver-db
            COPY --from=build /app/tokenserver-db/migrations /app/tokenserver-db/migrations
            COPY --from=build /usr/local/cargo/bin/diesel /usr/local/bin
    
            RUN { \
                echo '#!/bin/bash'; \
                echo 'diesel --database-url "mysql://$${MARIADB_USER}:$${MARIADB_PASSWORD}@localhost/$${MARIADB_DATABASE}" migration --migration-dir /app/tokenserver-db/migrations run'; \
                echo 'mariadb -u$$MARIADB_USER -p$$MARIADB_PASSWORD -D $$MARIADB_DATABASE <<EOF'; \
                echo 'INSERT INTO services (service, pattern)'; \
                echo "SELECT 'sync-1.5', '{node}/1.5/{uid}'"; \
                echo "WHERE NOT EXISTS (SELECT 1 FROM services);"; \
                echo ""; \
                echo 'INSERT INTO nodes (\`service\`, node, capacity, available, current_load, downed, backoff)'; \
                echo "SELECT LAST_INSERT_ID(), 'http://localhost:8080/firefox-sync', 1, 1, 0, 0, 0"; \
                echo "WHERE LAST_INSERT_ID() > 0;"; \
                echo "EOF"; \
              } > /docker-entrypoint-initdb.d/tokenserver-db.sh
        restart: unless-stopped
        environment:
          MARIADB_RANDOM_ROOT_PASSWORD: "yes"
          MARIADB_USER: token
          MARIADB_PASSWORD: password
          MARIADB_DATABASE: TokenServer
  2. Try to sync your browser against http://localhost:8080/firefox-sync/1.0/sync/1.5
  3. Take note that any request pointing to http://localhost:8080/firefox-sync/1.5/* fail with a 401 HTTP code

Issue(s)

Closes #1217 and closes #1649.

manuth added 6 commits March 13, 2025 08:01
When running behind a reverse proxy hosting
the service under a webroot other than `/`
causes 401 error codes due to mismatching
Message Authentication Codes (MACs).

Changes made in this commit allow users
hosting the sync server behind a reverse proxy
to specify the `public_url` of their service
in order to correct this behaviour.

In doing so, changes made in this commit fix mozilla-services#1217, mozilla-services#1649
@manuth manuth changed the title Allow Customizing the Public URL Fix URL Prefix Support Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant