Docker Hub Image #37

bytesnz · 2021-01-09T15:42:34Z

Just starting to add remote-storage (RS) to some projects I am working on, so am setting up my own RS server. I didn't see that you don't have a (recent) docker image on docker hub, so created a Dockerfile and a Github Action to build and publish one when a release is created.

The image is very basic - uses the node:lts-alpine docker image and the basic armadietto config (from armadietto -e). It exposes the armadietto http port to be used with a proxy.

Are you interested in including this in the repo? Figure you could create an organisation on docker hub to publish it under.

Need to at least:

figure out specifying the server url for server over a proxy (need to ensure X-Forwarded-Host is set)
complete github action file (or complete later)
agree on Dockerfile local copy vs yarn global / npm i -g
add tests to github action file

raucao · 2021-05-21T08:00:30Z

Anyone reading this has enough Docker experience/knowledge to be able to review this PR? Any and all help would be most appreciated! 🙏

peponi · 2021-05-23T09:41:20Z

I prefer standalone docker files, i.e. git clone within the image build and not before

reduces build steps and errors on the end user side

but this is not needed for this project IMHO
because there is already a finished package on npmjs

I would suggest using this

My Dockerfile

FROM alpine:latest AS appBuild

LABEL maintainer="Peponi <[email protected]>" \
      description="will run the Armadietto NodeJS web service (a RemoteStorageJS backend)"

ARG PROJECT_NAME="armadietto"
ARG PORT="8000"
ARG PKG_MANAGER="yarn"
ARG INSTALL_COMMAND="yarn global add"
ARG CONFIG_PATH_CERTS="/etc/letsencrypt/live/example.com/"
ARG CONFIG_PATH_STORAGE="/usr/share/armadietto"

ENV PROJECT_NAME=$PROJECT_NAME

RUN set -e;\
  apk add  \
    curl \
    git \
    nodejs \
    $PKG_MANAGER; \
  mkdir /opt/armadietto; \
  mkdir -m 0700 /$CONFIG_PATH_STORAGE; \
  mkdir -m 0700 /$CONFIG_PATH_CERTS; \
  $INSTALL_COMMAND $PROJECT_NAME; \
  apk del git $PKG_MANAGER; \
  adduser -u 6582 -HD $PROJECT_NAME;

COPY config.json /etc/armadietto.conf.json

VOLUME $CONFIG_PATH_STORAGE $CONFIG_PATH_CERTS
USER $PROJECT_NAME
WORKDIR ~
EXPOSE $PORT

CMD $PROJECT_NAME -c /etc/armadietto.conf.json

HEALTHCHECK CMD curl --fail http://127.0.0.1:$PORT/ || exit 1

### Install ###
#
# BUILD:
#
# default for amd64 architecture
#
# > docker build -t armadietto:latest .
# > docker build -t --build-arg PKG_MANAGER="npm" --build-arg INSTALL_COMMAND="npm i -g" armadietto:latest .
#
# RUN:
#
# > docker run --rm -p 8000:8000 rarmadietto:latest
#
# INFO: config.json needs to be byside the Dockerfile
#
# {
#   "allow_signup": true,
#   "storage_path": "/usr/share/armadietto",
#   "cache_views": true,
#   "http": {
#     "host": "0.0.0.0",
#     "port": 8000
#   },
#   "https": {
#     "enable": false,
#     "force": false,
#     "port": 4443,
#     "cert": "/etc/letsencrypt/live/example.com/cert.pem",
#     "key": "/etc/letsencrypt/live/example.com/privkey.pem"
#   },
#   "basePath": ""
# }
#
###

e2e test

image size

EPOSITORY                                     TAG                 IMAGE ID            CREATED             SIZE
armadietto                                     peponi              9a84b73851eb        21 minutes ago      52.4MB
armadietto                                     bytesnz             9829e4ad797b        About an hour ago   118MB

security scan with trivy

 ✘  ~/Code/private/DockerFiles/armadietto  docker run --rm  -v /var/run/docker.sock:/var/run/docker.sock -v cache:/root/.cache/ aquasec/trivy armadietto:peponi
2021-05-23T09:21:40.833Z        INFO    Detected OS: alpine
2021-05-23T09:21:40.833Z        INFO    Detecting Alpine vulnerabilities...
2021-05-23T09:21:40.836Z        INFO    Number of PL dependency files: 1
2021-05-23T09:21:40.836Z        INFO    Detecting yarn vulnerabilities...

armadietto:peponi (alpine 3.13.5)
=================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)


usr/local/share/.config/yarn/global/yarn.lock
=============================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

raucao · 2021-05-23T10:33:00Z

@peponi That looks great! 👏

@bytesnz Want to review @peponi's proposed changes to the Dockerfile?

raucao · 2021-05-23T10:44:49Z

created a Dockerfile and a Github Action to build and publish one when a release is created

I just created a remotestorage organization on Docker Hub and then created a repo for armadietto. However, I cannot find where to create custom access tokens, and the permissions tab doesn't let me use the "owners" team to create a new permission entry.

Maybe someone more experienced wants access to the RS team there? The free/community plan allows 2 more accounts to be added...

Edit: considering that we're stuck on GitHub for now, would it also make sense to consider GitHub's own Docker registry for publishing this? Seems to me like Docker Hub is rather crippled on free plans.
Edit 2: Looks a bit complicated to me.

bytesnz · 2021-05-23T12:11:41Z

I prefer standalone docker files, i.e. git clone within the image build and not before

reduces build steps and errors on the end user side

but this is not needed for this project IMHO
because there is already a finished package on npmjs

I would suggest using this

My Dockerfile

FROM alpine:latest AS appBuild

LABEL maintainer="Peponi <[email protected]>" \
      description="will run the Armadietto NodeJS web service (a RemoteStorageJS backend)"

ARG PROJECT_NAME="armadietto"
ARG PORT="8000"
ARG PKG_MANAGER="yarn"
ARG INSTALL_COMMAND="yarn global add"
ARG CONFIG_PATH_CERTS="/etc/letsencrypt/live/example.com/"
ARG CONFIG_PATH_STORAGE="/usr/share/armadietto"

ENV PROJECT_NAME=$PROJECT_NAME

RUN set -e;\
  apk add  \
    curl \
    git \
    nodejs \
    $PKG_MANAGER; \
  mkdir /opt/armadietto; \
  mkdir -m 0700 /$CONFIG_PATH_STORAGE; \
  mkdir -m 0700 /$CONFIG_PATH_CERTS; \
  $INSTALL_COMMAND $PROJECT_NAME; \
  apk del git $PKG_MANAGER; \
  adduser -u 6582 -HD $PROJECT_NAME;

COPY config.json /etc/armadietto.conf.json

VOLUME $CONFIG_PATH_STORAGE $CONFIG_PATH_CERTS
USER $PROJECT_NAME
WORKDIR ~
EXPOSE $PORT

CMD $PROJECT_NAME -c /etc/armadietto.conf.json

HEALTHCHECK CMD curl --fail http://127.0.0.1:$PORT/ || exit 1

### Install ###
#
# BUILD:
#
# default for amd64 architecture
#
# > docker build -t armadietto:latest .
# > docker build -t --build-arg PKG_MANAGER="npm" --build-arg INSTALL_COMMAND="npm i -g" armadietto:latest .
#
# RUN:
#
# > docker run --rm -p 8000:8000 rarmadietto:latest
#
# INFO: config.json needs to be byside the Dockerfile
#
# {
#   "allow_signup": true,
#   "storage_path": "/usr/share/armadietto",
#   "cache_views": true,
#   "http": {
#     "host": "0.0.0.0",
#     "port": 8000
#   },
#   "https": {
#     "enable": false,
#     "force": false,
#     "port": 4443,
#     "cert": "/etc/letsencrypt/live/example.com/cert.pem",
#     "key": "/etc/letsencrypt/live/example.com/privkey.pem"
#   },
#   "basePath": ""
# }
#
###

e2e test

image size

EPOSITORY                                     TAG                 IMAGE ID            CREATED             SIZE
armadietto                                     peponi              9a84b73851eb        21 minutes ago      52.4MB
armadietto                                     bytesnz             9829e4ad797b        About an hour ago   118MB

security scan with trivy

 ✘  ~/Code/private/DockerFiles/armadietto  docker run --rm  -v /var/run/docker.sock:/var/run/docker.sock -v cache:/root/.cache/ aquasec/trivy armadietto:peponi
2021-05-23T09:21:40.833Z        INFO    Detected OS: alpine
2021-05-23T09:21:40.833Z        INFO    Detecting Alpine vulnerabilities...
2021-05-23T09:21:40.836Z        INFO    Number of PL dependency files: 1
2021-05-23T09:21:40.836Z        INFO    Detecting yarn vulnerabilities...

armadietto:peponi (alpine 3.13.5)
=================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)


usr/local/share/.config/yarn/global/yarn.lock
=============================================
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)

Cool. Looks good. Will integrate some of the stuff with the MR. The reason I went for using copy instead of yarn install is for integrating with github actions I - it would need to be published to npm before the docker image is created, but that's doable.

I think it is good to have a docker image as well as it makes it super easy to deploy.

AngeloR · 2021-05-29T15:14:17Z

We may want to utilize GitHub actions to do the final build/push to docker hub as well. I'd be in favour of remaining on docker hub, but if we want we can always push the image to multiple locations.

bytesnz · 2021-05-29T17:37:49Z

We may want to utilize GitHub actions to do the final build/push to docker hub as well. I'd be in favour of remaining on docker hub, but if we want we can always push the image to multiple locations.

👍 The MR has a template Github Action for building docker images, but it will need some tweaking.

raucao · 2021-07-28T09:41:26Z

@bytesnz Did you find some time to integrate the things you wanted by any chance? No pressure, just making sure the PR doesn't go stale...

bytesnz · 2021-07-30T07:33:47Z

@raucao I did start making changes and looking at it. I unfortunately ran out of time before going on a trip. I was going to merge the two dockerfiles and add testing to the action from peponi. I can do this at the start of Sep (long trip without a laptop).

bytesnz · 2021-10-22T10:02:18Z

Sorry about the delay. Have done a bit of merging between the two Dockerfiles:

basing it off alpine with a build container, I got it down to 49MB
went for a local copy based approach thinking it would be good doing things like making images from branches, during dev for eg
went for wget instead of curl for the healthcheck (currently won't work if https is enabled though I think)
added trivy to the docker build job

For the github actions job, would be good to add an e2e test run to it as well, but would require a bit of test before rewriting I think (as each spec is starting up a server). Would also be good to split the job up into build, test, security test, but would need to either push a temporary image somewhere (or use some blugh caching I think).

docker/Dockerfile

docker/README.md

JakubNer · 2022-01-02T08:45:52Z

@bytesnz, was testing this a bit. Pulled your branch and...

docker build -t remotestorage/armadietto:latest -f .\docker\Dockerfile .
docker run -d -p 8000:8000 remotestorage/armadietto:latest

But it exits right away with ': No such file or directory in the logs.

So I went into the image:

docker run -it --rm -p "8000:8000" remotestorage/armadietto:latest /bin/sh

And running the linked /opt/armadietto/bin/armadietto.js conks out in the same manner.

Are we certain we can start the process like that?

I've only started it as a library as per /example/server.js

Would love to get this out and build on top of it. I see there are about 15k downloads of armadietto on docker-hub from various providers, would be nice to get this official one out 👍.

bytesnz · 2022-01-04T07:04:15Z

Hey @JakubNer. Hmm, that's weird. Just tried running through the build process again and no issues for me (commands below). Is the image bytesnz/armadietto:latest working for you (is built off this repo?

git clone https://github.com/bytesnz/armadietto
cd armadietto/
git checkout  add-dockerfile 
docker build -t remotestorage/armadietto:latest -f ./docker/Dockerfile .
docker run -p 8000:8000 --name armadietto remotestorage/armadietto:latest

JakubNer · 2022-01-07T05:22:55Z

@bytesnz, indeed when I pull your image (bytesnz/armadietto:latest) off docker hub, works like a charm 👍.

My own build fails like I mentioned. I did build off of the right place:

PS C:\jj\src\armadietto> git status
On branch add-dockerfile
Your branch is up to date with 'bytesnz/add-dockerfile'.

Just rebuilt it locally, same error.

Weird.

JakubNer · 2022-01-16T17:42:03Z

@bytesnz, I'm puzzled. I checked your docker-hub (https://hub.docker.com/r/bytesnz/armadietto/tags) and the image that runs for me is the image the git actions built and pushed last Octo (https://github.com/bytesnz/armadietto/actions/runs/1385063021) which is the same commit as of this PR and your tip-of-tree in your branch.

So I don't understand why the linked /usr/local/bin/armadietto invocation run inside that image doesn't execute when I re-build locally.

What always works is simply removing the link on line 44 of the Dockerfile and replacing line 52 with:

CMD node /opt/armadietto/bin/armadietto.js -c /etc/armadietto.conf.json

I wonder if the difference is my host. I'm building on Windows in PowerShell. I don't see how that should make a difference though.

raucao · 2022-01-16T18:19:24Z

I thought it might help if someone else tested this to confirm, so I tried the instructions from #37 (comment) on Arch Linux, and it worked without issues.

However, if it works on Windows without the symlink (which I also have no idea how to explain), I guess forgoing it in favor of using the full path in the final command wouldn't hurt anyone.

docker/Dockerfile

bytesnz · 2022-01-16T20:56:38Z

So I don't understand why the linked /usr/local/bin/armadietto invocation run inside that image doesn't execute when I re-build locally.

@JakubNer oooo that is weird. If you shell into your created docker image does the link exist and does it point to where it should?

/opt/armadietto $ ls -l /usr/local/bin/
total 0
lrwxrwxrwx    1 root     root            33 Oct 26 09:51 armadietto -> /opt/armadietto/bin/armadietto.js

I'm also curious to know what happens when you try and create a symbolic link in it

/opt/armadietto $ ln -s /opt/armadietto/bin/armadietto.js /tmp/
/opt/armadietto $ ls -l /tmp/
total 0
lrwxrwxrwx    1 armadiet armadiet        33 Jan 16 20:52 armadietto.js -> /opt/armadietto/bin/armadietto.js

But yes agreed, if the full path works, we should just switch it to that.

JakubNer · 2022-01-19T07:23:13Z

@bytesnz

If you shell into your created docker image does the link exist and does it point to where it should?

Indeed it exists but produces the same ': No such file or directory output:

PS C:\jj\src\overhide-ledger> docker run -it --rm -p "8000:8000" remotestorage/armadietto:latest /bin/sh
/opt/armadietto $ ls -l /usr/local/bin/
total 0
lrwxrwxrwx    1 root     root            33 Jan  2 07:59 armadietto -> /opt/armadietto/bin/armadietto.js
/opt/armadietto $ armadietto
': No such file or directory

Relinking works and lists in /tmp.

raucao · 2022-01-24T16:12:47Z

@bytesnz What's the status of the open tasks here? I see some unchecked ones about GitHub Actions, which I assume is about automatically building new images upon merging or for releases, right?

bytesnz · 2022-01-24T23:06:05Z

In terms of the tasks:

complete github action file (or complete later)

Looking at now (bytesnz/armadietto@add-dockerfile...bytesnz:expand-docker). Looking to:
- run the tests in the action
- allow the workflow to run on all pushes (except for the publish)
The only issue is nicely running the e2e tests. Looking at the specs, they create in instance of armadietto themselves rather than querying an external server instance. Not sure if there is a way to run the tests on an external server, but was thinking could either:
- mount and run the tests in the created docker container, or
- modify the tests so that there is an option to run the tests as they are or against an external server
add tests to github action file

See above
agree on Dockerfile local copy vs yarn global / npm i -g

Silently looking for agreement/protest - currently using local copy for the benefit of being able to run the actions on non-published versions.

raucao · 2022-01-25T00:05:51Z

Cool, thanks.

Not sure I understand the problem with the tests. There's already a working Action for that. Shouldn't they each do their own thing and simply kick off other Actions depending on success? I don't really know much about GitHub Actions, but pipelines composed of different workflows are pretty standard in CI these days.

Edit: what I mean by that is that the process should be something like: "test & lint" -> "publish npm package" -> "build docker image" -> "publish docker image"

As we don't have a good process for automatic versioning, these steps would probably not apply for every merge per se. The npm version would be manual, and pushing a new version, or publishing release notes for it, would probably trigger the npm publish and Docker publishing.

DougReeder · 2023-10-05T03:20:21Z

So far as I can tell, this PR could still be merged as-is. It would be nice to merge in https://github.com/bytesnz/armadietto/pull/7/files first, though.

raucao · 2023-10-05T10:48:31Z

@bytesnz Could you have a look at @DougReeder's PR to your repo?

bytesnz · 2023-10-06T09:41:04Z

@bytesnz Could you have a look at @DougReeder's PR to your repo?

reviewed and merged 👍

DougReeder · 2023-10-09T12:46:26Z

Now we just need to merge master into this, to ensure we won't lose any of the changes to the submitBtn and documentation.

- Fix healthcheck - Switch back to using local files for creation instead of global install - Use build container for installing node modules - Remove docs from Dockerfile

* Rewrite docker action to: * Run on all pushes * Only publish on a tag put * Run security tests * Split flow into multiple jobs * Change CMD to file instead of symlink Co-authored-by: Jakub Ner <[email protected]>

Make aquasec check error when issue found, but allow job to pass

- Use npm instead of yarn - Add a log directory - Update to the latest example config

DougReeder · 2025-03-28T15:35:20Z

Everyone's concerns appear to have been addressed, so I will merge this in a week, if no one objects.

bytesnz force-pushed the add-dockerfile branch from cc0a42c to 61c8f37 Compare January 9, 2021 15:48

bytesnz changed the title ~~Docker Hub Image~~ WIP: Docker Hub Image Jan 9, 2021

bytesnz changed the title ~~WIP: Docker Hub Image~~ Docker Hub Image Jan 9, 2021

bytesnz force-pushed the add-dockerfile branch from 0bbb030 to 2b7fffd Compare October 22, 2021 10:01

galfert reviewed Oct 22, 2021

View reviewed changes

docker/Dockerfile Outdated Show resolved Hide resolved

raucao reviewed Oct 23, 2021

View reviewed changes

docker/Dockerfile Outdated Show resolved Hide resolved

bytesnz force-pushed the add-dockerfile branch from 6354401 to 0ed6413 Compare October 26, 2021 11:24

JakubNer reviewed Jan 2, 2022

View reviewed changes

docker/README.md Show resolved Hide resolved

bencharp mentioned this pull request Jan 8, 2022

Fix YunoHost-Apps/armadietto_ynh#7

Merged

2 tasks

JakubNer reviewed Jan 16, 2022

View reviewed changes

docker/Dockerfile Show resolved Hide resolved

silverbucket self-requested a review December 29, 2023 23:27

bytesnz and others added 19 commits March 23, 2025 21:39

Add basic Dockerfile

9c5c461

Create Github action for publish docker image on release

2022091

Modify Dockerfile with suggested changes

5b22bf2

Clean Dockefile

932ebcb

- Fix healthcheck - Switch back to using local files for creation instead of global install - Use build container for installing node modules - Remove docs from Dockerfile

Add trivy test to docker build action

124044e

Remove unused var and move label to main container

146119f

first pass at Docker README

8aa5d38

update Dockerfile label

982c1e8

Rewrite dockerfile to include tests

b5399e4

* Rewrite docker action to: * Run on all pushes * Only publish on a tag put * Run security tests * Split flow into multiple jobs * Change CMD to file instead of symlink Co-authored-by: Jakub Ner <[email protected]>

Make aquasec check error when issue found

3dcb12f

Make aquasec check error when issue found, but allow job to pass

Remove Codacy badge and add details about building

3b4d998

Add information about using docker image behind proxy

177ae14

Clean and add references to the docker README

9ce6851

Update Dockerfile for changes

c7fe22b

- Use npm instead of yarn - Add a log directory - Update to the latest example config

Ensure bin/armadietto.js has unix line endings

b21aabb

Update location of config file to match main README

10f1530

Make sure job runs on tags

eaeb835

Ensures Docker image uses tested package versions

43a6e07

Rebases on latest master, updates workflow actions, and cleans up docs

a0d703b

DougReeder force-pushed the add-dockerfile branch from 098ddaf to a0d703b Compare March 28, 2025 15:33

DougReeder added 2 commits April 2, 2025 11:54

Fixes mis-named environment variable in Dockerfile

9f00a78

Merge branch 'master' into add-dockerfile

9321b26

DougReeder merged commit 76f3658 into remotestorage:master Apr 17, 2025
3 checks passed

DougReeder mentioned this pull request Apr 19, 2025

Docker.io authentification problem #130

Closed

Docker Hub Image #37

Docker Hub Image #37

Uh oh!

Conversation

bytesnz commented Jan 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

raucao commented May 21, 2021

Uh oh!

peponi commented May 23, 2021

My Dockerfile

e2e test

image size

security scan with trivy

Uh oh!

raucao commented May 23, 2021

Uh oh!

raucao commented May 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bytesnz commented May 23, 2021

My Dockerfile

e2e test

image size

security scan with trivy

Uh oh!

AngeloR commented May 29, 2021

Uh oh!

bytesnz commented May 29, 2021

Uh oh!

raucao commented Jul 28, 2021

Uh oh!

bytesnz commented Jul 30, 2021

Uh oh!

bytesnz commented Oct 22, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JakubNer commented Jan 2, 2022

Uh oh!

bytesnz commented Jan 4, 2022

Uh oh!

JakubNer commented Jan 7, 2022

Uh oh!

JakubNer commented Jan 16, 2022

Uh oh!

raucao commented Jan 16, 2022

Uh oh!

Uh oh!

bytesnz commented Jan 16, 2022

Uh oh!

JakubNer commented Jan 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

raucao commented Jan 24, 2022

Uh oh!

bytesnz commented Jan 24, 2022

Uh oh!

raucao commented Jan 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DougReeder commented Oct 5, 2023

Uh oh!

raucao commented Oct 5, 2023

Uh oh!

bytesnz commented Oct 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DougReeder commented Oct 9, 2023

Uh oh!

DougReeder commented Mar 28, 2025

Uh oh!

Uh oh!

Uh oh!

bytesnz commented Jan 9, 2021 •

edited

Loading

raucao commented May 23, 2021 •

edited

Loading

JakubNer commented Jan 19, 2022 •

edited

Loading

raucao commented Jan 25, 2022 •

edited

Loading

bytesnz commented Oct 6, 2023 •

edited

Loading