Add docker documentation

[Arcelone]

Hi everyone,
I've found myself needing to create a local Mastodon instance for a Lab (something that's at test/demo level).
First of all, I browsed the documentation and the official repository and I saw that the debate on the need / desire to add documentation for docker.
However, as I haven't found a satisfactory tutorial or alternative on the Internet, I've taken the liberty of making a functional version, as detailed as possible, of what I'd like to find in the official documentation.
I'm sure it's not perfect, but if anyone would like to add or modify it, please don't hesitate.

PS: I didn't know what value to enter for "Weight", please feel free to modify it.

This will (at least partially) resolve these issues :

• resolve Add Docker setup to official documentation #1547
• resolve Documentation on using the provided docker image #770
• resolve Specify and issue a more container-centric approach for deployment (in documentation) #793

[Arcelone]

Can anyone review my PR or at least give me some areas for improvement. 😃

[melroy89]

??? Insult, really? I see the community is benevolent. So why the thumbs-up?

Nevermind. It was to dump developers to look at it. It's not an insult. I agree with the content.

[Arcelone]

Ha ok mb, I did check the meaning, but I couldn't find any positive meaning.

[Arcelone]

@ClearlyClaire @oneiros @Gargron

[Arcelone]

More related issues :

• Documentation should mention and suggest Docker installation rather than installing from sources #1035
• docker-compose undocumented, not working due to redis errors mastodon#21025

Other people have already worked on this :

• Add documentation for running with docker #1090
• Add troubleshooting instructions for Docker database setup #964
• Docker notes #778

And other Gist :
https://gist.github.com/melroy89/6fe7d05bdc0cfd2153b77310abf62990
https://gist.github.com/TrillCyborg/84939cd4013ace9960031b803a0590c4

[Arcelone]

@andypiper @oneiros @renchap

[andypiper]

Wow, I'm sorry that this slipped under my radar. I'll try to take a look within the next 10 days (I'm travelling so I don't want to commit to anything sooner, but I'm now paying attention!)

[Arcelone]

• I updated my last message with some additional information.
• Did a rebase
• After some reading of related issues the documentation was never push to main due to a very opinionated point of view of only one people.

@Lastorder-DC @XanderStrike

[Arcelone]

@andypiper

[mjankowski]

Suggested change

	git clone https://github.com/tootsuite/mastodon.git
	git clone https://github.com/mastodon/mastodon.git

[mjankowski]

This is both in this section, and in the next section - on purpose?

[mjankowski]

@Arcelone curious if you are still interested/available in nudging this forward.

I closed some historical PRs/issues in favor of this one issue #1547 and this current PR.

If you are not interested, or I don't hear from you in a bit, I'll copy/rebase this and re-open elsewhere ... trying some bare minimum of "how to docker" in place.

[Arcelone]

First of all, a few things:

1. Yes, I am still interested in contributing. However, it has been almost two years since I opened this PR, rebased several times, and no one has ever taken care of it. I am willing to update it, but only if someone with write access to the repository takes care of it.
2. As previously mentioned, there is one person on the project who does not want any documentation on Docker. If this PR has no chance of success because of this person, I will not spend any more time on it.
3. The code in this PR remains my creation. If you reuse the code somewhere else, please add me as a co-author of the PR.

[ShadowJonathan]

Which person doesn't want docker documentation?

[mjankowski]

1. Yes, I am still interested in contributing.

Excellent, will leave this open and try to help with the rest of your points.

However, it has been almost two years since I opened this PR, rebased several times, and no one has ever taken care of it.

Yes, I'm sorry for this. I don't fully understand why the repo is not maintained, and I can't get anyone to answer that question for me either. It does appear we've got some recent momentum on closing/moving stuff so I'm cautiously optimistic.

I am willing to update it, but only if someone with write access to the repository takes care of it.

I will attempt to ping such persons, and report back.

2. As previously mentioned, there is one person on the project who does not want any documentation on Docker. If this PR has no chance of success because of this person, I will not spend any more time on it.

That seems like a weirdly strong view for something which has been present and maintained in the project for years, and for which thousands of people are using. Do you have any sense of the nature of the objections?

My understanding is that Docker is not the "officially recommended" way, but it is ... a way?! Maybe we can add an "This is not our official or best way to deploy, but here are some steps to Docker" type of warning to the top of the page? Is that the nature of the concern, or were there other objections?

If you reuse the code somewhere else, please add me as a co-author of the PR.

Yes, will absolutely leave you as the committer, and that will be preserved (to extent allow by GH repo settings).

[Arcelone]

Which person doesn't want docker documentation?

@ShadowJonathan @mjankowski

This person from the Mastodon organisation, Nightpool, is systematically against all Docker/container workflows (Compose or K8S).
He dismisses every issue opened as unplanned. He uses the argument that containers have brought them more bugs to troubleshoot and added more complexity than the classic systemD workflow.
However, the fact is that it is complex because there are no docs — it's a chicken and egg problem.

Here are his comments on issues 770, 1035 and 793:

#770 (comment)
#1035 (comment)
#793 (comment)

[andypiper]

I'll chip in here as someone on the core team. I don't believe we're against having this documentation. I recognise that I've previously said I'd look into this PR, and didn't get it merged; I'd need someone else to help with technical validation and checking of whatever ends up in the formal documentation. I'd also want someone on the core engineering team to look over the docs as well, to make sure that we're happy with the recommendations. I know this has been open for an overdue amount of time, so I'm glad to see interest from others in helping to get it "across the line" - thank you!

[Arcelone]

I'll update this PR with up to date informations, (repo name, install method, script etc..).
I'll also do my best to provide a compose compliant with the ANSSI and CIS v1.8 docker security standard.

As stated in my initial PR I just need informations about the header of the file

PS: I didn't know what value to enter for "Weight", please feel free to modify it.

But I do not have the necessary skills at the moment to provide a correct production ready K8S manifest.

[mjankowski]

Sounds good. Thanks for the updates.

I would definitely focus on just hitting the bare minimum to correctly use the docker setup from the main repo. We can hit edge cases, nuance, optional features, etc in future updates.

Also FYI the "weight" thing controls the order of appearance in the sidebar links for the section. Feel free to use whatever you think makes sense. I think if it's left blank it will fall to the bottom.

[mjankowski]

I see a commit push from last week ... is this ready for re-review, or was that just a rebase push and you're still going to edit further first?

[andypiper]

To set expectations - we still would want to do lot of review and test work here; this is not likely to happen very soon.

[Arcelone]

I see a commit push from last week ... is this ready for re-review, or was that just a rebase push and you're still going to edit further first?

No I just did a rebase, I didn't had time to work in this. I was on something else.
Sorry about the notifications of the rebase.

I'll add a message to tell you when it will be ready for review.

[oneiros]

Thanks for taking the time to look into this and all the patience!

I am no docker expert, so I just have two high-level observations / remarks:

1. I think it would be nice if this followed a similar format / flow as the "installing from source" guide (https://docs.joinmastodon.org/admin/install/). I think in its current state it mostly does already, but maybe a section for prerequisites (docker and compose versions for example) could be added.
2. I wonder why files we provide from the repo need to be changed. If this is absolutely necessary for them to work, I think this should be fixed in our main repo and not burden every single administrator who wants to use them.

[andypiper]

There is also a discussion about the current status of the docker content in the main project, which is very related to the ability to merge this PR - it would be best to come to a point that we feel comfortable with what the supported code / devops assets do, and have that documented, rather than making modifications in different places. That, overall, is largely what the delay has been on handling this particular doc update.

[Arcelone]

Just to let you know that it's a work in progress.
I'm working on it.

In addition to the documentation, I have some recommendations regarding the Docker-Compose.yml file in the main repository. For example, why use a Postgres V14 as the database container image? The last update to this line was 4 years ago.

The main documentation (for systemd setup) mention PGSQL v18.

EDIT : I haven't seen the link mentioned by andypiper.

[Arcelone]

I have rewrite almost all the doc, I'm at the nginx part.

I wanna wish you all a Merry Christmas. 🎅🏻 🎄 🎁

I think I'll also open PRs on the main repository, for example for the NGINX configuration compatible with Docker. This will avoid all sed commands.

@oneiros the nginx file needs to be change if you use a nginx container, if you use a local host nginx the file is almost good. It all depends of how you want setup your environment.

[Arcelone]

However, I need some help, because despite searching the source code, I haven't been able to find where the user and database are created during the first login for configuration.

I haven't found the code associated with the db:setup task.

I'm not familiar with Ruby, let alone Rails. Maybe it's included in the ActiveRecord from Rails idk.

[Arcelone]

cc @mjankowski @melroy89 @oneiros

[mjankowski]

Hi, I see the ping, but havent been following the edits/updates closely ... is this a ping because this is ready for review, or a ping because you need more info here?

If the latter, much of the initial data population is done by either https://github.com/mastodon/mastodon/tree/main/db/seeds / https://github.com/mastodon/mastodon/blob/main/lib/tasks/mastodon.rake

The db:setup task is built into rails - https://guides.rubyonrails.org/active_record_migrations.html#setting-up-the-database

[Arcelone]

I think it's ready for review. The following were taken into consideration:

• recent changes, such as refactoring the Nginx configuration and automatically validating admin accounts.
• added prerequisites
• I changed the nginx setup to follow the 'official' method (nginx on the host system and Mastodon in containers).

My question about the database setup was to understand the exact workflow for the initial setup.
The pgSQL config in the Compose file uses POSTGRES_HOST_AUTH_METHOD=trust, but I couldn't determine if a user and a password were created afterwards when using the wizard.

[mjankowski]

Suggested change

	And and the following environment variable to the `.env.production`:
	Add the following environment variable to the `.env.production`:

[mjankowski]

Suggested change

	The mastodon web and streaming service are listening on 127.0.0.1 so you need setup nginx to redirect the traffic from port 80 and 443 of you machine to the web and streaming container which are listening on 127.0.0.1.
	The mastodon web and streaming service are listening on 127.0.0.1 so you need to setup nginx to redirect the traffic from port 80 and 443 of your machine to the web and streaming container which are listening on 127.0.0.1.

[mjankowski]

I left a few minor wording comments, but I think overall it looks good. Some nice followups might be:

• Do once over on style/language/format/etc to align with rest of the docs (no huge stuff on this front, I just dont want to obsess over it for initial creation of this page)
• I'm not sure exactly how this would even work, but it would be delightful if we could somehow have these docs either literally be runnable code, or produce runnable code, and we could have the main repo and/or docs repo CI tasks validate that it continues to work as the main repo changes, docker changes, underlying OS assumptions changes, etc. Way out of scope for first pass, just a thought.
• To the extent things are repeated in this page and the other how to install pages, either have them link to each other, and/or find some way to keep in sync

All that said, I think given a standard of "we dont have any docker docs at all now", this does IMO get above that low bar of just getting something. Thanks for updating!

I know the core team is busy, but hopefully someone can take a look for you here.