Improve the Release workflow

[litetex]

Disclaimer: Long post - Please bring some time when reading this

Background:
I did the last release (for the first time) and it was well... let's say rough and a bit nerve wracking... in total I spent ~5-7 hours doing that release, which in comparison to other project is extremely long. I mainly attribute the long automation to zero automation and different SPOF.
I also noticed multiple problems in the release process that's why I'm creating this issue here.

So let's get started.

Documentation

The documentation of the release workflow is outdated and not very intuitive. I had to create a checklist to not lose control of the release.

Notable things that are problematic:

• The site describing releases is way too big and confusing (at least to me) - if I print it out I get 9-10 DIN A4 pages
• The workflow described there is completely outdated. There are e.g. no more options to create releaseCandidates anymore (simply use the GH action build from the PR now) and it doesn't even mention NewPipeExtractor in the "regular release" section.
• The documentation says that you have to create a PR to the F-droid repo - That's not required anymore.
• There is no statement how/when F-Droid builds the app. However we found out that this seems to happen when you create a new release tag: https://gitlab.com/fdroid/fdroiddata/-/blob/f1b77a43527d968b5cf39b73ad566679e99e5247/metadata/org.schabi.newpipe.yml#L786 - I presumed this happens when you push to the master branch
• ...

Weblate

This is a very big problem and it will likely result in the future in a minor or major catastrophe.

Problems:

• We are currently unable to sync weblate to GitHub because the dev branch (where weblate is synced to) is protected and requires a PR (#DoNotPressThePushButtonOnWeblateOrEverythingExplodes)
• Everything on weblate is operating on a single branch. That branch has the state of the develop. Problem: When you create a new release and want the changelog file to be translated - so that you can also include that in the release - you suddenly end up with changes from the develop inside the release branch.
  So either you now freeze the dev branch to the state of the release branch - which is completely unpractical because we can't merge things into the develop for 1-2 weeks - or you cheerypick the changes and hope that there aren't new string or something which break the release.
  Luckily the current release was unaffected.

Single point of failures

There were multiple situations where I was waiting for one person to either

• approve the release PR
• give me the rights for some process because I didn't have it (okay for the first time)
• or to sign the release APK

These are SPOF!
I don't know what happens when one of the above persons ever runs into the Bus factor, get's hacked and loses all it's data or something else but I think it's very likely that the complete project may be unable to do further releases.

These things/capabilities should be distributed and at best: get automated.

Proposed solutions

General

• The targeted state for doing a release is clicking a button and everything is done automatically (scripted).
• Adoption of the 4/5 eyes principle to everything that can't be done automated: No one works alone or has sole control about something. There has always to be at least someone else (active) who can do these things + maybe additionally a programmatic solution - the fifth eye - which has a look at that.

Documentation

The targeted state - where everything is automated - doesn't require documentation or only very little (a simple checklist should be enough).
If everything is scripted comments inside the scripts should be enough.

Changelog management

In my opinion it would be great if we had a Changelog.md file instead of GH draft's.
The main advantages I see:

• The file is visible to everyone (the draft is only visible to team members)
• The file is committed inside of Git
• The file can be automatically maintained by a program

Weblate

I think we have to do multiple fixes here:

• Split the weblate project into 2 parts
  • One for the app's string values
  • One for the changelogs
• Both parts get there corresponding branch inside the main NewPipe repo (e.g. weblate/appstrings and weblate/changelogs
• The appstrings branch is kept in sync with the dev branch - likely automatically by a script/GH actions
• The changelogs branch is kept in sync with the release branch - this may be done also automatically

Maybe we should also consider dropping the translated changelogs. The last release had 4 translated changelogs within 1-2 weeks.

Beta/Preview releases

A bit off topic but also notable (didn't include it in the automated release workflow below):
Publishing automated beta/preview releases to e.g. F-Droid / NewPipe's F-Droid repo.

Advantages:

• Users can faster get releases which might fix problems (YT dislikes/likes not working? Install the beta version of NewPipe until a new stable release is out)
• Problems can be detected before doing the "main" release, e.g. when F-Droid's signing doesn't work

Proposed automated release workflow

This is a proposal how most of the release can be done automated (with e.g. GitHub actions):

Beta/Nightly could potentially be a killer feature on this repo, and especially for future UI changes (I expect newpipe to take the form of Pure Tuber).

[XiangRongLin]

https://github.com/XiangRongLin/NewPipe-nightly could be transfered here if requested. It builds the release version of the app signed by me and releases it to github automatically.

[nicoursi]

I was wondering if it could be feasible to have a notification every time a new nighty gets released, just like with the official app release

[Poolitzer]

Maybe we can change the Updates menu to change to what type of NewPipe you want updates for (though currently debug apks ofc dont override the release ones), so you can set it to nightly/beta/standard and get a notification for new versions in the specified release cycle

[nicoursi]

I was wondering if it could be feasible to have a notification every time a new nighty gets released, just like with the official app release

I found a way by using an RSS reader and add: https://github.com/XiangRongLin/NewPipe-nightly.atom as a feed. This works like a charm! I've been using nightilies as my daily driver now.

[nicoursi]

@Stypox As nightlies are so cool, I was thinking if it was the case to pin an issue with the link to the nighty repository and perhaps to the RSS Feed to? ( https://github.com/XiangRongLin/NewPipe-nightly.atom)

[Stypox]

I would support even a link in the README, under the Installation and updates section. Please open a PR if you want ;-)

[XiangRongLin]

If wanted, I could transfer the project to the TeamNewPipe organization. It would make more sense, than having it under my name.
There is basically no maintenance required, seeing that I didn't do a single commit in nearly a year.
Only setup needed, would be creating the secrets used for signing the apk

[opusforlife2]

If it could be distributed through the repo, that would be great. A release version and a nightly version.

[Stypox]

Yes, it would be great if this was moved to the NewPipe org. I think we should generate some new signing keys just for the nightlies though, as we don't want bad code to be accidentally signed by our official keys, or users accidentally overwriting their release version with a nightly one because we do some changes to the gradle package name generation for nightly builds and introduce some bugs.

[XiangRongLin]

@Stypox One needs rights to create a repo in order to transfer them, which I'm missing

[Stypox]

@TobiGr or @TheAssassin could you take care of this, since you have permission?

[opusforlife2]

@TobiGr ^