Build question

[IzzySoft]

At IzzyOnDroid (where your app is listed since 3/2019 – be welcome to pick a badge to link there e.g. from your Readme) we support Reproducible Builds (see: Reproducible Builds, special client support and more at IzzyOnDroid). Trying for yours, I was able to successfully generate the APK using ./gradlew assembleRelease, but the resulting APKs were not identical. If almost looks as if I missed some build steps, but I could not find related instructions:

  -rw-r--r--  0.0 unx  4187284 b-  1521211 defN 1981-01-01 01:01:02 91106f9d classes.dex
- -rw-r--r--  0.0 unx      379 b-      232 defN 1981-01-01 01:01:02 84826e51 assets/any.css
- -rw-r--r--  0.0 unx      248 b-      248 stor 1981-01-01 01:01:02 67374c4e assets/images/back.png
- -rw-r--r--  0.0 unx      288 b-      288 stor 1981-01-01 01:01:02 077c106a assets/images/close.png
- -rw-r--r--  0.0 unx      280 b-      280 stor 1981-01-01 01:01:02 5e694fab assets/images/done.png
- -rw-r--r--  0.0 unx      301 b-      301 stor 1981-01-01 01:01:02 de764b8c assets/images/folders.png
- -rw-r--r--  0.0 unx      366 b-      366 stor 1981-01-01 01:01:02 78194494 assets/images/new.png
- -rw-r--r--  0.0 unx      281 b-      281 stor 1981-01-01 01:01:02 7cf51c57 assets/images/rm.png
- -rw-r--r--  0.0 unx      476 b-      476 stor 1981-01-01 01:01:02 44eeb029 assets/images/search.png
- -rw-r--r--  0.0 unx      217 b-      217 stor 1981-01-01 01:01:02 bebb049b assets/images/settings.png
- -rw-r--r--  0.0 unx     8845 b-     8845 stor 1981-01-01 01:01:02 10a5724b assets/images/sync.png
- -rw-r--r--  0.0 unx    47273 b-    10648 defN 1981-01-01 01:01:02 a4a36283 assets/index.html
- -rw-r--r--  0.0 unx     1910 b-      738 defN 1981-01-01 01:01:02 597e544f assets/ios.css
- -rw-r--r--  0.0 unx    23992 b-     8102 defN 1981-01-01 01:01:02 39d9ee43 assets/langs.js
- -rw-r--r--  0.0 unx     8519 b-     2187 defN 1981-01-01 01:01:02 111da398 assets/style.css
+ -rw-r--r--  0.0 unx      108 b-       98 defN 1981-01-01 01:01:02 89429104 assets/ReadMe.txt
  -rw----     2.4 fat        6 b-        6 stor 1981-01-01 01:01:02 5cd9fe4c META-INF/android.arch.core_runtime.version

So while your APK contains a bunch of files missing in my build from the tagged commit, mine has one file missing in yours. Confuses me a little. Is there some copy task missing on my end (the missing files seem to be in the Multi/ tree)? Checking with the Readme.txt I figured that would be the case – but copying all its contents, as the readme says, I end up with too many files in my APK. I have to copy very selectively:

build:
  - cd Source/Android
  - rm app/src/main/assets/ReadMe.txt
  - cp -a ../Multi/images ../Multi/*.css ../Multi/index.html ../Multi/langs.js app/src/main/assets/
  - rm app/src/main/assets/images/flashlight.png app/src/main/assets/images/qr.png app/src/main/assets/images/share.png app/src/main/assets/windows.css
  - chmod +x gradlew
  - ./gradlew assembleRelease

And even then, the index.html differs (so the APK was probably not built from the tagged commit). I had to go back to fa6712a (at which point the windows.css did not yet exist, so the rm failed and needed to be adjusted again) – then the build became reproducible.

I somehow doubt you'd do that copying manually; maybe you could include the copy script (or have a gradle task taking care for that automatically)? Ideally not just as a Windows batch file (you seem to build on Windows), but (also) as shell script or something OS agnostic 😉 And please, have the APK of future releases match the tagged commit. As your build seems not to provide vcsInfo, it otherwise means a lot of manual work to figure out which one it was…

We'd appreciate if you could help keeping your build reproducible. We've prepared some hints on reproducible builds for that.

Looking forward to your reply!

[r57zone]

Hi, I am doing multi-development here https://github.com/r57zone/EasyNotes/tree/master/Source/Multi

Android shell is separate.

You need to place files here so that the application is assembled with the web application inside https://github.com/r57zone/EasyNotes/tree/master/Source/Android/app/src/main/assets <- https://github.com/r57zone/EasyNotes/tree/master/Source/Multi

[IzzySoft]

You need to place files here

I did that, as pointed out above. But a closer definition of "the files" is needed to get it RB. Just copying over all of them would be easy – but then all of them end up in the APK. While not all of them are inside your APK, so we end up with a different one. For reproducible builds, it's crucial the apps are byte-identical.

So if your build process would make sure to pick the very same files for you and for us, even if there are additional files in the directory, we can just copy over all of them. Else, every time your collection changes, RB will fail here and we'll need to figure out manually what to adjust.

[IzzySoft]

Sad news, @r57zone – v1.3 failed RB again:

  -rw-r--r--  0.0 unx     1910 b-      738 defN 1981-01-01 01:01:02 597e544f assets/ios.css
- -rw-r--r--  0.0 unx    29172 b-     9660 defN 1981-01-01 01:01:02 4e0815e9 assets/langs.js
+ -rw-r--r--  0.0 unx    29173 b-     9661 defN 1981-01-01 01:01:02 2708b573 assets/langs.js
  -rw-r--r--  0.0 unx     8349 b-     2164 defN 1981-01-01 01:01:02 d88efb09 assets/style.css
+ -rw-r--r--  0.0 unx      781 b-      297 defN 1981-01-01 01:01:02 3970c615 assets/windows.css
  -rw----     2.4 fat        6 b-        6 stor 1981-01-01 01:01:02 5cd9fe4c META-INF/android.arch.core_runtime.version

Looking at the differences in langs.js it seems you didn't built from a clean tree at the tagged commit:

 let lang = navigator.language.toLowerCase();
-//navigator.language.toLowerCase() == 'ru-ru' || navigator.language == 'ru' // В iOS языки прописными буквами, вторая проверка нужна для Edg
+//navigator.language.toLowerCase() == 'ru-ru' || navigator.language == 'ru' // В iOS языки прописными буквами, вторая проверка нужна для Edge
 // Русский язык

As you do not include vcsInfo with your build (you most likely use an older AGP version which doesn't yet add that) it's hard to say what exactly happened – but I guess you've fixed those spelling errors after the APK was built, and did not build again as it were "just comments", but then committed the changes nevertheless. That breaks RB, as we of course build from the committed code.

Also, that assets/windows.css is not present in your build, so it was added later. I assume it won't be needed in this release – but then again, it was there for ours. Unfortunately, that commit contained other unrelated changes then as well, so we cannot simply go back one commit as that would just cause other differences to show up.

I've already linked you to our hints on reproducible builds. Let me name the "first basic rule" from there:

Always build from a clean tree at the commit the tag points to.

That means: no local changes, no artifacts left from previous builds.

---

TL;DR:

Last time, we had to remove some files to make your app RB. This time, those files haven't been there, but we had to remove others and patch one:

- rm app/src/main/assets/ReadMe.txt
- cp -a ../Multi/images ../Multi/*.css ../Multi/index.html ../Multi/langs.js app/src/main/assets/
- rm app/src/main/assets/windows.css
- sed -r 's/нужна для Edge/нужна для Edg/' -i app/src/main/assets/langs.js

Such checks cost us time and resources (multiple build runs, analysis, coming up with fixes, etc). It would tremendously help us if you did "clean builds". We've got almost 650 apps to cover currently, if we had to fix each of them we had to give up…

Thanks in advance! And yes, while I write this last line, a successful RB just completed. Due to those changes.

[r57zone]

Hi, I was wondering if there are any download or usage statistics for the app?

If the app isn’t being used much, perhaps it could be considered for removal to help reduce maintenance overhead.

Similar situations will probably arise in the future, since this is just my hobby project, and I would like to take a more relaxed approach to it.

[IzzySoft]

Hi, I was wondering if there are any download or usage statistics for the app?

Sure there are: see iodstats. Some manual work currently, but we're working on visualization – something that will soon™ go live and let you do all the analysis you want using our Statistics Dashboard. Sneak preview for EasyNotes, last 3 months?

perhaps it could be considered for removal to help reduce maintenance overhead.

Ah, come on. Is it really soo much work to have a clean build from the tagged commit? I don't think so.

Similar situations will probably arise in the future, since this is just my hobby project, and I would like to take a more relaxed approach to it.

Understood. That's why we fixed it each time on our end. I just made suggestions how that can be avoided – in the hope this one single "rule" could be adopted by you. Apart from that, it's really an easy build.