Publish releases from CI

[gotson]

I am looking at publishing releases automatically from CI, so that we get a reliable, repeatable process to perform releases.

I will try to summarize the high level process, and then how we can apply that using the existing tooling.

Version management

Versioning explained

Versioning of sqlite-jdbc is tied to the version of sqlite we are bundling with the jar, to which we add an optional suffix that increments for every fix we do on our side.

For example the version of sqlite in our last release is 3.36.0, so we released a version 3.36.0 of the jar. After a fix on our side, but keeping the same version of sqlite, we would release a 3.36.0.1.

If we upgrade to the latest SQLite (3.39.2), we would release 3.39.2.

How version change

Version can be changed by 2 processes:

1. upgrade of the sqlite bundled version
2. internal fix, without changing the bundled sqlite version

1 is done manually, as it needs updating the VERSION file as well as the version in pom.xml, a recompile of the native libs, and sometimes other changes in the makefile. This is out of scope of automation for the moment.

2 is done using the maven-release-plugin, which would prepare a release from the SNAPSHOT version, then increment for the next release. For example if we are at 3.36.0.3, the pom.xml would have 3.36.0.3-SNAPSHOT, and after the release it would be 3.36.0.4-SNAPSHOT.

Tooling

maven-release-plugin

The maven-release-plugin performs various actions:

• prepare the pom.xml before release, and commit a change with the release version
• add a tag in git for the version
• bump the snapshot version after the release, and commit a change with the next snapshot version
• perform the release by deploying to Maven Central

It doesn't do:

• create a release on Github
• generate the list of changes since the last release

In order to perform a release from CI, we would need:

1. Trigger the release in a way we can detect and differentiate from normal push events
2. Launch the maven-release-plugin to prepare the release
3. Perform a git push so the repo has the correct tags and commits
4. Perform the release, effectively deploying to Maven Central
5. Generate a Github release with a changelog

🎯 Proposals

➡️ I think we should always include the suffix, to make the versioning consistent. For example we should release 3.39.2.0, and not 3.39.2. That will also make the versioning using maven-release-plugin easier, as the next snapshot will always be computed by increasing the last digit, thus it would compute 3.39.2.1-SNAPSHOT instead of 3.39.3-SNAPSHOT.

➡️ To trigger CI, given the maven-release-plugin will already handle the tagging, we could run a workflow manually. This is a handy way to trigger a release from Github. I use this in a few projects already.

➡️ To push the commits/tags upstream, I think we can pass pushChanges as true to the maven-release-plugin during the release on CI, so that the changes will be pushed upstream. Else we could perform a git push in the CI steps.

➡️ To create a Github release, we can either:

• do that manually, and Github now has a way to generate the changelog. This would be tedious to do the release manually though.
• we could use JReleaser, which can create Github releases with associated changelog. I am already using it for one of my project, and I am quite happy with it.

---

@xerial when you have time, can you provide some feedback on this ? Thanks

[gotson]

Here is a sample changelog generated by JReleaser for the future 3.39.2 release.

[xerial]

• Always using (sqlite version).(patch version) scheme is ok.
• For mvn release and release note automation, I have no preference. Go ahead with your familiar steps. Just leave some notes in README.md or somewhere appropriate where can we find the configuration manual in case we need to customize the behavior.

[gotson]

Thanks. Revamping the documentation is on my TODO list too. I was not familiar with the maven-release-plugin and I had to spend a bit of time to experiment.

[gotson]

Release from CI has been added, and the first release 3.39.2.0 was released ! 🎉

There were a few hiccups, but I managed to solve it (mostly):

• at first i created the release commit before doing the deployment to Maven Central
• the first deployment to Maven Central failed because of missing Javadoc jar and missing project URL in pom.xml. Those were things that were automatically done when we previously used the sonatype parent pom, but needed to be configured explicitly once we removed the deprecated parent pom.
• i had to delete the release commit by force pushing in order to retry
• i forgot to delete the generated tag, so the release commit of the 2nd attempt failed (i added the force option afterwards to avoid reoccurence), hence the rest of the workflow didn't run (JReleaser to create Github release and commit for next snapshot)
• i fixed the release tag manually
• i ran the changelog generation with JReleaser locally, and created the GitHub release manually
• i pushed the commit for the next snapshot manually

All in all it worked quite well, and should work flawlessly for the next times (🤞🏻 ).

I have also updated the documentation to mention that new process, and added some comments in the workflow file to explain what's going on.

[xerial]

Thanks a lot! And found changes in https://github.com/xerial/sqlite-jdbc/blob/master/CONTRIBUTING.md!

It's like sqlite-jdbc is revived :) I'm very impressed

[gotson]

I forgot to mention that i dropped the maven-release-plugin. It tries to do too much, and without all the fine grained control i would like. For instance it tries to checkout the project a second time to perform the deploy, but I couldn't make it work so that it would checkout the full depth, which is a requirement to generate the changelog properly.

I took some liberty in updating the README.md too, it had a lot of content, with some duplication, and wasn't straight to the point IMO.

• There was some duplication between README.md and Usage.md (renamed USAGE.md), as well as between README.md and README_BUILD.md (renamed CONTRIBUTING.md).
• I think some of the instructions were dating from many years, were Maven was probably not as widespread as it is now. Nowadays no library will bother to explain where to place a jar or how to use Maven, it should be available on various websites, so i took those out.
• I also removed old instructions for Tomcat6 that applied only to versions <3.8.
• I removed some mentions about pureJava version, as it's very old and not supported anymore.
• I also pulled the News section into NEWS.md. GitHub releases should be the source of truth for changes, and will be automatically generated from now on.

I also tagged ALL the issues :)

[gotson]

The commit about documentation changes: c84a077

[aalmiray]

Hi there! I'm the author of JReleaser, glad the tool served its purpose 😄

FWIW the problem with missing elements inpom.xml could have been detected by running PomChecker, which you can execute as easily as

$ mvn org.kordamp.maven:pomchecker-maven-plugin:1.2.0:check-maven-central