Binary compatibility docker by logart · Pull Request #542 · JetBrains/youtrackdb

logart · 2025-09-23T09:17:17Z

This is a continuation of #459

The main goal of this PR is to introduce docker file which will allow users to run binary compatibility tests on their private databases in isolated environments.

Add custom class loader which will load legacy db code.

Db comparison basic tests.

Remove parent class loader to make sure classes are only loaded from provided jar. Do not recreate git repo all the time. Add git checkout progress bar.

…test # Conflicts: # tests/src/test/java/com/jetbrains/youtrackdb/auto/BaseDBCompatibilityTest.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/CommonDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/JarDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/github/GithubJarBuilder.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/github/GithubRepoDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/github/MavenBuilder.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/maven/MavenJarDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/data/CommonDbDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/data/DbDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/data/FileDbDownloader.java # tests/src/test/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/data/HttpDbDownloader.java

Remove parent class loader to make sure classes are only loaded from provided jar. Do not recreate git repo all the time. Add git checkout progress bar. Add ability to exclude properties from comparison.(Password hashes does not match sometimes:) )

…will be introduced)

…rom terminal. Adjust sample config to not contain my own folder structure. Create docker file for testing. Add shadow jar plugin to not play with dependencies.

Copilot

Pull Request Overview

This PR introduces Docker support for binary compatibility testing in YouTrackDB. It creates a new binary-compat module that allows users to run binary compatibility tests on their private databases in isolated Docker environments.

Key changes:

Added a new binary-compat module with comprehensive binary compatibility testing framework
Enhanced core database comparison functionality to support property exclusion
Created Docker infrastructure for running tests in isolated environments

Reviewed Changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
binary-compat/pom.xml	New module POM with dependencies for Maven resolution, Git operations, and archive handling
binary-compat/docker/Dockerfile	Docker configuration for running binary compatibility tests
binary-compat/src/main/java/com/jetbrains/youtrackdb/auto/BaseDBCompatibilityChecker.java	Main test orchestration class handling database exports, imports, and comparisons across versions
binary-compat/src/main/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/*.java	Code fetching infrastructure for Maven and Git-based artifacts
binary-compat/src/main/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/data/*.java	Database downloading and preparation utilities
core/src/main/java/com/jetbrains/youtrackdb/internal/core/record/impl/EntityHelper.java	Enhanced entity comparison to support excluded properties
core/src/main/java/com/jetbrains/youtrackdb/internal/core/db/tool/DatabaseCompare.java	Updated database comparison to use property exclusion
tests/pom.xml	Added dependencies for binary compatibility testing
pom.xml	Added binary-compat module reference

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-06T10:20:30Z

binary-compat/README.md

@@ -0,0 +1,18 @@
+# This module is inteded to test binary compatibility of YouTrackDB releases.


Corrected spelling of 'inteded' to 'intended'.

Suggested change

# This module is inteded to test binary compatibility of YouTrackDB releases.

# This module is intended to test binary compatibility of YouTrackDB releases.

Copilot · 2025-10-06T10:20:30Z

...ava/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/github/GithubRepoDownloader.java

+
+        @Override
+        public void update(int completed) {
+          // this method it too verbose, we don't need it


Corrected grammar from 'it too verbose' to 'is too verbose'.

Suggested change

// this method it too verbose, we don't need it

// this method is too verbose, we don't need it

Copilot · 2025-10-06T10:20:31Z

binary-compat/src/main/java/com/jetbrains/youtrackdb/auto/BaseDBCompatibilityChecker.java

+    // this class loader "sees", all the classes from the current class loader which could issues,
+    // in case some classes are removed in the newer version of the code, but still present in the older one
+    // but without it, I will need a mechanism to load all the db dependencies,
+    // without specifying them manually one more time in the db compatibility test
+    // this is an open question for now.
+    // to deliver test sooner I will leave it as is with this comment


Corrected grammar from 'which could issues' to 'which could cause issues'.

Copilot

Pull Request Overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-06T12:09:42Z

...ain/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/maven/UnixMavenBuilder.java

@@ -0,0 +1,32 @@
+package com.jetbrains.youtrackdb.auto.binarycompat.fetcher.code.maven;
+
+import com.jetbrains.youtrackdb.auto.binarycompat.fetcher.code.github.MavenBuilder;


The import references a MavenBuilder interface from the github package, but this class is in the maven package. Consider moving the MavenBuilder interface to a more appropriate shared package or the maven package to avoid confusing package organization.

Suggested change

import com.jetbrains.youtrackdb.auto.binarycompat.fetcher.code.github.MavenBuilder;

import com.jetbrains.youtrackdb.auto.binarycompat.fetcher.code.maven.MavenBuilder;

Copilot · 2025-10-06T12:09:42Z

.../src/main/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/data/HttpDbDownloader.java

+    try (var is = new BufferedInputStream(conn.getInputStream())) {
+      outputDir = untar(is);
+    } finally {
+      conn.disconnect();
+    }


The HTTP connection doesn't validate content type or implement size limits. This could allow processing of malicious content. Consider adding content-type validation and maximum download size limits to prevent resource exhaustion attacks.

Copilot · 2025-10-06T12:09:42Z

binary-compat/src/main/java/com/jetbrains/youtrackdb/auto/BaseDBCompatibilityChecker.java

+    // without specifying them manually one more time in the db compatibility test
+    // this is an open question for now.
+    // to deliver test sooner I will leave it as is with this comment
+    var loader = new URLClassLoader(new URL[]{importJars.toURI().toURL()});


Creating a URLClassLoader with dynamically loaded JARs from external sources poses security risks. The loaded classes have full access to the JVM. Consider implementing security restrictions or validating JAR signatures before loading.

Copilot

Pull Request Overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-07T17:19:49Z

distribution/docker/Dockerfile

 FROM azul/zulu-openjdk:8

-MAINTAINER OrientDB LTD (info@orientdb.com)
-
 RUN mkdir /orientdb


The removal of the MAINTAINER line is correct as it's deprecated, but the remaining OrientDB references (line 8) suggest this Dockerfile may not be properly updated for YouTrackDB usage.

Copilot · 2025-10-07T17:19:50Z

...in/java/com/jetbrains/youtrackdb/auto/binarycompat/fetcher/code/github/GithubJarBuilder.java

+    this.mavenBuilders = Map.of(
+        OS.LINUX, unixMavenBuilder,
+        OS.MAC, unixMavenBuilder,
+         OS.WINDOWS, new WindowsMavenBuilder()


Extra space before 'OS.WINDOWS' creates inconsistent indentation compared to other map entries.

Suggested change

OS.WINDOWS, new WindowsMavenBuilder()

OS.WINDOWS, new WindowsMavenBuilder()

Copilot

Pull Request Overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 1 comment.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-10T13:43:00Z

binary-compat/README.md

@@ -0,0 +1,18 @@
+# This module is inteded to test binary compatibility of YouTrackDB releases.


Corrected spelling of 'inteded' to 'intended'.

Suggested change

# This module is inteded to test binary compatibility of YouTrackDB releases.

# This module is intended to test binary compatibility of YouTrackDB releases.

artem.loginov and others added 16 commits May 26, 2025 09:12

Add base binary compatibility test.

4a31583

Add custom class loader which will load legacy db code.

WIP.

4bce5eb

Db comparison basic tests.

Add configuration to specify which db to test against which version.

619eafa

Add maven and git based db downloaders.

b1710c7

Replace log with LogManager.

2f9137a

Add ability to get db not only from file system but from http.

f30022a

Remove parent class loader to make sure classes are only loaded from provided jar. Do not recreate git repo all the time. Add git checkout progress bar.

Comment out local path(it will be fixed completely when docker image …

73bd281

…will be introduced)

Check if tar entry is within outside path.

ea36387

Merge branch 'develop' into binary-compatibility-test

3a8c03d

Merge branch 'develop' into binary-compatibility-test

23f79af

Move compatibility classes to src from test, we need it to run them f…

c1adf86

…rom terminal. Adjust sample config to not contain my own folder structure. Create docker file for testing. Add shadow jar plugin to not play with dependencies.

Add separate module for binary compatibility test.

e47b6f8

Merge branch 'jetbrains/develop' into binary-compatibility-docker

8a986ac

Escape : in path.

95995c3

Copilot AI review requested due to automatic review settings October 6, 2025 10:19

Copilot AI reviewed Oct 6, 2025

View reviewed changes

artem.loginov added 2 commits October 6, 2025 12:26

Fix typo.

6aacd96

Add ability to build os dependant maven.

1872a60

Copilot AI review requested due to automatic review settings October 6, 2025 12:08

Copilot AI reviewed Oct 6, 2025

View reviewed changes

artem.loginov added 2 commits October 6, 2025 14:31

Downgrade to current version of ytdb.

d21e031

Update maven repo.

4874f98

Copilot AI review requested due to automatic review settings October 7, 2025 17:18

Copilot AI reviewed Oct 7, 2025

View reviewed changes

artem.loginov added 2 commits October 8, 2025 19:16

One more try.

6363de2

Fix windows build.

c18b4de

Copilot AI review requested due to automatic review settings October 10, 2025 13:42

Copilot AI reviewed Oct 10, 2025

View reviewed changes

		@@ -0,0 +1,18 @@
		# This module is inteded to test binary compatibility of YouTrackDB releases.

	# This module is inteded to test binary compatibility of YouTrackDB releases.
	# This module is intended to test binary compatibility of YouTrackDB releases.

	// this method it too verbose, we don't need it
	// this method is too verbose, we don't need it

		@@ -0,0 +1,32 @@
		package com.jetbrains.youtrackdb.auto.binarycompat.fetcher.code.maven;

		import com.jetbrains.youtrackdb.auto.binarycompat.fetcher.code.github.MavenBuilder;

	OS.WINDOWS, new WindowsMavenBuilder()
	OS.WINDOWS, new WindowsMavenBuilder()

Conversation

logart commented Sep 23, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant