JBR-4578 Add Java ATK Wrapper #475

tanya011 · 2024-11-06T11:14:04Z

java-atk-wrapper used to be part of OpenJDK (JDK-8204862, IJPL-136231), but it was removed.
java-atk-wrapper is essential for accessibility in Java applications, including IntelliJ IDEA: IJPL-58438, JBR-4578.
AccessibleAnnouncerTest.java test is passed.
java-atk-wrapper has an LGPL 2.1 license, which can be up-cast to GPL-2.0, allowing the source code to be used.

vprovodin · 2024-11-22T12:49:17Z

All changes in JBR should be accompanied with tickets in YT related to the JBR project. Ticket-id should be placed at the beginning of the commit message. Within this JBR ticket we may make proper changes in docker files. Please note the libraries you aded to docker files actually are not available for Oracle Linux 8 that is build platform for jbr21.

OnePatchGuy

Thank you for the patch! It's been a hard work, hasn't it? :)

I've done reviewing only the changes in the build scripts (everything under the make directory). All the questions and suggestions are below. Reviewing the rest is coming.

From now on, please don't rewrite the commits history so that I'll be able to observe only the changed parts in newly added commits (we'll later squash them during merging).

OnePatchGuy · 2025-01-14T15:03:55Z

make/autoconf/libraries.m4

+    NEEDS_LIB_ATK=true
+    NEEDS_LIB_AT_SPI2_ATK=true
+    NEEDS_LIB_GLIB=true
+    NEEDS_LIB_GOBJECT=true
+    NEEDS_LIB_GLIBCONFIG=true


The patch adds new build-time dependencies, but:

Our CI doesn't respect them (yet), thus if we merge the patch as is, it'll immediately get broken. E.g. what I get if I try to make a build in a docker container as described in the README:

Package atspi-2 was not found in the pkg-config search path.
Perhaps you should add the directory containing 'atspi-2.pc'
to the PKG_CONFIG_PATH environment variable
Package 'atspi-2', required by 'virtual:world', not found
Creating support/modules_libs/jdk.accessibility/libatk-wrapper.so from 16 file(s)
/JetBrainsRuntime/src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c:25:10: fatal error: glib.h: No such > file or directory
#include <glib.h>
^~~~~~~~
compilation terminated.

(I get the same on Ubuntu 22 before installing the dependencies).

I can suggest 2 following ways of solving the problem:

to make the corresponding changes to our build scripts, docker images, etc. and include them into this PR ;

to make the changes provided in this patch disabled by default (i.e. apply --disable-java-atk-wrapper by default) and deal with all this CI stuff later after merging the patch .

Anyway, please note @vprovodin noticed earlier that Oracle Linux 8 which we're using as a build platform (via a docker image) doesn't provide these new dependencies, so it may become a serious obstacle for this patch.

The build instructions in README haven't been updated correspondingly.

The build instructions in README haven't been updated correspondingly.

I updated README, but not sure where to locate build instructions

not sure where to locate build instructions

Our README is currently describing 2 ways of how to build a JBR for Linux:

Using a specially designed docker container - the chapter Linux (Docker)

Using an Ubuntu as a host system - the chapter Ubuntu Linux

For now there's probably nothing to change in the Docker chapter, but Ubuntu chapter should be supplemented with the new packages and instructions how to enable/disable ATK wrapper and how to pass custom paths to it. Examples could be Windows's chapters Enable optional NVDA screen reader support and Disable optional JAWS screen reader support.

Done at f83061c

Looks good.
Let's file an issue about fixing CI and enabling back the patch by default and we can close this thread :).

make/autoconf/lib-at-spi2-atk.m4

make/autoconf/lib-atk.m4

.github/workflows/build-cross-compile.yml

OnePatchGuy · 2025-01-14T19:41:39Z

make/autoconf/libraries.m4

Just a note: here seems the full list of Ubuntu 22 packages that must be additionally installed to build the patch:

libatspi2.0-dev

libatk1.0-dev

libglib2.0-dev

libatk-bridge2.0-dev

make/autoconf/spec.gmk.in

make/modules/jdk.accessibility/Lib.gmk

OnePatchGuy · 2025-01-15T15:40:58Z

make/modules/jdk.accessibility/Lib.gmk

+                  -DATSPI_MINOR_VERSION=$(shell pkg-config --modversion atspi-2 | cut -d. -f2) \
+                  -DATSPI_MICRO_VERSION=$(shell pkg-config --modversion atspi-2 | cut -d. -f3), \
+        LDFLAGS := $(LDFLAGS_JDKLIB), \
+        LIBS_linux := -ljvm $(ATK_LIBS) $(AT_SPI2_ATK_LIBS) $(GLIB_LIBS) $(GOBJECT_LIBS), \


Consider adding -ljava and/or -lawt as well if needed (I don't know).

OnePatchGuy · 2025-01-15T15:56:48Z

My main question about the entire patch is the following: as you've written in the description, this functionality was removed some time ago due to a lot of bugs mentioned in e.g. IJPL-136231:

The problem is that the tool breaks the Swing threading rule and causes lots of exceptions & UI artefacts

and in JBR-4578:

Important! ATKWrapper contains bugs that can crash the JBR.

Also known issues:

Lists with complex renders work slowly;

Tree nodes are not voiced;

comboboxes are not spoken.

All these problems must be solved directly in ATKWrapper.

By putting all this logic back, do you state that all those problems have been fixed and now AWT wrapper is in production-ready quality?

UPD: still actual crash https://discourse.gnome.org/t/sigsegv-in-g-type-check-instance-is-fundamentally-a-0x11/24285

OnePatchGuy

I seem to have done reviewing everything related to the Wrapper initialization.
Besides that, this "round" also contains a few comments about the resource management and some general notes.

src/jdk.accessibility/linux/classes/META-INF/services/javax.accessibility.AccessibilityProvider

src/jdk.accessibility/linux/classes/module-info.java.extra

src/jdk.accessibility/linux/conf/accessibility.properties

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapper.java

src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapper.java

src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c

OnePatchGuy · 2025-02-01T01:31:29Z

BTW, I think each file that gets updated should also have its copyright header updated. One of the most recent examples of our header can be found e.g. in https://github.com/JetBrains/JetBrainsRuntime/blob/f89a6626d56dad71c3429c662bdad89f6b9a0591/src/java.desktop/unix/classes/sun/awt/wl/WLDisplay.java, but I don't know how to properly merge the existing headers with our one.

.github/workflows/build-cross-compile.yml

make/autoconf/lib-glib.m4

make/autoconf/lib-at-spi2-atk.m4

OnePatchGuy

Have finished reviewing the rest of the initial code. Have found a lot of issues, some of them are critical memory leaks or, in opposite, use-after-free errors

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkKeyEvent.java

OnePatchGuy · 2025-02-22T19:41:11Z

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkKeyEvent.java

+    private static void initializeMap() {
+        keyMap = new HashMap<>(146); // Currently only 110, so allocate 110 / 0.75
+
+        keyMap.put(Integer.valueOf(KeyEvent.VK_COLON), new GNOMEKeyInfo(0x20a1, "ColonSign")); // GDK_ColonSign


[refactoring] There are lots of meaningless integer boxing in this file, let's get rid of them.

done db7b3685d190f326c8786e7f7181c5db08f4cc1e

One is left at line 197

done 6ac03ba

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkKeyEvent.java

OnePatchGuy · 2025-02-22T20:19:35Z

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkKeyEvent.java

+                break;
+            }
+            default: {
+                char[] chars = new char[1];


Let's use String.valueOf(char) instead

done 253ad8c97f49241c7c900c886359b37c1112f283

Incorrect. Now an integer gets converted to a string, check the difference: https://www.programiz.com/online-compiler/8tH16xnBdMlKT

fixed cb05563

OnePatchGuy · 2025-02-27T16:23:32Z

src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c

+ * even been sent!
+ */
+static pthread_mutex_t jaw_vdc_dup_mutex = PTHREAD_MUTEX_INITIALIZER;
+static jobject jaw_vdc_last_ac = NULL;


Are the usages of this variable implemented correctly?

Comparisons to other JNI references are performed through "raw" pointer comparisons, although different JNI references may reference the same Java object.

It gets assigned to local JNI reference which are only valid during a JNI downcall.

done 1fb4bb8

Now it gets assigned to NewGlobalRef but ReleaseGlobalRef is never applied to it.

OnePatchGuy · 2025-02-27T17:26:14Z

src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c

+enum _SignalType {
+    Sig_Text_Caret_Moved = 0,
+    Sig_Text_Property_Changed_Insert = 1,
+    Sig_Text_Property_Changed_Delete = 2,
+    Sig_Text_Property_Changed_Replace = 3,
+    Sig_Object_Children_Changed_Add = 4,
+    Sig_Object_Children_Changed_Remove = 5,
+    Sig_Object_Active_Descendant_Changed = 6,
+    Sig_Object_Selection_Changed = 7,
+    Sig_Object_Visible_Data_Changed = 8,
+    Sig_Object_Property_Change_Accessible_Actions = 9,
+    Sig_Object_Property_Change_Accessible_Value = 10,
+    Sig_Object_Property_Change_Accessible_Description = 11,
+    Sig_Object_Property_Change_Accessible_Name = 12,
+    Sig_Object_Property_Change_Accessible_Hypertext_Offset = 13,
+    Sig_Object_Property_Change_Accessible_Table_Caption = 14,
+    Sig_Object_Property_Change_Accessible_Table_Summary = 15,
+    Sig_Object_Property_Change_Accessible_Table_Column_Header = 16,
+    Sig_Object_Property_Change_Accessible_Table_Column_Description = 17,
+    Sig_Object_Property_Change_Accessible_Table_Row_Header = 18,
+    Sig_Object_Property_Change_Accessible_Table_Row_Description = 19,
+    Sig_Table_Model_Changed = 20,
+    Sig_Text_Property_Changed = 21
+};
+


This enum is just a copy of AtkSignal.java. If you mark the constants from there with @Native annotation, they can be reused directly in native code (a native header org_GNOME_Accessibility_AtkSignal.h will be generated), thus avoiding the duplications.

Please also check all other native and Java enums for duplicates.

done 16166eb, and I renamed org_GNOME_Accessibility_AtkSignal.h to AtkSignal.h later. See src/jdk.accessibility/linux/native/libatk-wrapper/AtkSignal.h

AtkInterface: f4d4c23

The generated headers shouldn't be stored - they automatically get generated and integrated with the build system each time the project gets built. You just need to include such a header e.g.

#include "org_GNOME_Accessibility_AtkSignal.h"

to get access to the constants defined in the interface org.GNOME.Accessibility.AtkSignal.

So please remove all those committed headers.

The same situation is for AtkWrapper.h - just including org_GNOME_Accessibility_AtkWrapper.h instead will do the trick.

src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c

OnePatchGuy

Have reviewed the recent updates.

OnePatchGuy · 2025-02-28T16:28:30Z

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapperDisposer.java

+    private static final Object lock = new Object();
+    private static AtkWrapperDisposer disposerInstance;
+
+    static {


Let's convert the class to a lazy initialized singleton, so that threads will be created only when required and not when the class is getting loaded.

done c709d8c

Since now we check INSTANCE for null outside of the synchronized block, let's mark it volatile.

done 63aae9b

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapperDisposer.java

src/java.desktop/share/classes/module-info.java

src/jdk.accessibility/linux/native/libatk-wrapper/AtkWrapper.c

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapper.java

make/autoconf/lib-glib.m4

OnePatchGuy

Forgot to comment one piece.

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapperDisposer.java

… variables

OnePatchGuy self-requested a review November 6, 2024 13:45

OnePatchGuy assigned tanya011 Nov 6, 2024

tanya011 force-pushed the jbr21 branch 3 times, most recently from 4af7c31 to 081ca50 Compare November 19, 2024 16:56

tanya011 changed the title ~~IJPL-58438 Add Java ATK Wrapper~~ JBR-4578 Add Java ATK Wrapper Nov 28, 2024

tanya011 force-pushed the jbr21 branch from 73ed814 to 2acd460 Compare November 28, 2024 14:21

tanya011 force-pushed the jbr21 branch 2 times, most recently from 1d350f8 to 3fc3625 Compare December 18, 2024 16:15

OnePatchGuy requested changes Jan 15, 2025

View reviewed changes

OnePatchGuy requested changes Feb 1, 2025

View reviewed changes

OnePatchGuy reviewed Feb 1, 2025

View reviewed changes

.github/workflows/build-cross-compile.yml Outdated Show resolved Hide resolved

OnePatchGuy requested changes Feb 1, 2025

View reviewed changes

make/autoconf/lib-glib.m4 Outdated Show resolved Hide resolved

OnePatchGuy reviewed Feb 1, 2025

View reviewed changes

make/autoconf/lib-at-spi2-atk.m4 Outdated Show resolved Hide resolved

tanya011 requested a review from OnePatchGuy February 11, 2025 08:29

OnePatchGuy requested changes Feb 27, 2025

View reviewed changes

OnePatchGuy requested changes Feb 28, 2025

View reviewed changes

OnePatchGuy requested changes Mar 12, 2025

View reviewed changes

src/jdk.accessibility/linux/classes/org/GNOME/Accessibility/AtkWrapperDisposer.java Outdated Show resolved Hide resolved

tanya011 added 9 commits March 13, 2025 19:30

[args and vars null check] jawcomponent: null check for arguments and…

cd0281f

… variables

[args null check] jaweditabletext.c: null check for arguments

c26b94d

[vars null check] jaweditabletext.c: null check for variables

8bea066

[args null check] jawhypertext.c: null check for arguments

b0cc35c

[vars null check] jawhypertext.c: null check for variables

07dc49d

[args null check] jawhyperlink: null check for arguments

0d900ae

[vars null check] jawhyperlink: null check for variables

4b9c29b

[args null check] jawimage: null check for args

34fba3b

[vars null check] jawimage: null check for vars

5a94f8e

tanya011 added 15 commits December 22, 2025 11:44

run clang-format

6c91ed7

AtkKeyEvent: [refactoring] meaningless boxing

c998790

AtkWrapper.c: use DeleteLocalRef instead of PopLocalFrame

b53f2f9

AtkKeyEvent.java: do not cast char to int

f69e397

add todo

d077e2f

JAW_GET_OBJ_IFACE and JAW_GET_OBJ: create local ref instead of global

5b259e4

add debug comments

358cef8

jawcache.h: move variable declarations back to seperate files

ed1a7bf

jawcache.h: add #include <glib.h>

285016f

jawobject.c: cash some classes and methods

a34e067

jawimpl.c: cash some classes and methods

6a5998d

AtkWrapper.c: cash some classes and methods

3a6e888

introduce JAW_DEFAULT_LOCAL_FRAME_SIZE

3eb3261

JAW_GET_OBJ_IFACE and JAW_GET_OBJ: create local ref instead of global

7e22f06

add todo

5ea71b0

tanya011 force-pushed the jbr21 branch from 62fef26 to 5ea71b0 Compare December 22, 2025 10:46

tanya011 added 14 commits December 22, 2025 12:36

jawtablecell.c: remove unused description and jstrDescription

1b8f907

jawtable.c: add todo

30c8ade

add mutex to data

7735822

jawimpl.c: rename var

92c6977

Use const gchar* where string is not modified through this pointer

03abad3

AtkWrapper.c: remove PushLocalFrame

fc72063

remove wrong (*jniEnv)->DeleteGlobalRef(jniEnv, ac);

5e89df9

AtkText.java: return number of selected regions, not length

be5656a

AtkText.java and jawtext.c: fix typos

27410c7

AtkTableCell.java: check instanceof instead of casting

6ec65a0

AtkTableCell.java: add null checks

abd3264

Add null checks before addRecord

88e09b3

AtkWrapper.java: use equals so we don't send unnecessary signals

a06b6bc

AtkWrapper.java: add null check

91efa64

JBR-4578 Add Java ATK Wrapper #475

Are you sure you want to change the base?

JBR-4578 Add Java ATK Wrapper #475

Conversation

tanya011 commented Nov 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vprovodin commented Nov 22, 2024

Uh oh!

OnePatchGuy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OnePatchGuy Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OnePatchGuy commented Jan 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

OnePatchGuy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

OnePatchGuy commented Feb 1, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

OnePatchGuy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OnePatchGuy May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tanya011 commented Nov 6, 2024 •

edited

Loading

OnePatchGuy Jan 16, 2025 •

edited

Loading

OnePatchGuy commented Jan 15, 2025 •

edited

Loading

OnePatchGuy May 13, 2025 •

edited

Loading

tanya011 Mar 28, 2025 •

edited

Loading