how to force to use a specific identity from ssh agent?

[AlBundy33]

I've tried to run
gradle release -Prelease.versionIncrementer=incrementPatch
on a remote machine where I'm loggedin with ssh and forwarded agent

ssh-add -l shows my identities but when I run the mentioned command it ends with.

> Task :some-fancy-lib:verifyRelease
Looking for uncommitted changes..
Checking if branch is ahead of remote..
Checking for snapshot versions..

> Task :some-fancy-lib:release FAILED
Creating tag: v1.3.2
Pushing all to remote: origin
Exception occurred during push: org.eclipse.jgit.api.errors.TransportException: ssh://git@gitlab.fancycompany.de:2222/development/libs/some-fancy-lib.git: Auth fail for methods 'publickey'

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':some-fancy-lib:release'.
> org.eclipse.jgit.api.errors.TransportException: ssh://git@gitlab.fancycompany.de:2222/development/libs/some-fancy-lib.git: Auth fail for methods 'publickey'

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 861ms
2 actionable tasks: 2 executed

copying my private key to the remote-machine and setting

release.customKeyFile=/home/fancyuser/.ssh/id_ed25519
release.customKeyPassword=

works, but I don't want to distribute my private key on each machine.

Is there something I can do to force the usage of a specific forwarded identity?