failed to parse private key (decryption password incorrect)

[DrPyser]

Hi!
I'm trying out this cool utility for the first time.
Running it locally, I'm having trouble connecting to any of my ssh hosts from my config, as there seems to be an issue using my passphrases to decrypt the ssh key. And that even though I have an ssh agent configured(I also have IdentityFile settings on those hosts in my ssh config).

{"err":"failed to setup a authentication method: failed to parse private key: \"/home/drpyser/.ssh/id_drpyser\": x509: decryption password incorrect","level":"warn","msg":"root cause","time":"2024/01/10 19:51:26"}
{"err":"failed to setup a authentication method: failed to parse private key: \"/home/drpyser/.ssh/id_rsa\": x509: decryption password incorrect","level":"warn","msg":"root cause","time":"2024/01/10 19:58:52"}

I can verify my passphrases are working otherwise.

[caarlos0]

does it ask you for the password?

that said, it would probably work better with a ed25519 key...

[DrPyser]

So with latest versions it still doesn't work well for me.
My ssh config specifies which key should be used:

Host *.charlesnet
	...
        IdentityFile ~/.ssh/id_drpyser
	User root

but when I try to connect to a matching host(no issue with ssh directly) wishlist asks me for the passphrase(though it should be cached by my ssh agent), then asks me for the passphrase of another key.

Logs:

{"key.fingerprint":"SHA256:IgpptOk1XrJlC/LBuSd90D3MzRWrlq6niNvxyxbHCB0","key.path":"/home/drpyser/.ssh/id_drpyser","key.type":"ssh-ed25519","level":"info","msg":"offering public key","time":"2025/07/02 20:55:30"}
{"key.fingerprint":"SHA256:m3r7phrica4qETRt9s9YP4zdXOw0UAWnfZO0aKO8/6M","key.type":"ssh-rsa","level":"info","msg":"offering public key via ssh agent","time":"2025/07/02 20:55:30"}
{"key.fingerprint":"SHA256:oU98tLCMDrTKqlLjbtpb9Ehnxr5/gBXG4Wt+y69PAZ4","key.type":"ssh-rsa","level":"info","msg":"offering public key via ssh agent","time":"2025/07/02 20:55:30"}
{"key.fingerprint":"SHA256:IgpptOk1XrJlC/LBuSd90D3MzRWrlq6niNvxyxbHCB0","key.type":"ssh-ed25519","level":"info","msg":"offering public key via ssh agent","time":"2025/07/02 20:55:30"}
{"key.fingerprint":"SHA256:hjPwD+Fqw56rymzkfzAqKEaNHZP1tZmBkbH5JqvGnzY","key.type":"ssh-ed25519","level":"info","msg":"offering public key via ssh agent","time":"2025/07/02 20:55:30"}
{"key.fingerprint":"SHA256:oU98tLCMDrTKqlLjbtpb9Ehnxr5/gBXG4Wt+y69PAZ4","key.path":"/home/drpyser/.ssh/id_rsa","key.type":"ssh-rsa","level":"info","msg":"offering public key","time":"2025/07/02 20:55:34"}

So it starts by trying the right key, but then tries all keys from the ssh-agent. There's no log about why it the first one doesn't work. Is it possible to get more logs?