Performance issue for batch sign task?

[TommyLike]

We have setup up an environment to benchmark the singing performance of obs sign solution, considering the case:
server: single instance, 8 CPUs &8GB RAM
client: 1/2/4 instances 8 CPUs &10GB RAM
test data: about 4000 packages of openEuler 21.09 for each client

Since obs sign command doesn't support batch sign tasks, we wrap obs sign command with 50 golang goroutines in which the sign client will be launched individually, and the result shows it will take a long time to finish:

Results:
1 client: 21m9s
2 clients: 37m33s/40m16s
4 clients: 80m2s/80m3s/79m58s/80m

Question:
Is this a expected result or there is some concurrence related configuration? Anyway to improve the performance?

[mlschroe]

The problem is that the gpg's agent implementation does not support concurrent requests. It supports threads but there is a semaphore that makes sure only one thread is running at a time.

Because we ran into the same issue I've added the possibility to sign the libgcrypt in the last weeks. To make use of this a couple of prerequisites have to be met:

• "use-gcrypt" has to be set to "true" in the config
• the perl-Crypt-GCrypt package from OBS:Server:Unstable needs to be installed
• the signing keys in the keyring need to be unprotected (i.e. have an empty passphrase)

[TommyLike]

@mlschroe thanks for you explanation. now I get it. Actually we are creating a new rust version of obs-sign too, and it will utilize the openpgp library instead of pgp agent. As a result, it tests well and the total time of each clients will be optimized to about 3 minutes.

[TommyLike]

The problem is that the gpg's agent implementation does not support concurrent requests. It supports threads but there is a semaphore that makes sure only one thread is running at a time.

Because we ran into the same issue I've added the possibility to sign the libgcrypt in the last weeks. To make use of this a couple of prerequisites have to be met:

• "use-gcrypt" has to be set to "true" in the config
• the perl-Crypt-GCrypt package from OBS:Server:Unstable needs to be installed
• the signing keys in the keyring need to be unprotected (i.e. have an empty passphrase)

I will try this version too, thanks!

[mlschroe]

Just FYI: there's a bug in the golang-x-crypto's pgp implementation that makes it create signatures that violate the pgp standard. See the discussion in:
rpm-software-management/rpm#2351

[githubliuyang777]

The problem is that the gpg's agent implementation does not support concurrent requests. It supports threads but there is a semaphore that makes sure only one thread is running at a time.

Because we ran into the same issue I've added the possibility to sign the libgcrypt in the last weeks. To make use of this a couple of prerequisites have to be met:

• "use-gcrypt" has to be set to "true" in the config
• the perl-Crypt-GCrypt package from OBS:Server:Unstable needs to be installed
• the signing keys in the keyring need to be unprotected (i.e. have an empty passphrase)

@mlschroe we set up 4 clients and 1 server, the first 2 clients works well , but the other 2 clients block for long time. when block clients continue running, server print "zero size packet".

[TommyLike]

@mlschroe as @githubliuyang777 said, we tried the new version of pgp and when concurrent requests are made by several clients it's possible that part of the clients will get zero size packet error.

[githubliuyang777]

hi @mlschroe ，any update?