FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information

[Vista2003]

Expected behavior:

Refreshed the Streisand GPG keyring

Actual Behavior:

TASK [gpg : Refresh the Streisand GPG keyring with keyserver information] ************************************************************************************************************************************************************
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (10 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (9 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (8 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (7 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (6 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (5 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (4 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (3 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (2 retries left).
FAILED - RETRYING: Refresh the Streisand GPG keyring with keyserver information (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 10, "changed": true, "cmd": ["gpg2", "--no-default-keyring", "--keyring", "/root/.gnupg/streisand/pubring.gpg", " --keyserver-options", "timeout=120", "--refresh"], "delta": "0:00:00.529998", "end": "2021-03-10 13:07:32.375476", "msg": "non-zero return code", "rc": 2, "s tart": "2021-03-10 13:07:31.845478", "stderr": "gpg: refreshing 10 keys from hkps://gpg.mozilla.org\ngpg: keyserver refresh failed: General error", "stderr_l ines": ["gpg: refreshing 10 keys from hkps://gpg.mozilla.org", "gpg: keyserver refresh failed: General error"], "stdout": "", "stdout_lines": []}

Steps to Reproduce:

1. Run the setup

[ contents of streisand-diagnostics.md here ]

Additional Details:

Log output from Ansible or other relevant services (link to Gist for longer output):

Target Cloud Provider:

Oracle Cloud (Localhost)

Operating System of target host:

Ubuntu 16.04

Operating System of client:

Localhost

Version of Ansible, using ansible --version :

/home/ubuntu/streisand/venv/lib/python3.5/site-packages/ansible/parsing/vault/init.py:41: CryptographyDeprecationWarning: Python 3.5 support will be dropped in the next release of cryptography. Please upgrade your Python.
from cryptography.exceptions import InvalidSignature
ansible 2.8.4
config file = /home/ubuntu/streisand/ansible.cfg
configured module search path = ['/home/ubuntu/streisand/library']
ansible python module location = /home/ubuntu/streisand/venv/lib/python3.5/site-packages/ansible
executable location = /home/ubuntu/streisand/venv/bin/ansible
python version = 3.5.2 (default, Jan 26 2021, 13:30:48) [GCC 5.4.0 20160609]

Output from git rev-parse HEAD in your Streisand directory :

af5eb7d

[reallyasi9]

This appears to be affecting AWS targets as well. For reference:

Ansible Information

• Ansible version: 2.8.4
• Ansible system: Linux
• Host OS: Ubuntu
• Host OS version: 20.10
• Python interpreter: python3
• Python version: 3.8.6

Streisand Information

• Streisand Git revision: af5eb7d
• Streisand Git clone has untracked changes: yes
• Genesis role: genesis-amazon
• Custom SSH key: False

Untracked git changes:

diff --git a/util/dependencies.txt b/util/dependencies.txt
index a7a5f9b..7b40033 100644
--- a/util/dependencies.txt
+++ b/util/dependencies.txt
@@ -4,6 +4,6 @@ python3-openssl
 python3-dev
 python3-setuptools
 python3-venv
-python-cffi libffi-dev
+python3-cffi libffi-dev
 libssl-dev
 libcurl4-openssl-dev

Enabled Roles

• Shadowsocks enabled: False
• Wireguard enabled: True
• OpenVPN enabled: False
• stunnel enabled: False
• Tor enabled: True
• Openconnect enabled: False
• TinyProxy enabled: False
• SSH forward user enabled: False
• Configured number of VPN clients: 10

[Vista2003]

Honestly, at this point, I think the project has sadly been largely abandoned and we're just shouting into a black hole and hoping that someone will see it

[antoineclaval]

I'm still in the process and getting it back up and running fully. In my case the install succeed and serve the documentation and configs file, but iptables do not have adequat configuration.

For you particular issues, I did the following workaround.

ssh to the target :

cd /root/.gnupg/
mkdir S.dirmngr
mkdir S.gpg-agent
sudo chown -R $USER ~/.gnupg

[HN-Smith]

This is due to a long-standing PGP key server vulnerability, triggered starting almost 2 years ago:
https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f
https://gist.github.com/rjhansen/f716c3ff4a7068b50f2d8896e54e4b7e

There's a mitigation fix in GnuPG 2.2.17, however Ubuntu 16.04 has version 2.1.11:
https://lists.gnupg.org/pipermail/gnupg-announce/2019q3/000439.html

https://latacora.micro.blog/2019/07/16/the-pgp-problem.html
Seen elsewhere such as here: torproject/torbrowser-launcher#401

Fixed by commenting out the refresh for now.

diff --git a/playbooks/roles/gpg/tasks/main.yml b/playbooks/roles/gpg/tasks/main.yml
index 8f88833..e4a9f02 100644
--- a/playbooks/roles/gpg/tasks/main.yml
+++ b/playbooks/roles/gpg/tasks/main.yml
@@ -85,7 +85,8 @@
   # in the repo become too stale to be used without successsful refresh the
   # maintainers will notice failed builds and fix them by refreshing their own
   # keyrings and updating the static repo keys until the build passes again.
-  when: not streisand_ci
+  #when: not streisand_ci
+  when: false

 - name: "Set up a daily cronjob to refresh the Streisand GPG keyring"
   template:
@@ -95,4 +96,5 @@
     group: root
     mode: 0755
   # There's no point installing a cronjob in CI
-  when: not streisand_ci
+  #when: not streisand_ci
+  when: false

Then got to another error:

TASK [i18n-docs : Convert the OpenConnect mirror Markdown page into HTML] **********************************************
changed: [55.55.55.55] => (item=English)
changed: [55.55.55.55] => (item=Français)

TASK [openvpn : Add the official OpenVPN APT key; hiding 25 lines of log...] *******************************************
failed: [55.55.55.55] (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
fatal: [55.55.55.55]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}

Here it turns out the OpenVPN signing public key being installed is outdated:

# apt-key list
/etc/apt/trusted.gpg
--------------------
[...]
pub   2048R/E158C569 2011-08-03 [expired: 2020-07-25]
uid                  Samuli Seppänen (OpenVPN Technologies, Inc) <samuli@openvpn.net>

Found a fix here: https://community.openvpn.net/openvpn/wiki/OpenvpnSoftwareRepos#Notesonexpiredkeys

Updated like this:

wget -O - https://swupdate.openvpn.net/repos/repo-public.gpg > playbooks/roles/openvpn/files/openvpn_signing.key

The V2Ray option for shadowsocks also did not work. I've yet to retry without it.

TASK [shadowsocks : [Temporary] Clone v2ray-core repository manually to GOPATH] ****************************************
changed: [54.189.141.25]

TASK [shadowsocks : Get V2Ray-plugin] **********************************************************************************
fatal: [54.189.141.25]: FAILED! => {"changed": true, "cmd": "go get github.com/shadowsocks/v2ray-plugin", "delta": "0:00:26.172822", "end": "2021-04-14 03:44:36.428888", "msg": "non-zero return code", "rc": 2, "start": "2021-04-14 03:44:10.256066", "stderr": "# github.com/xtls/go\n/root/go/src/github.com/xtls/go/auth.go:29:7: undefined: ecdsa.VerifyASN1\n/root/go/src/github.com/xtls/go/key_schedule.go:177:16: xShared.FillBytes undefined (type *big.Int has no field or method FillBytes)\ngo build github.com/lucas-clemente/quic-go/internal/qtls: build constraints exclude all Go files in /root/go/src/github.com/lucas-clemente/quic-go/internal/qtls", "stderr_lines": ["# github.com/xtls/go", "/root/go/src/github.com/xtls/go/auth.go:29:7: undefined: ecdsa.VerifyASN1", "/root/go/src/github.com/xtls/go/key_schedule.go:177:16: xShared.FillBytes undefined (type *big.Int has no field or method FillBytes)", "go build github.com/lucas-clemente/quic-go/internal/qtls: build constraints exclude all Go files in /root/go/src/github.com/lucas-clemente/quic-go/internal/qtls"], "stdout": "", "stdout_lines": []}

Note: These Ansible playbooks are not idempotent, and not setup to allow the --start-at-task option to work. Configuration choices are not stored anywhere. Fixing bugs involves manually terminating instances and other things and restarting from scratch, answering all prompts (some in the middle), every time. Not the fastest. Perhaps there are tricks to debug and fix these scripts quickly I'm not aware of. The following is apparently needed for --start-at-task and doesn't seem to help:

diff --git a/ansible.cfg b/ansible.cfg
index 059fc44..8768dfe 100644
--- a/ansible.cfg
+++ b/ansible.cfg
@@ -24,3 +24,7 @@ library=library
 # Enables multiplexing (lets ansible reuse opened SSH connections)
 ssh_args = -o ControlMaster=auto -o ControlPersist=60s
 pipelining = True
+
+# Use static includes to allow --start-at-task to work
+task_includes_static = True
+handler_includes_static = True

[HN-Smith]

FYI, retrying without V2Ray (as per my last post), the process went further and failed trying to install the WireGuard PPA. Which led me to this web page: https://computingforgeeks.com/setup-vpn-server-on-linux-using-streisand/

WireGuard is now part of standard repos and the attempt to to add a custom package archive (PPA) can simply be removed. Install succeeded after that fix. That page also suggests a solution similar to the last post for the GPG keyring issue.

[antoineclaval]

@HN-Smith
Good link. That basically the steps I followed by gathering information here and there in this repo.

But at the end I end-up with a running streissand UI serving the config and doc, but the client can't reach the openVPN server. I suspect something is wrong between iptable and ufw.

I'm a bit stretch out by the amount of manual tweak I had to do.
In parallel, I setup a plain-boring openVPN server and that was actually faster to do so.

Streissand does provide very neat documentation and serve the client configs in a neat way. But... I'm starting to loose confidence in the setup, I don't trust myself to review the change I'm doing and I'm afraid to provide only a sense of privacy ( = a leaky VPN )

In addition, It's starting to be challenging to run the ansible script against the 16.04 as well, for instance my server provider don't propose that version anymore. I had to adapt to 18.04.

Shame, I like that project and used it successfully for a long time.