TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)

[arifm-glitch]

PVC for client keys storing:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: openvpn
namespace: openvpn
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi

generating cert files and storing it to pvc

apiVersion: batch/v1
kind: Job
metadata:
name: openvpn-init-pki
namespace: openvpn
spec:
template:
spec:
restartPolicy: Never
containers:
- name: openvpn-init
image: kylemanna/openvpn:latest
command:
- /bin/sh
- -c
- |
set -e
echo "[INFO] Generating OpenVPN config..."
ovpn_genconfig -u udp://192.xx.xx.xx:1194

          echo "[INFO] Forcing tls-auth direction..."
          sed -i '/^tls-auth /c\tls-auth /etc/openvpn/pki/ta.key 0' /etc/openvpn/openvpn.conf

          echo "[INFO] Initializing PKI..."
          echo -ne '\n' | ovpn_initpki nopass
      volumeMounts:
        - name: openvpn-pki
          mountPath: /etc/openvpn
  volumes:
    - name: openvpn-pki
      persistentVolumeClaim:
        claimName: openvpn

---

We are using MetalLB in our environment, and using a static IP for openvpn service.
kubectl get ipaddresspool -A
NAMESPACE NAME AUTO ASSIGN AVOID BUGGY IPS ADDRESSES
metallb-system default-addresspool true false ["192.xx.xx.xx/16"]

apiVersion: apps/v1
kind: Deployment
metadata:
name: openvpn
namespace: openvpn
spec:
replicas: 1
selector:
matchLabels:
app: openvpn
template:
metadata:
labels:
app: openvpn
spec:
containers:
- name: openvpn
image: kylemanna/openvpn:latest
args:
- ovpn_run
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: true
ports:
- containerPort: 1194
protocol: UDP
- containerPort: 443
protocol: TCP
volumeMounts:
- name: openvpn-pki
mountPath: /etc/openvpn
volumes:
- name: openvpn-pki
persistentVolumeClaim:
claimName: openvpn

apiVersion: v1
kind: Service
metadata:
name: openvpn
namespace: openvpn
annotations:
metallb.universe.tf/loadBalancerIPs: 192.xx.xx.xx
spec:
type: LoadBalancer
selector:
app: openvpn
ports:
- name: openvpn-udp
port: 1194
protocol: UDP
targetPort: 1194
- name: openvpn-tcp
port: 443
protocol: TCP
targetPort: 443

Client logs:

openvpn --config client1.ovpn
2025-09-14 00:23:29 Note: --cipher is not set. OpenVPN versions before 2.5 defaulted to BF-CBC as fallback when cipher negotiation failed in this case. If you need this fallback please add '--data-ciphers-fallback BF-CBC' to your configuration and/or add BF-CBC to --data-ciphers.
2025-09-14 00:23:29 OpenVPN 2.6.14 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2025-09-14 00:23:29 library versions: OpenSSL 3.0.13 30 Jan 2024, LZO 2.10
2025-09-14 00:23:29 DCO version: N/A
2025-09-14 00:23:29 TCP/UDP: Preserving recently used remote address: [AF_INET]192.xx.xx.xx:1194
2025-09-14 00:23:29 UDPv4 link local: (not bound)
2025-09-14 00:23:29 UDPv4 link remote: [AF_INET]192.xx.xx.xx:1194
2025-09-14 00:23:31 read UDPv4 [EHOSTUNREACH]: No route to host (fd=3,code=113)
2025-09-14 00:24:30 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
2025-09-14 00:24:30 TLS Error: TLS handshake failed
2025-09-14 00:24:30 SIGUSR1[soft,tls-error] received, process restarting
2025-09-14 00:24:31 TCP/UDP: Preserving recently used remote address: [AF_INET]192.xx.xx.xx:1194
2025-09-14 00:24:31 UDPv4 link local: (not bound)
2025-09-14 00:24:31 UDPv4 link remote: [AF_INET]192.xx.xx.xx:1194
2025-09-14 00:25:31 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
2025-09-14 00:25:31 TLS Error: TLS handshake failed
2025-09-14 00:25:31 SIGUSR1[soft,tls-error] received, process restarting
2025-09-14 00:25:32 TCP/UDP: Preserving recently used remote address: [AF_INET]192.xx.xx.xx:1194
2025-09-14 00:25:32 UDPv4 link local: (not bound)

---

nc -v -u 192.xx.xx.xx 1194
Connection to 192.xx.xx.xx 1194 port [udp/openvpn] succeeded!

---

openvpn pod logs:

kubectl logs openvpn-8668cd8f67-9j4w9 -n openvpn
Checking IPv6 Forwarding
Sysctl error for default forwarding, please run docker with '--sysctl net.ipv6.conf.default.forwarding=1'
Sysctl error for all forwarding, please run docker with '--sysctl net.ipv6.conf.all.forwarding=1'
Running 'openvpn --config /etc/openvpn/openvpn.conf --client-config-dir /etc/openvpn/ccd --crl-verify /etc/openvpn/crl.pem '
Sat Sep 13 20:14:09 2025 OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
Sat Sep 13 20:14:09 2025 library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
Sat Sep 13 20:14:09 2025 Diffie-Hellman initialized with 2048 bit key
Sat Sep 13 20:14:09 2025 CRL: loaded 1 CRLs from file /etc/openvpn/crl.pem
Sat Sep 13 20:14:09 2025 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Sat Sep 13 20:14:09 2025 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Sat Sep 13 20:14:09 2025 ROUTE_GATEWAY 169.254.1.1
Sat Sep 13 20:14:09 2025 TUN/TAP device tun0 opened
Sat Sep 13 20:14:09 2025 TUN/TAP TX queue length set to 100
Sat Sep 13 20:14:09 2025 /sbin/ip link set dev tun0 up mtu 1500
Sat Sep 13 20:14:09 2025 /sbin/ip addr add dev tun0 local 192.xx.xx.xx.1 peer 192.xx.xx.xx.2
Sat Sep 13 20:14:09 2025 /sbin/ip route add 192.168.254.0/24 via 192.xx.xx.xx.2
Sat Sep 13 20:14:09 2025 /sbin/ip route add 192.xx.xx.xx.0/24 via 192.xx.xx.xx.2
Sat Sep 13 20:14:09 2025 Could not determine IPv4/IPv6 protocol. Using AF_INET
Sat Sep 13 20:14:09 2025 Socket Buffers: R=[212992->212992] S=[212992->212992]
Sat Sep 13 20:14:09 2025 UDPv4 link local (bound): [AF_INET][undef]:1194
Sat Sep 13 20:14:09 2025 UDPv4 link remote: [AF_UNSPEC]
Sat Sep 13 20:14:09 2025 GID set to nogroup
Sat Sep 13 20:14:09 2025 UID set to nobody
Sat Sep 13 20:14:09 2025 MULTI: multi_init called, r=256 v=256
Sat Sep 13 20:14:09 2025 IFCONFIG POOL: base=192.xx.xx.xx.4 size=62, ipv6=0
Sat Sep 13 20:14:09 2025 Initialization Sequence Completed

---

local openvpn version:

openvpn --version
OpenVPN 2.6.14 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
library versions: OpenSSL 3.0.13 30 Jan 2024, LZO 2.10
DCO version: N/A
Originally developed by James Yonan
Copyright (C) 2002-2024 OpenVPN Inc sales@openvpn.net
Compile time defines: enable_async_push=no enable_comp_stub=no enable_crypto_ofb_cfb=yes enable_dco=yes enable_dco_arg=yes enable_debug=yes enable_dependency_tracking=no enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=needless enable_fragment=yes enable_iproute2=no enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_maintainer_mode=no enable_management=yes enable_option_checking=no enable_pam_dlopen=no enable_pedantic=no enable_pkcs11=yes enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=no enable_strict_options=no enable_systemd=yes enable_unit_tests=no enable_werror=no enable_win32_dll=yes enable_wolfssl_options_h=yes enable_x509_alt_username=yes with_aix_soname=aix with_crypto_library=openssl with_gnu_ld=yes with_mem_check=no with_openssl_engine=auto with_sysroot=no
root@JoyJunnu:#

---

Openvpn image versions in kubernetes cluster:

kubectl exec -it openvpn-8668cd8f67-9j4w9 -n openvpn -- bash
bash-5.0# openvpn --version
OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020
library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2018 OpenVPN Inc sales@openvpn.net
Compile time defines: enable_async_push='no' enable_comp_stub='no' enable_crypto='yes' enable_crypto_ofb_cfb='yes' enable_debug='yes' enable_def_auth='yes' enable_dlopen='unknown' enable_dlopen_self='unknown' enable_dlopen_self_static='unknown' enable_fast_install='yes' enable_fragment='yes' enable_iproute2='yes' enable_libtool_lock='yes' enable_lz4='yes' enable_lzo='yes' enable_management='yes' enable_multihome='yes' enable_pam_dlopen='no' enable_pedantic='no' enable_pf='yes' enable_pkcs11='no' enable_plugin_auth_pam='yes' enable_plugin_down_root='yes' enable_plugins='yes' enable_port_share='yes' enable_selinux='no' enable_server='yes' enable_shared='yes' enable_shared_with_static_runtimes='no' enable_small='no' enable_static='yes' enable_strict='no' enable_strict_options='no' enable_systemd='no' enable_werror='no' enable_win32_dll='yes' enable_x509_alt_username='no' with_aix_soname='aix' with_crypto_library='openssl' with_gnu_ld='yes' with_mem_check='no' with_sysroot='no'

kubectl get pod openvpn-8668cd8f67-9j4w9 -n openvpn -oyaml | grep -i image:
image: kylemanna/openvpn:latest
image: docker.io/kylemanna/openvpn:latest

---

Client1.ovpn

cat client1.ovpn

client
nobind
dev tun
remote-cert-tls server

remote 192.xx.xx.xx 1194 udp

-----BEGIN PRIVATE KEY-----

-----END PRIVATE KEY-----


-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----


-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----

key-direction 1

-----BEGIN OpenVPN Static key V1-----

-----END OpenVPN Static key V1-----

redirect-gateway def1

---

Openvpn pod conf file

kubectl exec -it openvpn-8668cd8f67-cmwtc -n openvpn -- cat /etc/openvpn/openvpn.conf
server 192.168.255.0 255.255.255.0
verb 3
key /etc/openvpn/pki/private/192.xx.xx.xx.key
ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/192.xx.xx.xx.crt
dh /etc/openvpn/pki/dh.pem
tls-auth /etc/openvpn/pki/ta.key 0
key-direction 1
keepalive 10 60
persist-key
persist-tun

proto udp

Rely on Docker to do port mapping, internally always 1194

port 1194
dev tun0
status /tmp/openvpn-status.log

user nobody
group nogroup
comp-lzo no

Route Configurations Below

route 192.168.254.0 255.255.255.0

Push Configurations Below

push "block-outside-dns"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
push "comp-lzo no"

---

kubernetes node iptables:

sudo iptables -L INPUT -n
[sudo] password for devops:

Warning: iptables-legacy tables present, use iptables-legacy to see them

Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT udp -- 192.xx.xx.xx 0.0.0.0/0 udp dpt:1194

[schwabe]

All the docker/kubernetes and other stuff is probably not correctly working your packets. The nc test does not help either as testing UDP is not that easy.

[12:24]arne@kamera:~% nc -v -u 8.8.8.8 1194

Connection to 8.8.8.8 1194 port [udp/openvpn] succeeded

also shows that google dns has openvpn port open, which is obviously wrong.

Try to get your setup with TCP working first.