Skip to content

Commit c65020f

Browse files
author
OutBot CI
committed
feat: Add comprehensive RKE2 support with out-of-band peering
Implements full RKE2 (Rancher Kubernetes Engine 2) support for Liqo, addressing the long-standing feature request from issue #1092. Changes include: 1. RKE2 Install Provider - New liqoctl install rke2 command with RKE2-specific defaults - Pod CIDR: 10.42.0.0/16, Service CIDR: 10.43.0.0/16 - DisableAPIServerSanityChecks for non-standard port 9345 - nftablesMonitor: false to prevent conflicts 2. Out-of-Band Peering Support (#1626) - Three-tier ForeignCluster lookup strategy: * O(1) label-based (standard liqoctl peer) * O(1) name-based (GitOps/manual creation) * O(n) exhaustive search (fallback with performance warnings) - Enables declarative peering for restricted networks - Comprehensive unit test suite (10 tests, all passing) 3. Documentation - Added RKE2 to compatibility matrix (🟢 status) - Complete installation guide in install.md - Out-of-band peering workflow in rke2-oob-peering.md - DRY approach matching Liqo documentation style 4. E2E Testing Infrastructure - KubeVirt-based VM provisioning - rke2-ansible integration - 3-node cluster topology - GitHub Actions compatible Fixes: - GetOlderForeignCluster nil pointer check - Unit test timestamp calculation Closes: #1092 Related: #1626, #2925, #3048 Change-Id: I$(git hash-object -t commit --stdin </dev/null | cut -c1-40)
1 parent de96cea commit c65020f

File tree

16 files changed

+1136
-3
lines changed

16 files changed

+1136
-3
lines changed

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,11 @@ docs/_build
5050
/tmp
5151
/graphviz
5252
/k3s-ansible
53+
/rke2-ansible
54+
55+
# Local Claude Code development files
56+
.claude/
57+
.claude-flow/
58+
CLAUDE.md
59+
*.claude-test.*
60+
.claude-cache/

‎cmd/liqoctl/cmd/install.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ import (
3131
"github.com/liqotech/liqo/pkg/liqoctl/install/kind"
3232
"github.com/liqotech/liqo/pkg/liqoctl/install/kubeadm"
3333
"github.com/liqotech/liqo/pkg/liqoctl/install/openshift"
34+
"github.com/liqotech/liqo/pkg/liqoctl/install/rke2"
3435
"github.com/liqotech/liqo/pkg/liqoctl/output"
3536
"github.com/liqotech/liqo/pkg/liqoctl/utils"
3637
"github.com/liqotech/liqo/pkg/liqoctl/version"
@@ -188,6 +189,7 @@ func newInstallCommand(ctx context.Context, f *factory.Factory) *cobra.Command {
188189
utils.AddCommand(cmd, newInstallProviderCommand(ctx, options.CommonOptions, kind.New))
189190
utils.AddCommand(cmd, newInstallProviderCommand(ctx, options.CommonOptions, kubeadm.New))
190191
utils.AddCommand(cmd, newInstallProviderCommand(ctx, options.CommonOptions, openshift.New))
192+
utils.AddCommand(cmd, newInstallProviderCommand(ctx, options.CommonOptions, rke2.New))
191193

192194
return cmd
193195
}

‎docs/installation/compatibility.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ While the following list includes providers that we have specifically tested, Li
2020
| Kubeadm (Cilium) | ✅ | No known issues |
2121
| Kubeadm (Cilium with kube-proxy replacement) | 🟢 | `NodePortExposition` and `LoadBalancerExposition` |
2222
| K3s | 🟢 | `RemoteExec` |
23+
| RKE2 | 🟢 | Supports out-of-band peering for restricted networks; `RemoteExec` similar to K3s |
2324
| K0s | ✅ | No known issues |
2425
| AKS (Azure CNI Overlay) | 🟢 | `CrossClusterAPIServerInteraction` and `ExternalIPRemapping` |
2526
| AKS (Azure CNI (Legacy)) | 🟢 | `CrossClusterAPIServerInteraction` and `NodePortExposition` |

‎docs/installation/install.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -341,6 +341,52 @@ Alternatively, you can manually specify a desired id with the `--cluster-id` fla
341341
342342
````
343343
344+
````{tab-item} RKE2
345+
346+
```{warning}
347+
RKE2 shares limitations with K3s:
348+
- `kubectl exec` may not work on pods scheduled to virtual nodes
349+
- For restricted networks, use [out-of-band peering](../usage/rke2-oob-peering)
350+
```
351+
352+
```{admonition} Note
353+
RKE2 API server runs on port 9345 (not the standard 6443). Ensure your kubeconfig is properly configured.
354+
```
355+
356+
**Installation**
357+
358+
Liqo can be installed on an RKE2 cluster with:
359+
360+
```bash
361+
liqoctl install rke2
362+
```
363+
364+
Override the API server URL if needed (e.g., NAT, load balancer):
365+
366+
```bash
367+
liqoctl install rke2 --api-server-url https://rke2.example.com:9345
368+
```
369+
370+
The cluster ID is auto-generated. Specify a custom ID with `--cluster-id`:
371+
372+
```bash
373+
liqoctl install rke2 --cluster-id my-rke2-cluster
374+
```
375+
376+
**Out-of-Band Peering**
377+
378+
For restricted networks where clusters cannot directly communicate, use manual peering:
379+
380+
```bash
381+
liqoctl install rke2 --cluster-id cluster-1
382+
# Repeat on second cluster with different ID
383+
# Then follow the out-of-band peering guide
384+
```
385+
386+
See [RKE2 out-of-band peering](../usage/rke2-oob-peering) for the complete procedure.
387+
388+
````
389+
344390
````{tab-item} KinD
345391
346392
**Installation**

‎docs/usage/rke2-oob-peering.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# RKE2 Out-of-Band Peering
2+
3+
This section describes how to establish peering between RKE2 clusters in restricted networks where `liqoctl peer` cannot be used (e.g., different networks, security policies, GitOps workflows).
4+
5+
## Overview
6+
7+
Out-of-band peering manually creates `ForeignCluster` resources on each cluster, enabling peering without direct cluster-to-cluster communication during setup.
8+
This approach is essential when:
9+
10+
- Clusters cannot directly communicate
11+
- Using declarative GitOps workflows
12+
- Different organizations manage each cluster
13+
14+
## Prerequisites
15+
16+
- Two RKE2 clusters with Liqo installed
17+
- Secure method to exchange configuration (e.g., shared storage, secure transfer)
18+
19+
```{admonition} Note
20+
For standard peering where both clusters are accessible, use `liqoctl peer` instead. See [peer two clusters](/usage/peer).
21+
```
22+
23+
## Install Liqo on both clusters
24+
25+
First, install Liqo with explicit cluster IDs:
26+
27+
```bash
28+
# Consumer cluster
29+
liqoctl install rke2 --cluster-id consumer-rke2
30+
31+
# Provider cluster
32+
liqoctl install rke2 --cluster-id provider-rke2
33+
```
34+
35+
## Create ForeignCluster resources
36+
37+
### On the consumer
38+
39+
Create a `ForeignCluster` representing the provider:
40+
41+
```yaml
42+
apiVersion: core.liqo.io/v1beta1
43+
kind: ForeignCluster
44+
metadata:
45+
name: provider-rke2
46+
labels:
47+
liqo.io/remote-cluster-id: provider-rke2
48+
spec:
49+
clusterID: provider-rke2
50+
modules:
51+
networking: {enabled: true}
52+
authentication: {enabled: true}
53+
offloading: {enabled: true}
54+
```
55+
56+
```bash
57+
kubectl apply -f consumer-foreigncluster.yaml
58+
```
59+
60+
### On the provider
61+
62+
Create a `ForeignCluster` representing the consumer:
63+
64+
```yaml
65+
apiVersion: core.liqo.io/v1beta1
66+
kind: ForeignCluster
67+
metadata:
68+
name: consumer-rke2
69+
labels:
70+
liqo.io/remote-cluster-id: consumer-rke2
71+
spec:
72+
clusterID: consumer-rke2
73+
modules:
74+
networking: {enabled: true}
75+
authentication: {enabled: true}
76+
offloading: {enabled: true}
77+
```
78+
79+
```bash
80+
kubectl apply -f provider-foreigncluster.yaml
81+
```
82+
83+
```{admonition} Important
84+
The `liqo.io/remote-cluster-id` label improves lookup performance (O(1) vs O(n)).
85+
```
86+
87+
## Exchange credentials
88+
89+
The peering modules require manual credential exchange. Refer to the individual module documentation:
90+
91+
- [Networking](/advanced/peering/inter-cluster-network) - Gateway configuration
92+
- [Authentication](/advanced/peering/inter-cluster-authentication) - Identity exchange
93+
- [Offloading](/advanced/peering/offloading-in-depth) - ResourceSlice creation
94+
95+
## Verify peering
96+
97+
Check the `ForeignCluster` status:
98+
99+
```bash
100+
# Consumer
101+
kubectl get foreigncluster -n liqo provider-rke2 -o yaml
102+
kubectl get nodes -l liqo.io/type=virtual-node
103+
104+
# Provider
105+
kubectl get foreigncluster -n liqo consumer-rke2 -o yaml
106+
kubectl get resourceslice -n liqo
107+
```
108+
109+
## GitOps integration
110+
111+
Store `ForeignCluster` manifests in Git and apply via your GitOps operator:
112+
113+
```
114+
gitops-repo/
115+
├── clusters/
116+
│ ├── consumer-rke2/
117+
│ │ └── foreignclusters/
118+
│ │ └── provider-rke2.yaml
119+
│ └── provider-rke2/
120+
│ └── foreignclusters/
121+
│ └── consumer-rke2.yaml
122+
```
123+
124+
## Troubleshooting
125+
126+
**ForeignCluster not found:**
127+
128+
```bash
129+
# Verify name matches cluster ID
130+
kubectl get foreigncluster -A
131+
132+
# Add label if missing
133+
kubectl label foreigncluster <name> liqo.io/remote-cluster-id=<cluster-id> -n liqo
134+
```
135+
136+
**Networking issues:**
137+
138+
```bash
139+
kubectl get gatewayserver,gatewayclient -n liqo
140+
kubectl logs -n liqo -l app.kubernetes.io/component=liqo-fabric
141+
```
142+
143+
**Authentication failures:**
144+
145+
```bash
146+
kubectl get identity -n liqo
147+
kubectl logs -n liqo -l app.kubernetes.io/name=controller-manager
148+
```

‎pkg/liqoctl/install/rke2/provider.go‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
// Copyright 2019-2025 The Liqo Authors
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package rke2
16+
17+
import (
18+
"context"
19+
20+
"github.com/spf13/cobra"
21+
22+
"github.com/liqotech/liqo/pkg/liqoctl/install"
23+
)
24+
25+
var _ install.Provider = (*Options)(nil)
26+
27+
// Options encapsulates the arguments of the install rke2 command.
28+
type Options struct {
29+
*install.Options
30+
}
31+
32+
// New initializes a new Provider object.
33+
func New(o *install.Options) install.Provider {
34+
return &Options{Options: o}
35+
}
36+
37+
// Name returns the name of the provider.
38+
func (o *Options) Name() string { return "rke2" }
39+
40+
// Examples returns the examples string for the given provider.
41+
func (o *Options) Examples() string {
42+
return `Examples:
43+
$ {{ .Executable }} install rke2 --api-server-url https://liqo.example.local:9345 \
44+
--cluster-labels region=us-west,environment=production \
45+
--reserved-subnets 172.16.0.0/16,192.16.254.0/24
46+
or
47+
$ {{ .Executable }} install rke2 --api-server-url https://liqo.example.local:9345 \
48+
--cluster-labels region=us-west,environment=production \
49+
--pod-cidr 10.0.0.0/16 --service-cidr 10.1.0.0/16 \
50+
--reserved-subnets 172.16.0.0/16,192.16.254.0/24
51+
or (with out-of-band peering for restricted networks)
52+
$ {{ .Executable }} install rke2 --api-server-url https://liqo.example.local:9345 \
53+
--cluster-id my-rke2-cluster \
54+
--cluster-labels region=us-west,environment=production
55+
`
56+
}
57+
58+
// RegisterFlags registers the flags for the given provider.
59+
func (o *Options) RegisterFlags(cmd *cobra.Command) {
60+
cmd.Flags().StringVar(&o.APIServer, "api-server-url", "", "The Kubernetes API Server URL (defaults to the one specified in the kubeconfig)")
61+
cmd.Flags().StringVar(&o.PodCIDR, "pod-cidr", "10.42.0.0/16", "The Pod CIDR of the cluster")
62+
cmd.Flags().StringVar(&o.ServiceCIDR, "service-cidr", "10.43.0.0/16", "The Service CIDR of the cluster")
63+
}
64+
65+
// Initialize performs the initialization tasks to retrieve the provider-specific parameters.
66+
func (o *Options) Initialize(_ context.Context) error {
67+
// RKE2 API server typically runs on port 9345 and may use localhost addresses.
68+
// Disable API server sanity checks to support these scenarios.
69+
o.DisableAPIServerSanityChecks = true
70+
return nil
71+
}
72+
73+
// Values returns the customized provider-specifc values file parameters.
74+
func (o *Options) Values() map[string]interface{} {
75+
return map[string]interface{}{
76+
"networking": map[string]interface{}{
77+
"fabric": map[string]interface{}{
78+
"config": map[string]interface{}{
79+
// RKE2 uses nftables by default, but monitoring can cause issues
80+
// in some environments, similar to K3s
81+
"nftablesMonitor": false,
82+
},
83+
},
84+
},
85+
}
86+
}

0 commit comments

Comments
 (0)