Skip to content

Commit 06ab12d

Browse files
committed
border: NAT gateway resource (border_agent / border_node / dedicated VM)
Introduce the Border control-plane resource: a machine-managed NAT gateway carrying its SNAT/DNAT rules inline (jsonb[]) on one flat resource, with a PaaS builder, user API, BorderBuilder unit tests and network.border.* permission seeds bound to project owners. Deployment shapes, calque of net_lb.type: * default -> border_agent capability on the core node's agent; * node set -> border_node pinned to that node's agent (distributed egress, e.g. a managed realm node); * type.kind: core -> border_node on a dedicated gateway VM the iaas builder provisions (net_border_iaas -> target_node_set). The core node's agent serves only border_agent: border_node is always pinned by the builder, and its driver ships where the data plane lives (realm node / border VM image), mirroring paas_lb_agent vs paas_lb_node. The bootstrap scheduler-section rewrite keeps border_agent, and a managed realm now registers in the ecosystem on first boot instead of waiting for the hourly telemetry loop. Also: single 0066 init migration (final net_border schema + permission seeds), gcl_sdk 3.1.0, drop core's own '/' static route on the shared LB vhost, point the CLI realm endpoint docs at Core's LB.
1 parent 5f9d703 commit 06ab12d

23 files changed

Lines changed: 979 additions & 29 deletions

File tree

docs/usage/local_deployment.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,10 +126,15 @@ field. Use this token as a `Bearer` token in all subsequent API requests.
126126
Configure the `exordos` CLI by registering a realm and a context with the admin credentials:
127127

128128
```bash
129-
exordos settings set-realm local --endpoint http://10.20.0.2:11010 --current
129+
exordos settings set-realm local --endpoint http://10.20.0.2:80/api/core --current
130130
exordos settings set-context local --name admin -u <ADMIN_USERNAME> -p <ADMIN_PASSWORD> --current
131131
```
132132

133+
The CLI authenticates against the `default` client alias (see above), which only Core's load
134+
balancer on port 80 knows how to rewrite. Pointing the realm at the node's own user-api port
135+
(`:11010`) instead skips the load balancer, so the alias can't be resolved and every command fails
136+
with `Can't parse value: uuid=default.`
137+
133138
- `set-realm` registers the platform endpoint under the name `local` and marks it as the active realm.
134139
- `set-context` creates a named context with the admin credentials and marks it as the active context.
135140

docs/usage/local_deployment.ru.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,10 +126,15 @@ curl --location 'http://10.20.0.2:80/api/core/v1/iam/clients/default/actions/get
126126
Настройте CLI `exordos`, зарегистрировав realm и контекст с учётными данными администратора:
127127

128128
```bash
129-
exordos settings set-realm local --endpoint http://10.20.0.2:11010 --current
129+
exordos settings set-realm local --endpoint http://10.20.0.2:80/api/core --current
130130
exordos settings set-context local --name admin -u <ADMIN_USERNAME> -p <ADMIN_PASSWORD> --current
131131
```
132132

133+
CLI аутентифицируется через алиас клиента `default` (см. выше), который умеет переписывать только
134+
балансировщик Core на порту 80. Если указать в качестве realm порт user-api самой ноды (`:11010`),
135+
балансировщик будет пропущен, алиас не резолвится, и любая команда падает с
136+
`Can't parse value: uuid=default.`
137+
133138
- `set-realm` — регистрирует endpoint платформы под именем `local` и устанавливает его как активный realm.
134139
- `set-context` — создаёт именованный контекст с учётными данными администратора и устанавливает его как активный.
135140

etc/exordos_universal_agent/exordos_universal_agent.conf.j2

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ caps_drivers =
1717
PasswordCapabilityDriver,
1818
CoreDNSCertificateCapabilityDriver,
1919
LBAgentCapabilityDriver,
20+
BorderAgentCapabilityDriver,
2021
GuestMachineCapabilityDriver,
2122
SSHKeyCapabilityDriver,
2223
RenderAgentDriver
@@ -28,7 +29,8 @@ capabilities =
2829
password,
2930
certificate,
3031
paas_lb_agent,
31-
repo_proxy_installed_element
32+
repo_proxy_installed_element,
33+
border_agent
3234

3335

3436
[CoreDNSCertificateCapabilityDriver]

exordos/manifests/core.yaml.j2

Lines changed: 0 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -268,26 +268,6 @@ resources:
268268
- kind: rewrite_url
269269
regex: "^/api/core/v1/iam/clients/default(.*)"
270270
replacement: f"/v1/iam/clients/{$core.vs.variables.$iam_default_client_uuid:value}$1"
271-
core_lb_static_route:
272-
project_id: "12345678-c625-4fee-81d5-f691897b8142"
273-
parent: $core.network.lb.$core_lb.vhosts.$core_lb_core_http:uuid
274-
condition:
275-
kind: prefix
276-
value: /
277-
allowed_ips:
278-
- 0.0.0.0/0
279-
modifiers:
280-
- kind: auto_header
281-
headers:
282-
- 'Host'
283-
- 'X-Forwarded-For'
284-
- 'X-Forwarded-Port'
285-
- 'X-Forwarded-Proto'
286-
- 'X-Forwarded-Prefix'
287-
actions:
288-
- kind: local_dir
289-
path: /var/www/html
290-
is_spa: true
291271

292272
exports:
293273
local_domain:

exordos_core/bootstrap/defaults.py

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
import typing as tp
2323
import uuid as sys_uuid
2424

25+
import bazooka
2526
from gcl_sdk.infra.dm import models as infra_models
2627
from restalchemy.dm import filters as dm_filters
2728
from restalchemy.storage import exceptions as ra_exceptions
@@ -673,3 +674,36 @@ def set_iam_default_client_secret_var(spec: dict) -> bool:
673674
spec["iam"]["default_client_secret"],
674675
c.VAR_IAM_DEFAULT_CLIENT_SECRET_UUID,
675676
)
677+
678+
679+
def register_realm_in_ecosystem(spec: dict[str, tp.Any]) -> bool:
680+
"""Self-register this realm in the ecosystem right after bootstrap.
681+
682+
Without this the first registration happens on the telemetry loop
683+
(hourly), so a managed realm stays PROVISIONING for up to an hour
684+
and races the ecosystem's provisioning timeout. Best-effort: the
685+
telemetry loop remains the retrying fallback, so a failure here
686+
must not fail the bootstrap.
687+
"""
688+
endpoint = spec.get("ecosystem_endpoint", "")
689+
realm_uuid = spec.get("realm_uuid", "")
690+
realm_secret = spec.get("realm_secret", "")
691+
if not (endpoint and realm_uuid and realm_secret):
692+
LOG.info("No realm identity in spec, skipping realm registration")
693+
return True
694+
695+
url = f"{endpoint.rstrip('/')}/api/ecosystem/v1/realms/"
696+
try:
697+
client = bazooka.Client(default_timeout=30)
698+
client.post(
699+
url,
700+
json={"uuid": realm_uuid, "secret": realm_secret},
701+
headers={"Content-Type": "application/json"},
702+
)
703+
LOG.info("Realm %s registered in ecosystem", realm_uuid)
704+
except Exception:
705+
LOG.exception(
706+
"Failed to register realm in ecosystem (the telemetry "
707+
"service will retry hourly)"
708+
)
709+
return True

exordos_core/cmd/bootstrap.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -415,7 +415,8 @@ def _migrate_installed_elements_configs() -> None:
415415
password,
416416
certificate,
417417
paas_lb_agent,
418-
repo_proxy_installed_element
418+
repo_proxy_installed_element,
419+
border_agent
419420
"""
420421
try:
421422
with open(UA_CONFIG_PATH, "r", encoding="utf-8") as f:
@@ -693,6 +694,10 @@ def _set_defaults_vs(spec: dict[str, tp.Any]):
693694
{"func": bootstrap_defaults.set_realm_secret_var, "args": [spec]},
694695
{"func": bootstrap_defaults.set_realm_access_token_var, "args": [spec]},
695696
{"func": bootstrap_defaults.set_realm_refresh_token_var, "args": [spec]},
697+
# Register in the ecosystem right away: waiting for the hourly
698+
# telemetry loop leaves a managed realm PROVISIONING for up to an
699+
# hour, racing the ecosystem's provisioning timeout.
700+
{"func": bootstrap_defaults.register_realm_in_ecosystem, "args": [spec]},
696701
{
697702
"func": bootstrap_defaults.set_hs256_jwks_encryption_key_var,
698703
"args": [CONF["iam"].hs256_jwks_encryption_key],

exordos_core/cmd/gservice.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,15 @@
3434
default="https://repo.exordos.com/exordos-lbaas/0.1.0/exordos-lbaas.raw.zst",
3535
help="URL to get image for LB dataplane VM",
3636
),
37+
cfg.StrOpt(
38+
"border-image",
39+
default=(
40+
"https://repo.exordos.com/exordos-border/0.1.0/exordos-border.raw.zst"
41+
),
42+
help="URL to get the image for a `core` (VM-based) border gateway. "
43+
"The image must ship the universal agent with "
44+
"BorderCapabilityDriver (border_node capability).",
45+
),
3746
]
3847

3948

exordos_core/gservice/service.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@
4848
from exordos_core.elements.services import builders as em_builders
4949
from exordos_core.janitor import service as janitor_service
5050
from exordos_core.network import service as n_network_service
51+
from exordos_core.network.border.builders import iaas as net_border_iaas
52+
from exordos_core.network.border.builders import paas as net_border_paas
53+
from exordos_core.network.border.dm import models as border_models
5154
from exordos_core.network.lb.builders import iaas as net_lb_iaas
5255
from exordos_core.network.lb.builders import paas as net_lb_paas
5356
from exordos_core.network.lb.dm import models as lb_models
@@ -123,6 +126,11 @@ def __init__(self, iter_min_period=3, iter_pause=0.1):
123126
project_id=nc.NODE_SET_PROJECT,
124127
)
125128
net_lb_paas_builder = net_lb_paas.LBBuilder()
129+
net_border_iaas_builder = net_border_iaas.BorderIaasBuilder(
130+
instance_model=border_models.IaasBorder,
131+
project_id=nc.NODE_SET_PROJECT,
132+
)
133+
net_border_paas_builder = net_border_paas.BorderBuilder()
126134

127135
# Infra scheduler
128136
infra_scheduler = ua_scheduler_service.UniversalAgentSchedulerService(
@@ -228,6 +236,8 @@ def __init__(self, iter_min_period=3, iter_pause=0.1):
228236
service_builder,
229237
net_lb_iaas_builder,
230238
net_lb_paas_builder,
239+
net_border_iaas_builder,
240+
net_border_paas_builder,
231241
secret_svc,
232242
password_builder,
233243
em_builder,

exordos_core/network/border/__init__.py

Whitespace-only changes.

exordos_core/network/border/builders/__init__.py

Whitespace-only changes.

0 commit comments

Comments
 (0)