Skip to content

Commit 3936442

Browse files
wbpcodewbpcode
authored
dns gateway: create manifest for every filter (#537)
1 parent 91a860c commit 3936442

26 files changed

Lines changed: 695 additions & 235 deletions

File tree

‎cli/internal/extensions/manifest_test.go‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -394,11 +394,10 @@ func TestValidateParentManifest(t *testing.T) {
394394
wantErr bool
395395
}{
396396
{"parent_valid.yaml", false},
397-
// NOTE(general-bundle): resolveVersions now lets non-Go children inherit from a bundle
398-
// parent (see TestResolveVersionsForBundleChildren), but manifest.schema.json still
399-
// requires a child's `type` to be `go`, so a Rust child currently fails schema
400-
// validation. Flip to `false` once the schema allows non-Go bundle children.
401-
{"parent_rust_valid.yaml", true},
397+
// A Rust child of an extension set is valid: manifest.schema.json permits a bundle
398+
// child's `type` to be `go` or `rust` (see TestResolveVersionsForBundleChildren).
399+
{"parent_rust_valid.yaml", false},
400+
// A type outside {go, rust} (here wasm) is still rejected for a bundle child.
402401
{"parent_invalid_type.yaml", true},
403402
{"parent_missing.yaml", true},
404403
{"parent_with_version.yaml", true},

‎cli/internal/extensions/testdata/parent_invalid_type.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ author: Test Author
1111
description: A child extension that inherits parent version
1212
longDescription: |
1313
This is a child extension.
14-
type: rust
14+
type: wasm
1515
tags:
1616
- test
1717
license: Apache-2.0

‎cli/internal/extensions/testdata/parent_rust_valid.yaml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Copyright Built On Envoy
2+
# SPDX-License-Identifier: Apache-2.0
3+
# The full text of the Apache license is available in the LICENSE file at
4+
# the root of the repo.
5+
6+
name: parent-rust-valid
7+
parent: dns-gateway
8+
categories:
9+
- Network
10+
author: Test Author
11+
description: A Rust child extension that inherits its version from a bundle parent
12+
longDescription: |
13+
This is a Rust child extension of an extension set.
14+
type: rust
15+
filterType:
16+
- network
17+
tags:
18+
- test
19+
license: Apache-2.0
20+
examples: []

‎extensions/dns-gateway/README.md‎

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,26 @@
11
# DNS Gateway
22

33
Transparent egress routing for Envoy. Applications make normal DNS lookups and TCP connections
4-
— this extension intercepts both so Envoy can control which upstream cluster handles each
4+
— this extension set intercepts both so Envoy can control which upstream cluster handles each
55
domain, attach metadata (auth tokens, policy IDs), and enforce routing policy, all without
66
any application changes.
77

8+
This is an **extension set**: the vendor manifest at the repo root
9+
([`manifest.yaml`](manifest.yaml)) declares the shared `dns-gateway` dynamic module, and two
10+
child extensions under [`manifests/`](manifests) expose its two filters independently:
11+
12+
| Extension | Filter type | Role |
13+
| --------- | ----------- | ---- |
14+
| [`resolver`](manifests/resolver) | `udp_listener` | Intercept DNS queries, allocate virtual IPs |
15+
| [`lookup`](manifests/lookup) | `network` | Resolve virtual IPs back to domain + metadata as filter state |
16+
17+
Both filters are compiled into the same `libdns_gateway.so` and share a process-wide virtual-IP
18+
cache, so they must run in the same Envoy instance.
19+
20+
> **Backward compatibility:** the legacy combined `dns-gateway` extension still works —
21+
> `--extension dns-gateway --filter-type udp_listener` and `--extension dns-gateway --filter-type network`
22+
> resolve to the same two filters. Both filter names are registered by the dynamic module.
23+
824
**Use cases:**
925
- Route outbound traffic through different upstream proxies based on destination domain
1026
- Attach per-domain credentials or policy metadata to proxied connections
@@ -21,16 +37,17 @@ Requires iptables/nftables rules to redirect application traffic to Envoy:
2137

2238
## How it works
2339

24-
Both filters are provided by the same `dns-gateway` dynamic module; enable each by its filter
25-
type (`udp_listener` and `network`).
40+
Both filters are provided by the same `dns-gateway` dynamic module; enable each via its own
41+
extension (`resolver` for the `udp_listener` filter, `lookup` for the
42+
`network` filter), referenced as `dns-gateway/resolver` and `dns-gateway/lookup`.
2643

27-
1. **UDP listener filter** — Intercepts DNS queries. If the queried domain matches
44+
1. **`resolver` (UDP listener filter)** — Intercepts DNS queries. If the queried domain matches
2845
a configured pattern, allocates a virtual IP from that domain's dedicated CIDR range and responds
2946
with an A record. Each domain pattern gets its own IP range, so `*.aws.com` and `*.google.com`
3047
allocate from separate subnets. Caches the mapping from virtual IP to domain and metadata.
3148
Non-matching queries pass through.
3249

33-
2. **Network filter** — On new TCP connections, looks up the destination virtual IP
50+
2. **`lookup` (Network filter)** — On new TCP connections, looks up the destination virtual IP
3451
in the shared cache and sets the resolved domain and metadata as Envoy
3552
[filter state](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/data_sharing_between_filters#primitives)
3653
for use in routing.
@@ -39,13 +56,13 @@ type (`udp_listener` and `network`).
3956
Application
4057
| DNS query: "bucket-1.aws.com"
4158
v
42-
UDP listener filter
59+
resolver (UDP listener filter)
4360
| matches "*.aws.com", allocates 10.0.0.0 from *.aws.com's range, responds with A record
4461
v
4562
Application
4663
| TCP connect to 10.0.0.0:443
4764
v
48-
network filter
65+
lookup (network filter)
4966
| resolves 10.0.0.0 -> domain="bucket-1.aws.com", metadata.cluster="aws"
5067
v
5168
tcp_proxy
@@ -56,9 +73,9 @@ type (`udp_listener` and `network`).
5673

5774
## Filter state
5875

59-
The network filter sets Envoy filter state keys readable via `%FILTER_STATE(<key>:PLAIN)%`.
76+
The `lookup` network filter sets Envoy filter state keys readable via `%FILTER_STATE(<key>:PLAIN)%`.
6077
The default prefix is `io.builtonenvoy.dns_gateway` and is configurable via `filter_state_prefix`.
61-
See the [extension page](https://builtonenvoy.io/extensions/dns-gateway) for the full key reference.
78+
See the [extension page](https://builtonenvoy.io/extensions/lookup) for the full key reference.
6279

6380
## Domain matching
6481

@@ -67,7 +84,7 @@ See the [extension page](https://builtonenvoy.io/extensions/dns-gateway) for the
6784

6885
## Configuration
6986

70-
### UDP listener filter
87+
### `resolver` (UDP listener filter)
7188

7289
| Field | Type | Description |
7390
| ----------------------- | ------- | ------------------------------------------------------------------ |
@@ -78,7 +95,7 @@ See the [extension page](https://builtonenvoy.io/extensions/dns-gateway) for the
7895
| `domains[].metadata` | object | String key-value pairs exposed via filter state |
7996
| `fail_open` | boolean | If `true`, forward queries upstream when a CIDR range is exhausted. Default: `false` (return NODATA) |
8097

81-
### Network filter
98+
### `lookup` (network filter)
8299

83100
| Field | Type | Description |
84101
| ---------------------- | ------ | ------------------------------------------------------------------------------------ |

‎extensions/dns-gateway/config.schema.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"$schema": "https://json-schema.org/draft/2020-12/schema",
33
"$id": "https://github.com/tetratelabs/built-on-envoy/extensions/dns-gateway/config.schema.json",
44
"title": "DNS Gateway Configuration",
5-
"description": "Configuration for the dns-gateway extension. Pass with --filter-type udp_listener for the dns_gateway filter, or --filter-type network for the cache_lookup filter.",
5+
"description": "Configuration for the legacy combined dns-gateway extension. Pass with --filter-type udp_listener for the resolver filter (see dns-gateway/resolver), or --filter-type network for the lookup filter (see dns-gateway/lookup).",
66
"$defs": {
77
"DnsGatewayConfig": {
88
"type": "object",
@@ -36,7 +36,7 @@
3636
},
3737
"metadata": {
3838
"type": "object",
39-
"description": "String key-value pairs exposed in Envoy filter state as io.builtonenvoy.dns_gateway.metadata.<key>.",
39+
"description": "String key-value pairs exposed in Envoy filter state as io.builtonenvoy.dns_gateway.metadata.\\<key\\>.",
4040
"additionalProperties": {
4141
"type": "string"
4242
},
@@ -58,7 +58,7 @@
5858
"properties": {
5959
"filter_state_prefix": {
6060
"type": "string",
61-
"description": "Prefix for filter state keys written by the cache_lookup filter. Downstream filters read domain and metadata under this prefix. Default: \"io.builtonenvoy.dns_gateway\".",
61+
"description": "Prefix for filter state keys written by the lookup filter. Downstream filters read domain and metadata under this prefix. Default: \"io.builtonenvoy.dns_gateway\".",
6262
"default": "io.builtonenvoy.dns_gateway"
6363
}
6464
}

‎extensions/dns-gateway/manifest.yaml‎

Lines changed: 19 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ author: Palantir Technologies
1313
description: Transparent egress routing — intercept DNS and route TCP by domain
1414
longDescription: |
1515
Transparent egress routing for Envoy. Applications make normal DNS lookups and TCP
16-
connections — this extension intercepts both so Envoy can control which upstream cluster
16+
connections — this extension set intercepts both so Envoy can control which upstream cluster
1717
handles each domain, attach metadata (auth tokens, policy IDs), and enforce routing
1818
policy, all without any application changes.
1919
@@ -25,54 +25,23 @@ longDescription: |
2525
2626
![img](/extensions/dns-gateway.png)
2727
28-
## How it works
28+
## Extensions
2929
30-
The extension is a pair of filters that work together. Both are provided by the same
31-
`dns-gateway` dynamic module; enable each by its filter type (`udp_listener` and `network`).
30+
The DNS Gateway is a pair of filters that work together, both provided by the same
31+
`dns-gateway` dynamic module and sharing a process-wide virtual-IP cache:
3232
33-
1. **UDP listener filter** — Intercepts DNS queries for configured domain
34-
patterns. For each matched query, allocates a virtual IP from a dedicated CIDR range and
35-
responds with an A record. The application receives this virtual IP and connects to it
36-
as if it were the real destination. Non-matching queries pass through to upstream resolvers.
33+
* **[resolver](resolver)** — a UDP listener filter that intercepts DNS
34+
queries for configured domain patterns, allocates a virtual IP from a dedicated CIDR range,
35+
and responds with an A record.
36+
* **[lookup](lookup)** — a network filter that, when the application
37+
opens a TCP connection to a virtual IP, looks up the original domain and its configured
38+
metadata from the shared cache and exposes them as Envoy filter state.
3739
38-
2. **Network filter** — When the application opens a TCP connection to a
39-
virtual IP, this filter looks up the original domain and its configured metadata from a
40-
shared cache and sets them as Envoy
41-
[filter state](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/data_sharing_between_filters#primitives).
42-
Downstream filters (e.g. `tcp_proxy`, `set_filter_state`) can then use this to select
43-
the upstream cluster, set headers, or apply policy.
40+
## Backward compatibility
4441
45-
## Configuration
46-
47-
### UDP listener filter
48-
49-
| Field | Type | Required | Description |
50-
|-------|------|----------|-------------|
51-
| `domains` | array | yes | List of domain patterns to intercept |
52-
| `domains[].domain` | string | yes | Exact (`example.com`) or single-level wildcard (`*.aws.com`) |
53-
| `domains[].base_ip` | string | yes | Base IPv4 address of the virtual IP CIDR range (e.g. `10.0.0.0`) |
54-
| `domains[].prefix_len` | integer | yes | CIDR prefix length (1–32). A /24 gives 256 virtual IPs |
55-
| `domains[].metadata` | object | no | String key-value pairs propagated to filter state |
56-
| `fail_open` | boolean | no | If `true`, forward queries upstream when a CIDR range is exhausted. Default: `false` (return NODATA) |
57-
58-
### Network filter
59-
60-
| Field | Type | Required | Description |
61-
|-------|------|----------|-------------|
62-
| `filter_state_prefix` | string | no | Prefix for filter state keys. Default: `io.builtonenvoy.dns_gateway` |
63-
64-
## Filter State
65-
66-
The network filter writes the following keys to Envoy
67-
[filter state](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/data_sharing_between_filters#primitives),
68-
readable by downstream filters via `%FILTER_STATE(<key>:PLAIN)%`:
69-
70-
| Key | Example | Description |
71-
|-----|---------|-------------|
72-
| `<prefix>.domain` | `bucket-1.aws.com` | The matched domain from the DNS query |
73-
| `<prefix>.metadata.<key>` | `aws_cluster` | Per-domain metadata values from config |
74-
75-
The default prefix is `io.builtonenvoy.dns_gateway`.
42+
The combined `dns-gateway` extension is preserved: the legacy invocation
43+
`--extension dns-gateway --filter-type udp_listener` (and `--filter-type network`) still works
44+
and resolves to the same filters as the split extensions above.
7645
7746
## Prerequisites
7847
@@ -91,61 +60,11 @@ tags:
9160
- virtual-ip
9261
license: Apache-2.0
9362
type: rust
94-
featured: true
63+
extensionSet: true
64+
# Retained for backward compatibility: the legacy combined `dns-gateway` extension is still
65+
# directly runnable via `--extension dns-gateway --filter-type <udp_listener|network>`. The set
66+
# is excluded from the catalog (extensionSet), which surfaces the split children instead.
9567
filterType:
9668
- udp_listener
9769
- network
98-
examples:
99-
- title: Single Wildcard Domain
100-
description: |
101-
Intercept DNS queries for `*.aws.com` and route TCP connections to virtual IPs via the
102-
shared cache. Both filter types are required: specify the extension twice, once per
103-
filter type. The `udp_listener` filter carries the domain config; the `network` filter
104-
requires no configuration.
105-
code: |
106-
boe run \
107-
--extension dns-gateway --filter-type udp_listener --config '
108-
{
109-
"domains": [
110-
{
111-
"domain": "*.aws.com",
112-
"base_ip": "10.0.0.0",
113-
"prefix_len": 24,
114-
"metadata": {
115-
"cluster": "aws_cluster"
116-
}
117-
}
118-
]
119-
}' \
120-
--extension dns-gateway --filter-type network
121-
- title: Multiple Domains with Routing Metadata
122-
description: |
123-
Route different domain patterns to different upstream clusters using per-domain metadata.
124-
Downstream filters can read the domain and metadata from Envoy filter state, e.g.
125-
`%FILTER_STATE(io.builtonenvoy.dns_gateway.domain:PLAIN)%` in a `tcp_proxy` route.
126-
code: |
127-
boe run \
128-
--extension dns-gateway --filter-type udp_listener --config '
129-
{
130-
"domains": [
131-
{
132-
"domain": "*.aws.com",
133-
"base_ip": "10.0.0.0",
134-
"prefix_len": 24,
135-
"metadata": {
136-
"cluster": "aws_cluster",
137-
"token": "abc123"
138-
}
139-
},
140-
{
141-
"domain": "example.com",
142-
"base_ip": "10.0.1.0",
143-
"prefix_len": 24,
144-
"metadata": {
145-
"cluster": "example_cluster",
146-
"token": "def456"
147-
}
148-
}
149-
]
150-
}' \
151-
--extension dns-gateway --filter-type network
70+
examples: []

‎extensions/dns-gateway/manifests/lookup/config.schema.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"$id": "https://github.com/tetratelabs/built-on-envoy/extensions/dns-gateway/manifests/lookup/config.schema.json",
4+
"title": "DNS Gateway Lookup Configuration",
5+
"description": "Configuration for the dns-gateway lookup network filter, the virtual-IP resolution half of the dns-gateway extension set.",
6+
"type": "object",
7+
"additionalProperties": false,
8+
"properties": {
9+
"filter_state_prefix": {
10+
"type": "string",
11+
"description": "Prefix for filter state keys written by the dns-gateway lookup filter. Downstream filters read domain and metadata under this prefix. Default: \"io.builtonenvoy.dns_gateway\".",
12+
"default": "io.builtonenvoy.dns_gateway"
13+
}
14+
}
15+
}

0 commit comments

Comments
 (0)