Skip to content

Commit 6c2d921

Browse files
author
Ted Avery
committed
OSDOCS-13371-2:Ingress Controller Parameters
1 parent 41cee05 commit 6c2d921

3 files changed

+202
-46
lines changed

modules/microshift-ingress-controller-conc.adoc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,6 @@ With more configuration options, you can fine-tune ingress to meet your specific
1313
* If your application starts processing requests from clients but the connection is
1414
closed before it can respond, you can set the `ingress.tuningOptions.serverTimeout` parameter in the configuration file to a higher value to accommodate the speed of the response from the server.
1515
16-
* If the router has many connections open because an application running on the cluster does not close connections properly, you can set the `ingress.tuningOptions.serverTimeout` and `spec.tuningOptions.serverFinTimeout` parameters to a lower value, forcing those connections to close sooner.
16+
* If the router has many connections open because an application running on the cluster does not close connections properly, you can set the `ingress.tuningOptions.serverTimeout` and `spec.tuningOptions.serverFinTimeout` parameters to a lower value, forcing those connections to close sooner.
17+
18+
You use a TLS (Transport Layer Security) security profile to define which TLS ciphers are required by various {product-title} components. The {microshift-short} TLS security profiles are based on link:https://wiki.mozilla.org/Security/Server_Side_TLS[Mozilla recommended configurations]. The `tlsSecurityProfile` parameter in the configuration file sets the minimum TLS version and specifies the TLS ciphers settings for TLS connections for Ingress Controllers.

modules/microshift-ingress-controller-config.adoc

Lines changed: 130 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -33,27 +33,42 @@ You can use detailed ingress control settings by updating the {microshift-short}
3333
apiServer:
3434
# ...
3535
ingress:
36-
defaultHTTPVersion: 1
37-
forwardedHeaderPolicy: Append
38-
httpCompression:
39-
mimeTypes:
40-
- ""
41-
httpEmptyRequestsPolicy: Respond
42-
# ...
43-
logEmptyRequests: Log
44-
# ...
45-
tuningOptions:
46-
clientFinTimeout: 1s
47-
clientTimeout: 30s
48-
headerBufferBytes: 0
49-
headerBufferMaxRewriteBytes: 0
50-
healthCheckInterval: 5s
51-
maxConnections: 0
52-
serverFinTimeout: 1s
53-
serverTimeout: 30s
54-
threadCount: 4
55-
tlsInspectDelay: 5s
56-
tunnelTimeout: 1h
36+
certificateSecret: router-certs-custom
37+
clientTLS:
38+
allowedSubjectPatterns: ^/CN=example.com/ST=NC/C=US/O=Security/OU=OpenShift$
39+
clientCA: ca-config-map
40+
clientCertificatePolicy: Required
41+
defaultHTTPVersion: 1
42+
forwardedHeaderPolicy: Append
43+
httpCompression:
44+
mimeTypes:
45+
- ""
46+
httpEmptyRequestsPolicy: Respond
47+
logEmptyRequests: Log
48+
routeAdmission:
49+
namespaceOwnership: InterNamespaceAllowed
50+
wildcardPolicy: WildcardPolicyAllowed
51+
tlsSecurityProfile:
52+
type: Custom
53+
custom:
54+
ciphers:
55+
- ECDHE-ECDSA-CHACHA20-POLY1305
56+
- ECDHE-RSA-CHACHA20-POLY1305
57+
- ECDHE-RSA-AES128-GCM-SHA256
58+
- ECDHE-ECDSA-AES128-GCM-SHA256
59+
minTLSVersion: VersionTLS12
60+
tuningOptions:
61+
clientFinTimeout: 1s
62+
clientTimeout: 30s
63+
headerBufferBytes: 0
64+
headerBufferMaxRewriteBytes: 0
65+
healthCheckInterval: 5s
66+
maxConnections: 0
67+
serverFinTimeout: 1s
68+
serverTimeout: 30s
69+
threadCount: 4
70+
tlsInspectDelay: 5s
71+
tunnelTimeout: 1h
5772
# ...
5873
----
5974
+
@@ -62,6 +77,28 @@ ingress:
6277
|===
6378
|Parameter |Description
6479

80+
|`certificateSecret`
81+
|The `certificateSecret` value is a reference to a secret that contains the default certificate that is served by the Ingress Controller. When Routes do not specify their own certificate, `certificateSecret` is used.
82+
83+
The secret must contain the following keys and data:
84+
* `tls.crt`: certificate file contents
85+
* `tls.key`: key file contents
86+
87+
If not set, a wildcard certificate is automatically generated and used. The certificate is valid for the Ingress Controller `domain` and `subdomains`, and
88+
the generated certificate's CA is automatically integrated with the
89+
cluster's truststore.
90+
91+
The in-use certificate, whether generated or user-specified, is automatically integrated with {product-title} built-in OAuth server.
92+
93+
|`clientTLS`
94+
|`clientTLS` authenticates client access to the cluster and services; as a result, mutual TLS authentication is enabled. If not set, then client TLS is not enabled.
95+
96+
`clientTLS` has the required subfields, `spec.clientTLS.clientCertificatePolicy` and `spec.clientTLS.ClientCA`.
97+
98+
The `ClientCertificatePolicy` subfield accepts one of the two values: `Required` or `Optional`. Note that the ingress controller only checks client certificates for edge-terminated and reencrypt TLS routes; it cannot check certificates for cleartext HTTP or passthrough TLS routes. The `ClientCA` subfield specifies a config map that is in the openshift-ingress namespace. The config map should contain a CA certificate bundle. A config map is required for this field.
99+
100+
The `AllowedSubjectPatterns` is an optional value that specifies a list of regular expressions, which are matched against the distinguished name on a valid client certificate to filter requests. The regular expressions must use PCRE syntax. This field must contain a valid expression or the MicroShift service will fail. At least one pattern must match a client certificate's distinguished name; otherwise, the Ingress Controller rejects the certificate and denies the connection. If not specified, the Ingress Controller does not reject certificates based on the distinguished name.
101+
65102
|`defaultHTTPVersion`
66103
|Sets the HTTP version for the ingress controller. Default value is `1` for HTTP 1.1.
67104
//Q: do we need to configure a load balancer for 2 and 3?
@@ -75,64 +112,112 @@ ingress:
75112
* `Never`, preserves any existing headers by specifying that the ingress controller never sets the headers.
76113
77114
|`httpCompression`
78-
|Defines the policy for HTTP traffic compression.
115+
|`httpCompression` defines the policy for HTTP traffic compression.
79116

80-
* `httpCompression.mimeTypes` defines a list of or a single MIME type to which compression is applied. For example, `text/css; charset=utf-8`, `text/html`, `text/*`, `image/svg+xml`, `application/octet-stream`, `X-custom/customsub`, using the format pattern, `type/subtype; [;attribute=value]`. The `types` are: application, image, message, multipart, text, video, or a custom type prefaced by `X-`. To see the full notation for MIME types and subtypes, see link:https://datatracker.ietf.org/doc/html/rfc1341#page-7[RFC1341] (IETF Datatracker documentation).
117+
* `mimeTypes` defines a list of MIME types to which compression should be applied. For example, `text/css; charset=utf-8`, `text/html`, `text/*`, `image/svg+xml`, `application/octet-stream`, `X-custom/customsub`, using the format pattern, `type/subtype; [;attribute=value]`. The `types` are: application, image, message, multipart, text, video, or a custom type prefaced by `X-`. To see the full notation for MIME types and subtypes, see link:https://datatracker.ietf.org/doc/html/rfc1341#page-7[RFC1341]
81118
82119
|`httpEmptyRequestsPolicy`
83-
|Describes how HTTP connections are handled if the connection times out before a request is received. Allowed values for this field are `Respond` and `Ignore`. The default value is `Respond`. The following are valid values:
120+
|`httpEmptyRequestsPolicy` describes how HTTP connections are handled if the connection times out before a request is received. Allowed values for this field are `Respond` and `Ignore`. The default value is `Respond`.
121+
122+
The `httpEmptyRequestsPolicy` type accepts either one of two values:
123+
124+
* `Respond`:If the field is set to `Respond`, the Ingress Controller sends an HTTP `400` or `408` response, logs the connection if access logging is enabled, and counts the connection in the appropriate metrics.
84125
85-
* `Respond`, causes the ingress controller to send an HTTP `400` or `408` response, logs the connection if access logging is enabled, and counts the connection in the appropriate metrics.
86-
* `Ignore`, adds the `http-ignore-probes` parameter in the `HAproxy` configuration. If the field is set to `Ignore`, the ingress controller closes the connection without sending a response, then logs the connection or incrementing metrics.
126+
* `Ignore`: Setting the option to `Ignore` adds the `http-ignore-probes` parameter in the HAproxy configuration. If the field is set to `Ignore`, the Ingress Controller closes the connection without sending a response, then logs the connection, or incrementing metrics.
87127
88-
Usually, empty request connections come from load balancer health probes or web browser preconnects and can be safely ignored. However, network errors and port scans can also create these empty requests, so setting this field to `Ignore` can impede detecting or diagnosing problems and also impede the detection of intrusion attempts.
128+
These connections come from load balancer health probes or web browser speculative connections (preconnect) and can be safely ignored. However, these requests can be caused by network errors, so setting this field to `Ignore` can impede detection or diagnosis of problems. These requests can be caused by port scans, in which case logging empty requests can aid in detecting intrusion attempts.
89129

90130
|`logEmptyRequests`
91-
|Specifies connections for which no request is received and logged. Usually, these empty requests come from load balancer health probes or web browser speculative connections such as preconnects. Logging these types of empty requests can be undesirable. However, network errors and port scans can also create empty requests, so setting this field to `Ignore` can impede detecting or diagnosing problems and also impede the detection of intrusion attempts.
131+
|`logEmptyRequests` specifies connections for which no request is received and logged. These empty requests come from load balancer health probes or web browser speculative connections (preconnect) and logging these requests can be undesirable. However, these requests can be caused by network errors, in which case logging empty requests can be useful for diagnosing the errors. These requests can be caused by port scans, and logging empty requests can aid in detecting intrusion attempts. Allowed values for this field are `Log` and `Ignore`. The default is `Log`.
132+
133+
The `LoggingPolicy` type accepts either one of two values:
134+
135+
* `Log`: Setting this value to `Log` indicates that an event should be logged.
136+
* `Ignore`: Setting this value to `Ignore` sets the `dontlognull` option in the HAproxy configuration.
137+
138+
|`routeAdmission`
139+
|`routeAdmission` defines a policy for handling new route claims, such as allowing or denying claims across namespaces.
140+
141+
`namespaceOwnership` describes how hostname claims across namespaces should be handled. The default is `InterNamespaceAllowed`.
142+
143+
* `Strict`: does not allow routes to claim the same hostname across namespaces.
144+
* `InterNamespaceAllowed`: allows routes to claim different paths of the same hostname across namespaces.
145+
146+
`wildcardPolicy` describes how routes with wildcard policies are handled by the Ingress Controller.
92147

93-
The following are valid values:
148+
* `WildcardsDisallowed`: Indicates only routes with a wildcard policy of `None` are admitted by the Ingress Controller. Updating `wildcardPolicy` from `WildcardsAllowed` to `WildcardsDisallowed` causes admitted routes with a wildcard policy of `Subdomain` to stop working. These routes must be recreated to a wildcard policy of `None` to be readmitted by the Ingress Controller. `WildcardsDisallowed` is the default setting.
94149
95-
* `Log`, which indicates that an event should be logged.
96-
* `Ignore`, which sets the `dontlognull` option in the `HAproxy` configuration.
150+
|`tlsSecurityProfile`
151+
|`tlsSecurityProfile` specifies settings for TLS connections for Ingress Controllers.
152+
153+
If not set, the default value is based on the `apiservers.config.openshift.io/cluster` resource.
154+
155+
`Intermediate` is the default setting.
156+
157+
The minimum TLS version for Ingress Controllers is `1.1`, and the maximum TLS version is `1.3`.
158+
159+
[NOTE]
160+
====
161+
Ciphers and the minimum TLS version of the configured security profile are reflected in the `TLSProfile` status.
162+
====
163+
164+
[IMPORTANT]
165+
====
166+
The Ingress Operator converts the TLS `1.0` of an `Old` or `Custom` profile to `1.1`.
167+
====
97168

98169
|`tuningOptions`
99-
|Specifies options for tuning the performance of ingress controller pods.
170+
|`tuningOptions` specifies options for tuning the performance of Ingress Controller pods.
100171

101-
* The `tuningOptions.clientFinTimeout` parameter specifies how long a connection is held open while waiting for the client response to the server closing the connection. The default timeout is `1s`.
172+
* `clientFinTimeout` specifies how long a connection is held open while waiting for the client response to the server closing the connection. The default timeout is `1s`.
102173
103-
* The `tuningOptions.clientTimeout` parameter specifies how long a connection is held open while waiting for a client response. The default timeout is `30s`.
174+
* `clientTimeout` specifies how long a connection is held open while waiting for a client response. The default timeout is `30s`.
104175
105-
* The `tuningOptions.headerBufferBytes` parameter specifies how much memory is reserved, in bytes, for Ingress Controller connection sessions. This value must be at least `16384` if HTTP/2 is enabled for the Ingress Controller. If not set, the default value is `32768` bytes.
176+
* `headerBufferBytes` specifies how much memory is reserved, in bytes, for Ingress Controller connection sessions. This value must be at least `16384` if HTTP/2 is enabled for the Ingress Controller. If not set, the default value is `32768` bytes.
106177
+
107178
[IMPORTANT]
108179
====
109180
Setting this field not recommended because `headerBufferMaxRewriteBytes` parameter values that are too small can break the ingress controller. Conversely, values for `headerBufferMaxRewriteBytes` that are too large could cause the ingress controller to use significantly more memory than necessary.
110181
====
111182
112-
* The `tuningOptions.headerBufferMaxRewriteBytes` parameter specifies how much memory should be reserved, in bytes, from `headerBufferBytes` for HTTP header rewriting and appending for Ingress Controller connection sessions. The minimum value for `headerBufferMaxRewriteBytes` is `4096`. The `headerBufferBytes` value must be greater than the `headerBufferMaxRewriteBytes` value for incoming HTTP requests.
113-
* If not set, the default value is `8192` bytes.
183+
* `headerBufferMaxRewriteBytes` specifies how much memory should be reserved, in bytes, from `headerBufferBytes` for HTTP header rewriting and appending for Ingress Controller connection sessions. The minimum value for `headerBufferMaxRewriteBytes` is `4096`. `headerBufferBytes` must be greater than the `headerBufferMaxRewriteBytes` value for incoming HTTP requests. If not set, the default value is `8192` bytes.
114184
+
115185
[IMPORTANT]
116186
====
117-
Setting this field is not recommended because `headerBufferMaxRewriteBytes` parameter values that are too small can break the ingress controller. Conversely, values for `headerBufferMaxRewriteBytes` that are too large could cause the ingress controller to use significantly more memory than necessary.
187+
Setting this field is not recommended because `headerBufferMaxRewriteBytes` values that are too small can break the Ingress Controller and`headerBufferMaxRewriteBytes` that are too large could cause the Ingress Controller to use significantly more memory than necessary.
118188
====
119189
120-
* The `tuningOptions.healthCheckInterval` parameter specifies how long the router waits between health checks. The default is `5s`.
190+
* `healthCheckInterval` specifies how long the router waits between health checks. The default is `5s`.
121191
122-
* The `tuningOptions.serverFinTimeout` parameter specifies how long a connection is held open while waiting for the server response to the client that is closing the connection. The default timeout is `1s`.
192+
* `serverFinTimeout` specifies how long a connection is held open while waiting for the server response to the client that is closing the connection. The default timeout is `1s`.
123193
124-
* The `tuningOptions.serverTimeout` parameter specifies how long a connection is held open while waiting for a server response. The default timeout is `30s`.
194+
* `serverTimeout` specifies how long a connection is held open while waiting for a server response. The default timeout is `30s`.
125195
126-
* The `tuningOptions.threadCount` parameter specifies the number of threads to create per HAProxy process. Creating more threads allows each ingress controller pod to handle more connections, at the cost of more system resources being used. `HAProxy` supports up to `64` threads. If this field is empty, default value is `4` threads.
196+
* `threadCount` specifies the number of threads to create per HAProxy process. Creating more threads allows each Ingress Controller pod to handle more connections, at the cost of more system resources being used. `HAProxy` supports up to `64` threads. If this field is empty, the Ingress Controller uses the default value of `4` threads. The default value can change in future releases.
127197
+
128198
[IMPORTANT]
129199
====
130-
Setting this field is not recommended because increasing the number of `HAProxy` threads allows ingress controller pods to use more CPU time under load, and prevent other pods from receiving the CPU resources they need to perform.
200+
Setting this field is not recommended because increasing the number of `HAProxy` threads allows Ingress Controller pods to use more CPU time under load, and prevent other pods from receiving the CPU resources they need to perform. Reducing the number of threads can cause the Ingress Controller to perform poorly.
131201
====
132202
133-
* The `tuningOptions.tlsInspectDelay` parameter specifies how long the router can hold data to find a matching route. Setting this value too short can cause the router to fall back to the default certificate for edge-terminated, re-encrypted, or passthrough routes, even when using a better-matched certificate. The default inspect delay is `5s`.
203+
* `tlsInspectDelay` specifies how long the router can hold data to find a matching route. Setting this value too short can cause the router to fall back to the default certificate for edge-terminated, re-encrypted, or passthrough routes, even when using a better-matched certificate. The default inspect delay is `5s`.
204+
205+
* `tunnelTimeout` specifies how long a tunnel connectiotabl, including websockets, remains open while the tunnel is idle. The default timeout is `1h`.
206+
207+
* `maxConnections` specifies the maximum number of simultaneous connections that can be established per HAProxy process. Increasing this value allows each ingress controller pod to handle more connections at the cost of additional system resources. Permitted values are `0`, `-1`, any value within the range `2000` and `2000000`, or the field can be left empty.
208+
209+
** If this field is left empty or has the value `0`, the Ingress Controller will use the default value of `50000`. This value is subject to change in future releases.
210+
211+
** If the field has the value of `-1`, then HAProxy will dynamically compute a maximum value based on the available `ulimits` in the running container. This process results in a large computed value that will incur significant memory usage compared to the current default value of `50000`.
212+
213+
** If the field has a value that is greater than the current operating system limit, the HAProxy process will not start.
214+
215+
** If you choose a discrete value and the router pod is migrated to a new node, it is possible the new node does not have an identical `ulimit` configured. In such cases, the pod fails to start.
216+
217+
** If you have nodes with different `ulimits` configured, and you choose a discrete value, it is recommended to use the value of `-1` for this field so that the maximum number of connections is calculated at runtime.
218+
219+
**
134220

135-
* The `tuningOptions.tunnelTimeout` parameter specifies how long a tunnel connection, including websockets, remains open while the tunnel is idle. The default timeout is `1h`.
136221
|===
137222

138223
. Complete any other configurations you require, then start or restart {microshift-short} by running one the following commands:

0 commit comments

Comments
 (0)