Skip to content

Conversation

SkyeYoung
Copy link
Member

@SkyeYoung SkyeYoung commented Aug 27, 2025

Description

In the past, APISIX trusted the X-Forwarded-* headers from all IPs, which caused many problems. For example:

When using the redirect plugin, even if http_to_https is enabled, as long as the user tries to request curl http://route -H "X-Forwarded-Proto: https" (or uses any value other than http. Allow any value will be fixed in #12561), they can bypass the redirection logic and downgrade to http.

local proxy_proto = core.request.header(ctx, "X-Forwarded-Proto")
local _scheme = proxy_proto or core.request.get_scheme(ctx)
if conf.http_to_https and _scheme == "http" then
if ret_port == nil or ret_port == 443 or ret_port <= 0 or ret_port > 65535 then

This PR implements that X-Forwarded-* headers are only trusted if they are sent by IPs or CIDRs configured in apisix.trusted_addresses. Therefore, all plugins or features that use X-Forwarded-* headers require configuration of apisix.trusted_addresses to take effect.

apisix.trusted_addresses supports matching both IPs and CIDRs.

Solve part of #12500

Which issue(s) this PR fixes:

Fixes #

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Comment on lines -293 to -309

local proto = api_ctx.var.http_x_forwarded_proto
if proto then
api_ctx.var.var_x_forwarded_proto = proto
end

local x_forwarded_host = api_ctx.var.http_x_forwarded_host
if x_forwarded_host then
api_ctx.var.var_x_forwarded_host = x_forwarded_host
end

local port = api_ctx.var.http_x_forwarded_port
if port then
api_ctx.var.var_x_forwarded_port = port
end
Copy link
Member Author

@SkyeYoung SkyeYoung Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apisix.handle_upstream = orig_handle_upstream
apisix.http_balancer_phase = orig_http_balancer_phase
else
-- replace the upstream and balancer module
apisix.handle_upstream = ai_upstream
apisix.http_balancer_phase = ai_http_balancer_phase

Because the code above will replace handle_upstream with ai_upstream below:

local function ai_upstream()
core.log.info("enable sample upstream")
end

This will cause this deleted part of the code to be skipped.

Now that we need to handle X-Forwarded-*, we need to update these var_x_forwarded_*, so we need to extract this part of the code as update_var_x_forwarded_headers.

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $var_x_forwarded_proto;
proxy_set_header X-Forwarded-Host $var_x_forwarded_host;
proxy_set_header X-Forwarded-Port $var_x_forwarded_port;

@SkyeYoung SkyeYoung changed the title feat: use trusted_addresses to handle X-Forwarded-* headers feat: only trust X-Forwarded-* headers from trusted_addresses Aug 29, 2025
@SkyeYoung SkyeYoung changed the title feat: only trust X-Forwarded-* headers from trusted_addresses fix: only trust X-Forwarded-* headers from trusted_addresses Aug 29, 2025
@SkyeYoung SkyeYoung marked this pull request as ready for review August 29, 2025 08:43
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Aug 29, 2025
membphis
membphis previously approved these changes Sep 1, 2025
@moonming moonming requested a review from Copilot September 2, 2025 01:22
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements security hardening by restricting APISIX to only trust X-Forwarded-* headers from configured trusted IP addresses/CIDRs. Previously, APISIX accepted these headers from any source, creating security vulnerabilities where malicious clients could bypass features like HTTP-to-HTTPS redirection.

  • Adds trusted_addresses configuration to specify which IPs/CIDRs can send trusted X-Forwarded-* headers
  • Creates utility module to validate and match trusted addresses using IP/CIDR notation
  • Modifies request processing to override untrusted X-Forwarded-* headers with server-generated values

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
apisix/utils/trusted-addresses.lua New utility module for validating and matching trusted IP addresses/CIDRs
apisix/init.lua Core logic to handle X-Forwarded headers based on client trust status
conf/config.yaml.example Configuration example for trusted_addresses setting
t/core/trusted-addresses.t Comprehensive tests for trusted address functionality
docs/en/latest/plugins/*.md Documentation updates explaining trusted address requirements
docs/zh/latest/plugins/*.md Chinese documentation updates for trusted address requirements
t/APISIX.pm Default test configuration with localhost as trusted address
t/plugin/*.t Test configuration updates to include trusted addresses

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

node_listen: 1984
enable_admin: false
trusted_addresses:
- "127.0.0.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change this test case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in batches, removed

my $yaml_config = $block->yaml_config // <<_EOC_;
apisix:
node_listen: 1984
trusted_addresses:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enable_encrypt_fields: true
keyring:
- edd1c9f0985e76a2
trusted_addresses:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in batches, removed

@@ -0,0 +1,219 @@
#
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to test the situation where trusted-addresses contain multiple IP addresses

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and need test cases in plugins.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member Author

@SkyeYoung SkyeYoung Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than chaitin-waf, the original PR didn't find any tests covering the real_client_ip=true scenario, and after spending time researching, I also couldn't find the actual purpose of this parameter.

# When set to true, it overrides all upstream healthcheck configurations and globally disabling healthchecks.
# trusted_addresses: # When enabled, APISIX will trust the `X-Forwarded-*` Headers
# - 127.0.0.1 # passed in requests from the IP/CIDR in the list.
# - 0.0.0.0/0 # CAUTION: When not configured, APISIX will remove `X-Forwarded-*` headers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No IP restriction is a bad example

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

# or (standalone mode) the config isn't loaded yet either via file or Admin API.
# disable_upstream_healthcheck: false # A global switch for healthcheck. Defaults to false.
# When set to true, it overrides all upstream healthcheck configurations and globally disabling healthchecks.
# trusted_addresses: # When enabled, APISIX will trust the `X-Forwarded-*` Headers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# trusted_addresses: # When enabled, APISIX will trust the `X-Forwarded-*` Headers
# trusted_addresses: # When enabled, APISIX will trust the `X-Forwarded-*` Headers

how to disabled?

Copy link
Member Author

@SkyeYoung SkyeYoung Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not configured, it is disabled by default.

This statement references

# status: # When enabled, APISIX will provide `/status` and `/status/ready` endpoints

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

write it in this file and docs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

default = false,
description = "a global switch to disable upstream health checks",
},
trusted_addresses = {
Copy link
Member

@nic-6443 nic-6443 Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref:

trusted_addresses = {
type = "array",
items = {anyOf = core.schema.ip_def},
minItems = 1
},

we can add more restrction to schema like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding core.schema.ip_def doesn't seem to work.

https://github.com/apache/apisix/blob/8bb6802bd2b2ccd4dabe78e90beb8f87ec046035/apisix/schema_def.lua depends on https://github.com/apache/apisix/blob/0fd582bc299a9d90893d3843cd3ec21f217cb40e/apisix/core/schema.lua

Using it will cause an error /usr/local/openresty//luajit/bin/luajit: ./apisix/core/lrucache.lua:22: module '\''resty.lrucache'\'' not found:

image image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using it will cause an error /usr/local/openresty//luajit/bin/luajit: ./apisix/core/lrucache.lua:22: module '''resty.lrucache''' not found:

What is the reason for this error?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this error is that there is no openresty environment in the apisix cli, which makes it impossible to use the resty.lru library.
I have considered the dependency relationship between schema_def.lua and schema.lua, and I think their dependency relationship is reversed. It should be that schema.lua depends on schema_def.lua, allowing schema_def.lua to retain only the definition of schema, in line with the naming of this file.
Added a new commit to this PR to fix this dependency issue: 30ce9c2

nic-6443
nic-6443 previously approved these changes Sep 15, 2025
nic-6443
nic-6443 previously approved these changes Sep 15, 2025
--- request
GET /old_uri
--- error_log
invalid IP/CIDR '1.0.0' exists in trusted_addresses
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we also need to check the value of trusted_addresses when apisix starts, not only at runtime.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

type = "array",
minItems = 1,
items = {
type = "string",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"oneOf": [
{"format": "ipv4"},
{"format": "ipv6"}
]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few problems with doing this:

  1. Need CIDR support
  2. As discussed in fix: only trust X-Forwarded-* headers from trusted_addresses #12551 (comment), try to use the existing schema_def. A feasible solution is to copy the corresponding schema again.

Copy link
Member Author

@SkyeYoung SkyeYoung Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, I will try to copy a schema from schema_def to implement this function.

apisix/init.lua Outdated
api_ctx.var.http_x_forwarded_port = port
api_ctx.var.http_x_forwarded_for = nil

-- override the x-forwarded-* headers to the trusted ones
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the if not trusted logic of this code, where the trusted ones come from?

Copy link
Member Author

@SkyeYoung SkyeYoung Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set $var_x_forwarded_proto $scheme;
set $var_x_forwarded_host $host;
set $var_x_forwarded_port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $var_x_forwarded_proto;
proxy_set_header X-Forwarded-Host $var_x_forwarded_host;
proxy_set_header X-Forwarded-Port $var_x_forwarded_port;

Referenced the implementation of APISIX.

Comment on lines +621 to +624
local proto = api_ctx.var.http_x_forwarded_proto
if proto then
api_ctx.var.var_x_forwarded_proto = proto
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not set X-Forwarded-Proto as http_x_forwarded_proto?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand.

  1. This function is extracted from the following code without any adjustments.

https://github.com/apache/apisix/pull/12551/files/528116c496c9c5ea3ef2b3030e1da6596bac67cb#diff-7439ffe94cc4bcad12066d5d83f6c1004b8e9b3ff5097b645fa7c700eb3b3743L293-L307

  1. X-Forwarded-Proto in the header == http_x_forwarded_proto

local _M = {}


local function validate_trusted_addresses(trusted_addresses)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we also need to check this when apisix starts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

end


local function handle_x_forwarded_headers(api_ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some comments to explain the purpose of each code segment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

required = {"prefix", "host"}
}

-- copy from apisix/schema_def.lua
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please do not copy. we should find the root reason

Copy link
Member Author

@SkyeYoung SkyeYoung Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My suggestion is to extract this schema separately.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part has been handled by @nic-6443 . Thanks to nic for the assistance.

@SkyeYoung SkyeYoung requested a review from bzp2010 September 18, 2025 01:11
@SkyeYoung SkyeYoung merged commit 5f0ff46 into apache:master Sep 19, 2025
93 of 102 checks passed
@SkyeYoung SkyeYoung deleted the young/feat/trusted_addresses branch September 19, 2025 01:30
jizhuozhi pushed a commit to jizhuozhi/apisix that referenced this pull request Oct 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants