[BUG] Potential regression in cross cluster searches from 3.7 -> 2.19

[zakerf]

What is the bug?
Potential regression in cross cluster searches from OpenSearch 3.7 -> OpenSearch 2.19. Same security configuration allows for successful cross cluster search on OpenSearch 2.19 -> OpenSearch 2.19 setup. It's not clear to me whether this is an intended change and the security setup I had was always wrong or a regression. I wasn't able to find anything in release notes pointing at a change here

How can one reproduce the bug?

• Reproduction can be found in this repo: https://github.com/zakerf/opensearch-ccs-issue-reproduction

What is the expected behavior?
For my setup to keep working, or a notice of a breaking change somewhere

[github-actions]

Possible Duplicate

Found 1 possible duplicate issue(s):

• [BUG] Cross Cluster Search to same node fails auth #5846

If this is not a duplicate:

• Add a comment on this issue, or
• 👎 this comment to prevent auto-closure

Otherwise, this issue will be automatically closed in 7 days.

[zakerf]

summary of an AI assisted investigation:

Code Context

• In 2.19.6.0, PrivilegesEvaluatorImpl calls user.addSecurityRoles(mappedRoles) before forwarding. User.writeTo then serializes securityRoles.
• Commit 64c3709 removes that mutation as part of making User immutable.
• In 3.7.0.0, actionlevel/nextgen/PrivilegesEvaluatorImpl creates PrivilegesEvaluationContext(user, mappedRoles, ...), but does not use user.withSecurityRoles(mappedRoles).
• SecurityInterceptor forwards origUser.toSerializedBase64() in OPENDISTRO_SECURITY_USER_HEADER; ThreadContextUserInfo writes mapped roles only to the local user-info transient.

This leaves the OS3 forwarded user without the source-mapped Security role, while OS2 forwarded it.

All code references are real but whether it's really the cause I can't say.