Skip to content

Add REST & Transport layers section to GeneralArchitectureGuide.md #126377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
May 29, 2025

Conversation

nicktindall
Copy link
Contributor

@nicktindall nicktindall commented Apr 7, 2025

As part of the effort to document onboarding topics I tried to document the contents of "REST vs Transport layer and coordinating nodes" topic.

There's a lot more that could be said, advice on how much more of that to include is appreciated. I was hoping to keep the section(s) roughly aligned with onboarding topics, but I think I've already said too much about TransportActions

See rendered content at https://github.com/nicktindall/elasticsearch/blob/add_rest_and_transport_layers/docs/internal/GeneralArchitectureGuide.md

Closes: ES-7885

@nicktindall nicktindall added >docs General docs changes :Distributed Coordination/Network Http and internode communication implementations labels Apr 7, 2025
Copy link
Member

@ywangd ywangd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nicktindall nicktindall marked this pull request as ready for review April 8, 2025 03:40
@elasticsearchmachine elasticsearchmachine added Team:Docs Meta label for docs team Team:Distributed Coordination Meta label for Distributed Coordination team labels Apr 8, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, a few suggestions but nothing blocking


The node that a REST request arrives at is called the "coordinating" node. Its job is to coordinate the execution of the request with the other nodes in the cluster and when the requested action is completed, return the possibly aggregated response to the REST client via HTTP.

By default, all nodes will act as coordinating nodes, but by specifying `node.roles` to be empty you can create a [coordinating-only node](https://www.elastic.co/guide/en/elasticsearch/reference/current/node-roles-overview.html#coordinating-only-node-role).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a little confusing to talk about node roles given we're considering coordinating REST requests here. All nodes handle REST requests, this is not optional.

Copy link
Contributor

@DiannaHohensee DiannaHohensee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look at the intro and REST sections, but didn't get to the transport section.

I think you have a lot of great content, but I think it's missing high level explanations to A) make that content easier to understand/digest and B) tie the components together into a big picture.

@DiannaHohensee
Copy link
Contributor

DiannaHohensee commented Apr 10, 2025

Uhoh. I started feeling like I'd already dug around in this code: we have the REST layer explained over here hiding under networking https://github.com/elastic/elasticsearch/blob/main/docs/internal/DistributedArchitectureGuide.md#networking. You have some more REST bits, though, and the transport layer was not explained.

@DiannaHohensee
Copy link
Contributor

Uhoh. I started feeling like I'd already dug around in this code: we have the REST layer explained over here hiding under networking https://github.com/elastic/elasticsearch/blob/main/docs/internal/DistributedArchitectureGuide.md#networking. You have some more REST bits, though, and the transport layer was not explained.

I'm sorry I didn't notice this earlier. They'll need to be reconciled. Looks like I was connecting all those topics under networking because the ActionListeners tie responses back to Netty and get passed through the REST layer on netty threads to the Transport layer to then run on potentially different thread pools... That was a little too clever, in retrospect 😅

@DiannaHohensee
Copy link
Contributor

I opened #126643 to move the REST documentation into the General guide, so your final diff will be cleaner. But feel free to propose something different.

…t_layers

# Conflicts:
#	docs/internal/GeneralArchitectureGuide.md
@nicktindall
Copy link
Contributor Author

Sorry, I did a fairly substantial re-work to try and incorporate the best bits of the new and old text. I tried to address all the feedback but it's quite different to what was there, so probably needs to be re-reviewed altogether.

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM still

Copy link
Contributor

@DiannaHohensee DiannaHohensee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left many wording suggestions, and a bit puzzled by a link format, but LGTM and no need for further approval 👍

Lots of detail! 🙂

@nicktindall nicktindall enabled auto-merge (squash) May 29, 2025 00:41
@nicktindall nicktindall merged commit 83d7b7d into elastic:main May 29, 2025
7 checks passed
@nicktindall nicktindall deleted the add_rest_and_transport_layers branch May 29, 2025 00:44
Samiul-TheSoccerFan pushed a commit to Samiul-TheSoccerFan/elasticsearch that referenced this pull request Jun 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Network Http and internode communication implementations >docs General docs changes Team:Distributed Coordination Meta label for Distributed Coordination team Team:Docs Meta label for docs team v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants