Description
I'm trying to think of the code components as a stack of layers, and IndexShard#close() destructs pieces of the stack but not all. The storage layer is still alive, at the bottom of the stack, serving reads, but now we can't get to it through the IndexShard, which is the higher level distributed layer access point.
The IndexShard#close method resets the Engine reference. While active search operation keep the engine alive via reference in a ReaderContext, until the search completes.
In ES-6685 I need to get access, from the distributed layer, to an ES Engine implementation where active read information is tracked. I plan to create an external 'service' to hold the active read information because the ES Engine is gone. We should logically retain the IndexShard in a closed state until the storage engine is gone, and this would eliminate the need for external components with different lifecycles.
This is tech debt. Looks like the choice to reset the Engine reference on shard close was made ~10 years ago.