Performance issues with a large number of file lists

[FiblackBrother]

Please help me.
Retrieve 40000 files from the solid-state drive list in just 1 second.
GlusterFS creates a Brick 4 replica 2 list query for 40000 files (takes 7 seconds) and a list query for millions of files (takes a few minutes).
How can I optimize the configuration of GlusterFS.

[anon314159]

Without additional context or information (gluster volume info ) troubleshooting issues may be a bit difficult but below are few commands (version dependent) that you can execute to speed up metadata performance. The information below assumes your issue isn't hardware or network related and your cluster is not conducting some sort of heal operation.

#Enables Negative Lookup Cache:
gluster volume set "volume name" group nl-cache

#Enables and extends Metadata Cache Lifetime:
gluster volume set "volume name" group metadata-cache

#Additional Performance Tuning:
gluster volume set "volume name" performance.parallel-readdir on
gluster volume set "volume name" performance.readdir-ahead on

Possible unresolved root cause(s):
#4335
#4195
#3980

Something to keep in mind, GlusterFS and other distributed file systems perform poorly when crawling or traversing directories that contain lots of files or many nested subdirectories. However, the problem is even further amplified with gluster because it lacks a centralized metadata storage location and relies heavily on DHT (Distributed Hash Table's) or AFR (Automatic File Replication) for file/directory access and lookups (i.e. brick traversal at the cost of much higher metadata latency). Neither of these implementations scale well with large volumes of files or directories. Enabling negative lookup and metadata caching helps resolve some of these issues but at the cost of increased memory consumption within the glusterd process but the performance impact should be noticeable (still perceivably bad from an end-user perspective). Also, using the long listing argument (-l) with the ls command adds significant overheads as your shell to lookup even more information (extended attributes).