Guarantee a deterministic order when processing files for directory arguments.

[akamat10]

Bug description

While trying to investigate the primer indeterminacy issues we are seeing in PR, one thing I observed is that we don't impose any order in which we process files or directories when we call os.walk to discover files to process here. AFAIK the order of directories for files is not guaranteed in os.walk. This may cause inconsistent no-name-in-module or import-error due to name shadowing issues during different runs. I think we need to guarantee the order of filenames and directories that are processed. This is not the source of the primer issue as far as I can see but we should fix this anyway. Thoughts on this?

Configuration

No response

Command used

pylint my_directory

Pylint output

None

Expected behavior

Provide guaranteed order of processing files when a directory is passed.

Pylint version

pylint 3.2.7
astroid 3.2.4
Python 3.12.3

OS / Environment

No response

Additional dependencies

No response

[Pierre-Sassoulas]

Thanks for the investigation. Imo primer need to be independant together whatever the order (we should clear the cache between primers). The sorting in walk might fix the indeterminacy but we would still have test pollution from inference we shouldn't have in projects depending on other primed project (I.e. residual django inferences in sentry if sentry is primed after django).

[akamat10]

Sorry I should have clarified that this has nothing to primer issue and just that the order of processing could cause indeterminacy in general. The cache clearing bug issue with primer has been addressed in pylint-dev/astroid#2572.

[Pierre-Sassoulas]

Ha, my bad, indeed let's make it deterministic then.