[Core] Fix sorting of Dictionary keys

[AThousandShips]

StringName keys were sorted as StringName which is unstable.

• Fixes: Animation's data in *.tscn file always change keys order in Dictionary #97522

[AThousandShips]

Considering an alternative solution which allows fetching keys with StringName converted to String to improve behavior, will push as a separate commit soon when tested

Added a few cases I could find of this, will remove any that aren't relevant (the code in PackedDataContainer isn't affected as it sorts differently)

[dalexeev]

Given that there is already StringLikeVariantComparator, I have a minor question. As far as I know, there are three main types of comparators:

1. Equality (a == b). Not suitable for sorting.
2. Strict or non-strict comparison (a < b or a <= b).
3. Three-way comparison (a <=> b in C++20). Not sure if we use something like this.

It would be nice to make sure that we use consistent naming for these types. But that's not directly relevant to the issue, just a note.

struct \w+(Compare|Comparator|Sort|Sorter|Order)\b matches

core/config/project_settings.cpp: 1
381:  1: struct _VCSort {
core/debugger/local_debugger.cpp: 1
 37:  2: 	struct ProfileInfoSort {
core/math/convex_hull.cpp: 1
1573:  1: struct PointComparator {
core/string/string_name.h: 1
167:  2: 	struct AlphCompare {
core/string/ustring.h: 3
515:  1: struct NoCaseComparator {
521:  1: struct NaturalNoCaseComparator {
527:  1: struct FileNoCaseComparator {
core/templates/list.h: 1
705:  2: 	struct AuxiliaryComparator {
core/templates/pair.h: 2
 63:  1: struct PairSort {
108:  1: struct KeyValueSort {
core/templates/sort_array.h: 1
 44:  1: struct _DefaultComparator {
core/variant/array.cpp: 1
681:  1: struct _ArrayVariantSort {
core/variant/callable.h: 1
203:  1: struct CallableComparator {
core/variant/variant.h: 2
831:  1: struct VariantComparator {
835:  1: struct StringLikeVariantComparator {
drivers/gles3/rasterizer_scene_gles3.h: 1
155:  2: 	struct InstanceSort {
drivers/gles3/storage/particles_storage.h: 1
 71:  2: 	struct ParticlesViewSort {
editor/animation_bezier_editor.h: 1
172:  2: 	struct PairCompare {
editor/connections_dialog.cpp: 1
919:  1: struct _ConnectionsDockMethodInfoSort {
editor/debugger/script_editor_debugger.h: 1
173:  2: 	struct ThreadSort {
editor/doc_tools.cpp: 3
 80:  1: struct ConstructorCompare {
 98:  1: struct OperatorCompare {
114:  1: struct MethodCompare {
editor/editor_command_palette.h: 2
 62:  2: 	struct CommandEntryComparator {
 68:  2: 	struct CommandHistoryComparator {
editor/editor_file_system.h: 1
231:  2: 	struct DirectoryComparator {
editor/editor_help.h: 1
204:  2: 	struct PropertyCompare {
editor/editor_quick_open.h: 1
 58:  2: 	struct EntryComparator {
editor/editor_settings.cpp: 1
238:  1: struct _EVCSort {
editor/file_info.h: 2
 60:  1: struct FileInfoTypeComparator {
 66:  1: struct FileInfoModifiedTimeComparator {
editor/groups_editor.cpp: 1
 66:  1: struct _GroupInfoComparator {
editor/plugins/mesh_instance_3d_editor_plugin.cpp: 1
359:  1: struct MeshInstance3DEditorEdgeSort {
editor/plugins/node_3d_editor_plugin.cpp: 2
9291:  1: struct _GizmoPluginPriorityComparator {
9300:  1: struct _GizmoPluginNameComparator {
editor/plugins/node_3d_editor_plugin.h: 2
 74:  2: 	struct Axis2DCompare {
420:  2: 	struct ShortcutCheckSetComparator {
editor/plugins/text_shader_editor.h: 1
 61:  2: 	struct WarningsComparator {
editor/plugins/tiles/tiles_editor_plugin.h: 1
 64:  2: 	struct SourceNameComparator {
editor/plugins/visual_shader_editor_plugin.h: 1
403:  2: 	struct _OptionComparator {
editor/project_manager/project_list.cpp: 1
322:  1: struct ProjectListComparator {
modules/gdscript/gdscript.cpp: 2
280:  1: struct _GDScriptMemberSort {
2496:  1: struct GDScriptDepSort {
modules/gridmap/editor/grid_map_editor_plugin.cpp: 1
791:  1: struct _CGMEItemSort {
modules/lightmapper_rd/lightmapper_rd.h: 1
206:  2: 	struct TriangleSort {
modules/mono/csharp_script.cpp: 1
548:  1: struct CSharpScriptDepSort {
modules/text_server_adv/text_server_adv.h: 1
694:  2: 	struct GlyphCompare { // For line breaking reordering.
scene/2d/tile_map_layer.h: 2
164:  1: struct CellDataYSortedXReversedComparator {
195:  2: 	struct CoordsWorldComparator {
scene/3d/gpu_particles_collision_3d.h: 1
142:  2: 	struct FaceSort {
scene/3d/voxelizer.h: 1
 64:  2: 	struct CellSort {
scene/gui/code_edit.h: 1
515:  1: struct CodeCompletionOptionCompare {
scene/gui/control.h: 1
156:  2: 	struct CComparator {
scene/gui/text_edit.cpp: 1
4847:  1: struct _CaretSortComparator {
scene/main/scene_tree.h: 1
106:  2: 	struct ProcessGroupSort {
scene/resources/3d/importer_mesh.h: 1
 66:  3: 		struct LODComparator {
scene/resources/resource_format_text.h: 1
188:  2: 	struct ResourceSort {
scene/resources/surface_tool.h: 1
128:  2: 	struct WeightSort {
servers/debugger/servers_debugger.cpp: 1
197:  2: 	struct ProfileInfoSort {
servers/rendering/renderer_canvas_cull.h: 2
 93:  2: 	struct ItemIndexSort {
 99:  2: 	struct ItemPtrSort {
servers/rendering/renderer_rd/storage_rd/light_storage.h: 2
165:  2: 	struct LightInstanceDepthSort {
312:  2: 	struct ReflectionProbeInstanceSort {
servers/rendering/renderer_rd/storage_rd/texture_storage.h: 1
312:  2: 	struct DecalInstanceSort {
servers/rendering/rendering_device_graph.h: 1
232:  2: 	struct RecordedCommandSort {
servers/rendering/shader_language.h: 1
748:  2: 	struct UniformOrderComparator {

[AThousandShips]

Agreed, would have gone with Compare if it wasn't already taken, can go for Sort as it is more common it seems, will change when final decision on the solution is taken

[AThousandShips]

If we were to standardize this I'd change the ones named Compare but does equality checks to Eq or similar and the ones comparing order to Compare (as it seems the most common for those)

[AThousandShips]

Added a unit test for the Dictionary case, there are no stringify tests currently for JSON so left that for now, but can add if desired

[AThousandShips]

This static method isn't technically needed with the alternative solution (or when using List<Variant> in the visual shader code) but I think it's useful to keep so we can use it when sorting Array as it takes a Callable and we can use callable_mp_static

I will look at adding such static methods to other comparators to aid in using them for such cases

[Macksaur]

Allocating memory, filling it, then doing a comparison on the copy before dropping it immediately on the floor feels very slapdash. The memory should already be meaningful and available to use for such a comparison.

Perhaps this can be reused?:

godot/core/string/ustring.h

Line 534 in f032af7

_FORCE_INLINE_ bool is_str_less(const L *l_ptr, const R *r_ptr) {

edit: I don't trust this line 😆 perhaps I can be assuaged that there are no allocations in the general cases?

[AThousandShips]

It doesn't allocate memory except when the StringName is created via a c-strinf, which is rare comparatively

[Macksaur]

I'll take it, thank you!

[AThousandShips]

But will keep in mind for a future helper allowing comparison with both String and StringName generally using this correctly!

[AThousandShips]

Removed the alternative fix and will look at it for the future, to get this out there soon

Was looking at the new sort method of Dictionary which I had forgotten to look at after it was added for this purpose, but it doesn't suite this without some changes as it suffers from the same ordering limitations, namely preserving compatibility for Variant ordering in general

So for context, this solves that OP_LESS of Variant sorts by type, so all String keys are sorted separately and StringName keys are sorted separately, further StringName keys are sorted by pointer for performance, but that's unstable

This was introduced by changes that removed the conversion from StringName to String on all keys in:

• StringName Dictionary keys #70096

Tl;Dr;
This aims to solve sorting of Dictionary keys in serialisation in a compatibility preserving way while not impacting performance in general

[fire]

Does this fix the animation tscn sorting issue?

[AThousandShips]

Yes it should, will confirm fully today if no one else has, the relevant change is in variant_parser.cpp

Edit: Confirmed it does indeed make the sorting stable

[Repiteo]

Thanks!

[AThousandShips]

Thank you!