urls_sort_benchmark.py is doing partition sort ?

[junli1026]

A quick question about the urls_sort_benchmark.py, the code is doing the Hash partition and then local/partition sort,
but the result is not globally sorted at all, do I understand the code correctly ?

Either radix sort (replace string hashing with prefix ), or merge sort (not implemented in the code) makes sense to me, but the currently implementation seems problematic ?

[wangrunji0408]

Yes, it's not a global sort. Typically, we don't need global ordering, which can be very costly. URLs are partitioned by host, then sorted within each partition. The goal is to perform some aggregations on URLs with the same prefix.