Skip to content

Commit 7bedcbe

Browse files
Nicoshevmeta-codesync[bot]
authored andcommitted
Improve successful find speed by 1 cycle on Aarch64
Summary: X-link: facebook/folly#2589 The result of SparseMaskIter's next() is often used as an index on an 8-byte element array. In this case, the index needs to be shifted left by 3 to access the desired memory position. The return statement of the mentioned function contains i >> 2. The compiler is simplifying the shifts by only issuing a lsl 1 while ommitting the lsr 2. However, it then ANDs the shifted value by 0xf8, to ensure correctness when variable i is not a multiple of 4. We do know that variable i will always be a multiple of 4. We add the assume clause so the compiler avoids emitting the &0xf8 Before the assembly looked like this: clz x16, x16 lsl x16, x16, #1 and x16, x16, #0xf8 ldr x16, [x14, x16] After the changes, we verified the AND is omitted: clz x16, x16 lsl x16, x16, #1 ldr x16, [x14, x16] By removing a pipelined instruction in the codepath, execution latency is reduced by 1 cycle 🤗 It also allows the processor to foresee proceeding instructions up to 1 cycle earlier Reviewed By: yfeldblum Differential Revision: D94030304 fbshipit-source-id: c40a692345b051634c78c262eef8ee966a804104
1 parent 6755381 commit 7bedcbe

1 file changed

Lines changed: 12 additions & 0 deletions

File tree

  • third-party/folly/src/folly/container/detail

third-party/folly/src/folly/container/detail/F14Mask.h

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,18 @@ class SparseMaskIter {
156156
unsigned i =
157157
kIsArchAArch64 ? findLastSetNonZero(mask_) : findFirstSetNonZero(mask_);
158158
mask_ &= kIsArchAArch64 ? (lo63 >> i) : (mask_ - 1);
159+
if constexpr (kIsArchAArch64 && (kMaskSpacing == 4)) {
160+
// The result of this function is often used as an index on an 8-byte
161+
// element array. In this case, the index needs to be shifted left by 3 to
162+
// access the desired memory position. The return statement of this
163+
// function contains i >> 2. The compiler is simplifying the shifts by
164+
// only issuing a lsl 1 while ommitting the lsr 2. However, it then ANDs
165+
// the shifted value by 0xf8, to ensure correctness when i is not a
166+
// multiple of 4. We do know that i will always be a multiple of 4. We add
167+
// the assume clause so the compiler avoids emitting the &0xf8
168+
auto loadIndex = i << 1;
169+
assume(loadIndex == (loadIndex & 0xf8));
170+
}
159171
return i / kMaskSpacing;
160172
}
161173
};

0 commit comments

Comments
 (0)