[BUG] Scalar loc-based lookup in integer categorical indices is incorrect

[wence-]

Describe the bug

loc-based lookup does (I think incorrect) fallback to positional indexing rather than label-based lookup when the index is a categorical one with integer values.

Steps/Code to reproduce bug

import cudf

s = cudf.Series([1, 2], index=cudf.CategoricalIndex([3, 4], categories=[3, 4]))
s.loc[3] # IndexError: single positional indexer is out-of-bounds

Expected behavior

s.to_pandas().loc[3]
# 1

Annoyingly, one can't just stop doing positional indexing fallback in all cases because if the index is (say) a string index then integer indexing does fall back to positional. This is for Series.__getitem__ and the behaviour is deprecated in pandas 2.