Originally NumberedObjectCollection was very wary of self.__num_cache because it was using "mutable data" as the keys. With #867 now though there should in theory be no case in which the cache is not up to date. Right now whenever this is a cache miss it is assumed it is a true miss, and not that the object is missing, so an $O(N)$ search is completed before a KeyError is raised. This is expensive, especially for renumbering tasks.
I believe now that this guardrail of search before a KeyError is no longer necessary. See this guardrail that is no longer covered because of this.