Skip to content

Commit 343545a

Browse files
committed
Added Logarithmic counting
1 parent 5d9828b commit 343545a

File tree

1 file changed

+34
-0
lines changed

1 file changed

+34
-0
lines changed

text/blk-counting.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,3 +144,37 @@ macro_rules! count_idents {
144144
This method does have two drawbacks. First, as implied above, it can *only* count valid identifiers (which are also not keywords), and it does not allow those identifiers to repeat.
145145

146146
Secondly, this approach is *not* hygienic, meaning that if whatever identifier you use in place of `__CountIdentsLast` is provided as input, the macro will fail due to the duplicate variants in the `enum`.
147+
148+
## Logarithmic counting
149+
150+
This approach is similar to Recursion, with the difference that every (one or two) recursive calls, the number of tokens to count is *divided by two*, instead of being reduced by one. Therefore, the recursion depth is the binary logarithm of the number of tokens to count and the expanded tree is likewise very small.
151+
152+
```rust
153+
macro_rules! count_tts {
154+
() => {0usize};
155+
($one:tt) => {1usize};
156+
($($pairs:tt $_p:tt)*) => {
157+
count_tts!($($pairs)*) << 1usize
158+
};
159+
($odd:tt $($rest:tt)*) => {
160+
count_tts!($($rest)*) | 1usize
161+
};
162+
}
163+
```
164+
165+
Here is the expansion for 51 tokens:
166+
167+
```rust
168+
count_tts!(---------------------------------------------------)
169+
count_tts!(--------------------------------------------------) | 1
170+
count_tts!(-------------------------) << 1 | 1
171+
count_tts!(------------------------) | 1 << 1 | 1
172+
count_tts!(------------) << 1 | 1 << 1 | 1
173+
count_tts!(------) << 1 << 1 | 1 << 1 | 1
174+
count_tts!(---) << 1 << 1 << 1 | 1 << 1 | 1
175+
count_tts!(--) | 1 << 1 << 1 << 1 | 1 << 1 | 1
176+
count_tts!(-) << 1 | 1 << 1 << 1 << 1 | 1 << 1 | 1
177+
1 << 1 | 1 << 1 << 1 << 1 | 1 << 1 | 1
178+
```
179+
180+
With this approach, the default recursion limit (64) is enough to count up to 2^64 tokens, which is more data than hard drives will be able to hold for the foreseeable future. It is quite fast (twice as fast as the Slice Length, for example, with an input of 100,000 tokens) and it produces a constant expression.

0 commit comments

Comments
 (0)