[WIP] Functionality for `int_format_into` for integer types #138338

madhav-madhusoodanan · 2025-03-11T06:11:47Z

Context

Implemented integer formatting into a fixed-size buffer without relying on fmt.
Added a NumBuffer type to help with this process.

Associated Issue

Tracking Issue for integer formatting into a fixed-size buffer #138215

rustbot · 2025-03-11T06:11:51Z

Failed to set assignee to hanna-kruppe: invalid assignee

Note: Only org members with at least the repository "read" role, users with write permissions, or people who have commented on the PR may be assigned.

madhav-madhusoodanan · 2025-03-11T06:12:14Z

@rustbot author

hanna-kruppe · 2025-03-11T18:34:05Z

FYI, I'm going to unsubscribe from notifications here for now. As the bot said, I'm not a reviewer on this repository and you should pick someone else once the PR is ready for review. Although I can't approve any PR, I wouldn't mind also giving some input on this one, so feel free to @ me again at that point. But right now I'm getting a firehose of uninformative emails that makes it likely I'll miss the point where it would be useful for me to take a look.

madhav-madhusoodanan · 2025-03-12T04:56:41Z

Apologies @hanna-kruppe for the notifications from this PR. I was finally able to get a PASS on the doctests I created for this functionality.

madhav-madhusoodanan · 2025-03-12T04:57:52Z

library/core/src/num/int_format.rs

+    /// An alternative to `contents.len()` is `BUF_SIZE`.
+    pub contents: [MaybeUninit<u8>; BUF_SIZE],
+}
+


I have a couple of questions:

Should we move NumBuffer to another module (such as array) so that it could be useful for other modules too in the future?

If this type remains a very thin wrapper around a [MaybeUninit<u8>; CONST_GENERIC_PARAM] then I don't see the point of having the type at all -- just use the array. If it's sized to fit decimal representations of integers, as originally planned, then it's specific to numbers and seems out of place in other modules.

madhav-madhusoodanan · 2025-03-12T04:59:09Z

library/core/src/num/int_format.rs

+    offset
+}
+
+#[inline]


I've decomposed the process into 2-3 parts (so that I can reuse functionality for both signed and unsigned integers):

Write raw digits into the buffer

Write the negative sign into the buffer (only for signed types, incase number is negative)

Generate the &str from the buffer

Is this decomposition a good idea? My concern is about readability and the safety assumptions of the unsafe blocks.

Edit: removed #[inline] tags and added comments for clarity.

madhav-madhusoodanan · 2025-03-12T05:10:15Z

r? @scottmcm
@rustbot review

hanna-kruppe

Took a first look. I'm surprised by the changes in API design (user-chosen buffer size, panic if that's not sufficient) so I didn't go into detail with the unsafe code or really most of the implementation in general.

hanna-kruppe · 2025-03-12T18:55:15Z

library/core/src/num/int_format.rs

+/// `MaybeUninit<u8>`.
+#[unstable(feature = "int_format_into", issue = "138215")]
+#[derive(Debug)]
+pub struct NumBuffer<const BUF_SIZE: usize> {


The tracking issue only mentions a non-generic buffer type (as suggested by T-libs-api members), with sufficient size for any integer. While I still have sympathies for making the buffer generic over the integer type to guard against much larger integer types in the future, it's not clear to me why users should be allowed to pick any (overly large or far too small) buffer size they like.

My bad, I mistook NumBuffer<$Int> as generic over the size of the buffer, not the integer type.

Will fix this

Now that you mention it, I can see how Buffer<$int> is kinda ambiguous, oops.

The first solution that comes to mind is to use the nightly feature generic_const_exprs to calculate the max buffer sizes for each integer type.

I don’t think this would be recommended, right?

The alternative I think might be to manually set the buffer size for each integer type using macros.

I would suggest first implementing the API as it was accepted in the ACP, i.e., with a single non-generic struct NumBuffer { ... }.

hanna-kruppe · 2025-03-12T18:57:30Z

library/core/src/num/int_format.rs

+                    let decimal_string_size: usize = if self < 0 {
+                        self.unsigned_abs().ilog(10) as usize + 1 + 1
+                    } else if self == 0 {
+                        1
+                    } else {
+                        self.ilog(10) as usize + 1
+                    };
+
+                    // `buf` must have minimum size to store the decimal string version.
+                    // BUF_SIZE is the size of the buffer.
+                    if BUF_SIZE < decimal_string_size {
+                        panic!("Not enough buffer size to format into!");
+                    }


... for example, all of this is only necessary because BUF_SIZE is chosen by the user and not determined by the integer type. And since it's based on the actual value of self, a smaller value may work sometimes. Is this intentional? It's rather different from what the existing standard library code, the itoa crate, and the API change proposal do.

hanna-kruppe · 2025-03-12T19:00:42Z

library/core/src/num/int_format.rs

+}
+
+/// Macro to write ascii digits into the buffer, towards the end.
+macro_rules! raw_write_digits_into {


How is this different from the existing code in the Display impls? I would have expected that code to be reused, not effectively duplicated.

hanna-kruppe · 2025-03-12T19:02:03Z

library/core/src/num/int_format.rs

+fn raw_extract_as_str<const BUF_SIZE: usize>(buf: &NumBuffer<BUF_SIZE>, offset: usize) -> &str {
+    // SAFETY: All contents of `buf` since `offset` is set.
+    let written = unsafe { buf.contents.get_unchecked(offset..) };
+
+    // SAFETY: Writes use ASCII from the lookup table
+    // (and `NEGATIVE_SIGN` in case of negative numbers) exclusively.


This function should be unsafe since it does not itself ensure the properties the safety comments rely on, it relies on the caller to do so.

hanna-kruppe · 2025-03-12T19:08:58Z

library/core/src/num/int_format.rs

+    /// An alternative to `contents.len()` is `BUF_SIZE`.
+    pub contents: [MaybeUninit<u8>; BUF_SIZE],
+}
+


If this type remains a very thin wrapper around a [MaybeUninit<u8>; CONST_GENERIC_PARAM] then I don't see the point of having the type at all -- just use the array. If it's sized to fit decimal representations of integers, as originally planned, then it's specific to numbers and seems out of place in other modules.

hanna-kruppe · 2025-03-12T19:18:34Z

library/core/src/num/int_format.rs

+                #[doc = concat!("assert_eq!(n3.format_into(&mut buf3), ", stringify!($SignedT::MAX), ".to_string());")]
+                /// ```
+                ///
+                pub fn format_into<const BUF_SIZE: usize>(self, buf: &mut crate::num::NumBuffer<BUF_SIZE>) -> &str {


Ah, one more thing -- it would be very unfortunate to monomorphize all of this code for every distinct choice of buffer size (multiplied further by duplicate instantiations across separate codegen units). There's really not much benefit to be had by specializing the code for a particular buffer size (the only one I can think of is that a sufficiently clever compiler could elide the "is the size sufficient?" check if it's large enough for any value of the integer), so it's just a waste of binary size. As far as I know, no existing library on crates.io makes this choice either, and they're all quite performance conscious.

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Mar 11, 2025

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 11, 2025