[Question] about checksum field validation in is_tar_header(): why require leading '0' or space?

[EMROF-H]

Hi Toybox maintainers,

First, thank you for your work on Toybox—it’s a great project and widely used (especially in Android).

I’m looking at the is_tar_header() function in toys/posix/tar.c, which includes this check:

if (p[148] != '0' && p[148] != ' ') return 0;

This appears to reject any tar header whose checksum field (at offset 148) does not start with either '0' or a space ' '.

However, I did not find this regulation in the POSIX ustar specification (IEEE Std 1003.1):

The fields magic, uname, and gname are character strings ...... typeflag contains a single character. All other fields are leading zero-filled octal numbers using digits from the ISO/IEC 646:1991 standard IRV. Each numeric field is terminated by one or more or NUL characters.

The chksum field shall be the ISO/IEC 646:1991 standard IRV representation of the octal value of the simple sum of all octets in the header logical record. Each octet in the header shall be treated as an unsigned value. These values shall be added to an unsigned integer, initialized to zero, the precision of which is not less than 17 bits. When calculating the checksum, the chksum field is treated as if it were all characters.

The standard does not require the checksum string to begin with '0' or a space—it only specifies that it be an octal, leading zero-filled number (Does that mean the meaning of that is must start with zero). Therefore, a valid checksum such as "123456 " should be acceptable, and its first byte would be '1'.

Could you clarify the rationale behind this restriction? Is it:

A deliberate compatibility choice (e.g., for Android’s expected tar format)?
An optimization to quickly reject malformed headers?
Or possibly an overly strict heuristic that could be relaxed to improve standards compliance?
If the goal is robustness, perhaps the check could be removed and rely solely on sscanf("%8o", ...) + checksum verification, which would align Toybox more closely with POSIX and other implementations.

Thanks again for your time and for maintaining this excellent codebase!

Best regards,
EMROF-H

[landley]

This appears to reject any tar header whose checksum field (at offset 148) does not start with either `'0'` or a space `' '`.

Do you have a real world tarball this check breaks on? If so, what created it?

However, I did not find this regulation in the POSIX ustar specification (IEEE Std 1003.1):

Joerg Schilling removed "tar" from the posix standard in favor of "pax", which was (as far as I can tell) universally ignored outside of solaris. Since the "tar" command is no longer standardized by posix, why should I care what they say about it?

> The fields magic, uname, and gname are character strings ...... typeflag contains a single character. All other fields are leading zero-filled octal numbers using digits from the ISO/IEC 646:1991 standard IRV. Each numeric field is terminated by one or more <space> or NUL characters.

Which seems to mean my test is accepting MORE inputs than theirs is. (They only accept 0, not space.) I am already violating their "standard" by accepting leading spaces, which I did because I encountered real tarballs that did that. (I have historical tarballs used to build linux from scratch going back to the 3.0 release, tarballs generated by git, tarballs that came from BSD... And then people sent me tarballs they had that didn't work, which is where 922bdeb and 3470a82 and d1101f2 and so on came from.)

> The chksum field shall be the ISO/IEC 646:1991 standard IRV representation

"The" representaion, not "a" representation. The representation has the length (not a length), and you quoted above where it said to pad it with leading zeroes. Trailing spaces are not part of such representation, and are often rejected by tools, random example from the debian system I'm sitting at: $ seq "1 " "2 " seq: invalid floating point argument: ‘1 ’ Try 'seq --help' for more information. $ seq " 1" " 2" 1 2

When calculating the checksum, the chksum field is treated as if it were all <space> characters.

Which is presumably where the tarball creators that padded with leading space instead of leading zero got it from. (Although I vaguely recall it was only the FIRST character that was space, but the field has more digits than the contents can actually add up to, which is why the first digit ALWAYS has to be zero when padded appropriately, hence the test being "zero or space" not "isdigit or space". Nonzero can't happen. This is also why it doesn't use the otoi() logic that handles the binary encoding with the high bit set: it can't be as big AS the field, let alone too big for the field.)

The standard does not require the checksum string to begin with '0' or a space—it only specifies that it be an octal, leading zero-filled number (Does that mean the meaning of that is must start with zero). Therefore, a valid checksum such as "123456 " should be acceptable, and its first byte would be '1'.

What tool generated the header in the real world tarball toybox couldn't extract?

Could you clarify the rationale behind this restriction?

Send me the test case that broke it, and describe (if you can without NDA) the use case that test case came from.

Is it: A deliberate compatibility choice (e.g., for Android’s expected tar format)?

I have never encountered a real world tarball it didn't work with, nor have you provided one.

An optimization to quickly reject malformed headers? Or possibly an overly strict heuristic that could be relaxed to improve standards compliance?

If posix removed the "tar" command from its standard ~20 years ago, and I didn't remove tar from busybox (which I used to maintain) or toybox (which I wrote most of) when they said to, why should I care what they have to say about tar? The comment block at the top of tar.c links to https://www.gnu.org/software/tar/manual/html_node/Tar-Internals.html which was at least as useful as anything posix had to say. And still an awful lot of staring at hexdumps and receiving bug report regardless of what any documentation had to say about it.