Skip to content

allow non-null post tree pre block padding#1918

Merged
braingram merged 4 commits into
asdf-format:mainfrom
braingram:junk_data
Apr 29, 2025
Merged

allow non-null post tree pre block padding#1918
braingram merged 4 commits into
asdf-format:mainfrom
braingram:junk_data

Conversation

@braingram
Copy link
Copy Markdown
Contributor

@braingram braingram commented Apr 24, 2025

Description

On main if padding bytes are found between the end of the tree and the beginning of the first block they must be all null.

asdf/asdf/_block/reader.py

Lines 172 to 173 in ad470c0

# this is the first block, allow empty bytes before block
buff = buff.strip(b"\0")

This doesn't agree with the standard:

This empty space may be filled with any content (as long as it doesn’t contain the block_magic_token described in Blocks).

This PR updates the block parsing to allow non-null bytes as long as they don't match the block magic. Since the block header does not contain a checksum of the header bytes it's not possible to detect invalid padding (that includes the block magic). See asdf-format/asdf-standard#460

Once this is merged and asdf-format/asdf-standard#456 are merged asdf-format/asdf-standard#441 can be closed.

Tasks

  • run pre-commit on your machine
  • run pytest on your machine
  • Does this PR add new features and / or change user-facing code / API? (if not, label with no-changelog-entry-needed)
    • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
    • update relevant docstrings and / or docs/ page
    • for any new features, add unit tests
news fragment change types...
  • changes/<PR#>.feature.rst: new feature
  • changes/<PR#>.bugfix.rst: bug fix
  • changes/<PR#>.doc.rst: documentation change
  • changes/<PR#>.removal.rst: deprecation or removal of public API
  • changes/<PR#>.general.rst: infrastructure or miscellaneous change

@braingram braingram marked this pull request as ready for review April 24, 2025 18:00
@braingram braingram requested a review from a team as a code owner April 24, 2025 18:00
Copy link
Copy Markdown
Member

@zacharyburnett zacharyburnett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I can tell this looks good to me

might as well add a SorcerersTome class with all this reading of magic blocks

@braingram braingram merged commit 97a9fe4 into asdf-format:main Apr 29, 2025
35 checks passed
@braingram braingram deleted the junk_data branch April 29, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants