Skip to content

fix(scripts/bundle): Pack NVMe-SPINOR tarballs separately#354

Open
vishwasudupa wants to merge 1 commit intoqualcomm-linux:mainfrom
vishwasudupa:flash-nvme-spinor
Open

fix(scripts/bundle): Pack NVMe-SPINOR tarballs separately#354
vishwasudupa wants to merge 1 commit intoqualcomm-linux:mainfrom
vishwasudupa:flash-nvme-spinor

Conversation

@vishwasudupa
Copy link
Copy Markdown
Contributor

Place SPINOR content under the NVME folder to align with the current content.xml layout.

Use _UFS and _NVME directories to distinguish storage types instead of inferring from partition conf.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 6, 2026

Test Results

 2 files  ±0   6 suites  ±0   3m 19s ⏱️ ±0s
20 tests ±0  20 ✅ ±0  0 💤 ±0  0 ❌ ±0 
64 runs  ±0  64 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 3f1b69f. ± Comparison against base commit f120944.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 6, 2026

Test jobs for commit 00807f9

Copy link
Copy Markdown
Contributor

@lool lool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for this!

Your changes look clean but they kind of go counter the current idea: we currently generate two tarballs, grouping platforms by the sector size they use. To do this, we're scanning generated files to see if the 512B or 4k sector size was used (disk-sdcard or -ufs).

Right now, nvme platforms end up in the emmc tarball because they use 512B sectors, and spinor ends up with them due to the default case – a bit of a hack.

What I would propose instead is to:

  1. stop including spinor in emmc by default
  2. for each _spinor dir, copy it to each of the corresponding _emmc, _ufs and _nvme directories with the same platform name

This avoids special casing NVMe and special cases spinor as a sign of a NHLOS storage.

I don't think you need to do it in this PR, but I'd like us to rename disk-sdcard/disk-ufs and flash-ufs/flash-emmc to better names related to the sector size.

@lool
Copy link
Copy Markdown
Contributor

lool commented Apr 8, 2026

Hey, thanks for this!

Your changes look clean but they kind of go counter the current idea: we currently generate two tarballs, grouping platforms by the sector size they use. To do this, we're scanning generated files to see if the 512B or 4k sector size was used (disk-sdcard or -ufs).

Right now, nvme platforms end up in the emmc tarball because they use 512B sectors, and spinor ends up with them due to the default case – a bit of a hack.

What I would propose instead is to:

  1. stop including spinor in emmc by default
  2. for each _spinor dir, copy it to each of the corresponding _emmc, _ufs and _nvme directories with the same platform name

This avoids special casing NVMe and special cases spinor as a sign of a NHLOS storage.

I don't think you need to do it in this PR, but I'd like us to rename disk-sdcard/disk-ufs and flash-ufs/flash-emmc to better names related to the sector size.

I slept on this, and realized we probably don't want to change the contents of the flash dirs in the late bundling stage, so idea in 2) should probably be implemented in the logic which is currently in the flash recipe, but that Agathe started moving out in #334

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

Test jobs for commit b2ba056

@lool
Copy link
Copy Markdown
Contributor

lool commented Apr 9, 2026

We discussed this out of band with Vishwas:

  • I pointed out I want to keep the two tarballs coming out of CI (one per disk sector size)
  • perhaps we can add a yaml metadata entry to point out to the NHLOS ptool config to use (spinor for glymur-crd)
  • Vishwas pointed out ptool has the information in partitions.conf about the sector size; this is great news, and we can use it to route files to one or the other (probably testing rawprogram0.xml to see which sector size it refers to)
  • I haven't checked how the contents.xml.in in ptool currently expect things to be laid out, but I expect that it could look like:
    • flash_glymur-crd_nvme/
      • files to flash on the NVMe (disk-media.img1, disk-media.img2, rawprogram0.xml, patch0.xml...)
      • spinor/
        • files to flash on spinor (cdt.bin, uefi.bin, rawprogram0.xml, patch0.xml...)
      • contents.xml listing both
  • and since it's likely going to use 512B sectors, I expect it will be zipped into flash-emmc.tgz for now

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2026

Test jobs for commit c871bee

@vishwasudupa
Copy link
Copy Markdown
Contributor Author

Hey, thanks for this!

Your changes look clean but they kind of go counter the current idea: we currently generate two tarballs, grouping platforms by the sector size they use. To do this, we're scanning generated files to see if the 512B or 4k sector size was used (disk-sdcard or -ufs).

Right now, nvme platforms end up in the emmc tarball because they use 512B sectors, and spinor ends up with them due to the default case – a bit of a hack.

What I would propose instead is to:

  1. stop including spinor in emmc by default
  2. for each _spinor dir, copy it to each of the corresponding _emmc, _ufs and _nvme directories with the same platform name

This avoids special casing NVMe and special cases spinor as a sign of a NHLOS storage.

I don't think you need to do it in this PR, but I'd like us to rename disk-sdcard/disk-ufs and flash-ufs/flash-emmc to better names related to the sector size.

I have enabled spinor to be going to its respective directory in latest patch

Copy link
Copy Markdown
Contributor

@lool lool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, this seems needlessly complicated, there's already a top loop on all directories earlier and a simpler way to write this

My main issue though is that I think we could do that in the flash recipe itself, so that this behavior is the result of image builds, not just the result in our CI artifacts (this script is used from the github workflow).

I would think the same logic should work from the main flash recipe; after the $platforms loop; something like:

for dir in flash_*; do
    # skip directories that are named _spinor
    if [ -z "${d%%*_spinor}" ]; then
        continue
    fi
    # strip _storage suffix, and check for a _spinor directory
    spinor_dir="${d%_*}_spinor"
    if [ -d "$spinor_dir" ]; then
        # embed a copy of spinor contents
        cp ...preserve args... "$spinor_dir" "$d"
    fi
done


# Organize spinor directories under their base directories
for dir in $emmc_dirs $ufs_dirs; do
[ -d "$dir" ] || continue
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this line is useful? we just assembled a list of directories, and you're checking if the directory is present?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok will remove it and simplify

# Organize spinor directories under their base directories
for dir in $emmc_dirs $ufs_dirs; do
[ -d "$dir" ] || continue
echo "$dir" | grep -q "_spinor$" || continue
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we loop through all dirs that mix emmc, ufs, nvme and spinor, just to spot the spinor ones; then we have to do another loop to see where to copy them to. That seems needlessly complicated.

You could either:

  • do this earlier
  • or check if the directory is not suffixed _spinor, and then see if there's a corresponding _spinor dir to copy into it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added in start and simplified code without && for better visibility

echo "$dir" | grep -q "_spinor$" || continue
prefix=$(echo "$dir" | sed 's/_spinor$//')
for folder in $emmc_dirs $ufs_dirs; do
echo "$folder" | grep -q "^${prefix}_" && [ "$folder" != "$dir" ] && [ -d "$folder" ] && cp -r "${dir}"/* "${folder}/" && break
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The && chained commands are hard to read; also, you seem to stop the loop after the first cp (I guess that's the point of break), but what if we support nvme and ufs, and we want spinor in both?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes in those cases need to maintain metadata somewhere to see which HLOS once get spinor, i am thinking id that info can be got via any xml/conf from ptool

let me know if any place if this mapping can be found

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a way to get the content.xml in spinor works with which other storage

I have used that logic in latest patch, this should resolve for case where there is both nvme/ufs

…ctory

When a platform has a *_spinor directory, place it under the corresponding
platform target storage (emmc/ufs/nvme).

Signed-off-by: Vishwas Udupa <vudupa@qti.qualcomm.com>
@vishwasudupa
Copy link
Copy Markdown
Contributor Author

Hey, this seems needlessly complicated, there's already a top loop on all directories earlier and a simpler way to write this

My main issue though is that I think we could do that in the flash recipe itself, so that this behavior is the result of image builds, not just the result in our CI artifacts (this script is used from the github workflow).

I would think the same logic should work from the main flash recipe; after the $platforms loop; something like:

for dir in flash_*; do
    # skip directories that are named _spinor
    if [ -z "${d%%*_spinor}" ]; then
        continue
    fi
    # strip _storage suffix, and check for a _spinor directory
    spinor_dir="${d%_*}_spinor"
    if [ -d "$spinor_dir" ]; then
        # embed a copy of spinor contentsd
        cp ...preserve args... "$spinor_dir" "$d"
    fi
done

I taught since this is bundling decision ,it was better to keep in bundle-flash-dirs.
alterations if needed, can it be done in future PR?

@github-actions
Copy link
Copy Markdown

Test jobs for commit 3f1b69f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants