Skip to content

examples/elf: Extend nxpkg validation coverage.#3531

Open
aviralgarg05 wants to merge 1 commit into
apache:masterfrom
aviralgarg05:gsoc/elf-validation-fixtures-pr4
Open

examples/elf: Extend nxpkg validation coverage.#3531
aviralgarg05 wants to merge 1 commit into
apache:masterfrom
aviralgarg05:gsoc/elf-validation-fixtures-pr4

Conversation

@aviralgarg05

@aviralgarg05 aviralgarg05 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Generate reproducible nxpkg validation fixtures from the examples/elf
hello artifact.

A C host helper creates:

  • index.json for a valid package and a mismatched target;
  • bad-index.json for a missing artifact;
  • pkgtest.nsh for the successful install/list path; and
  • pkgfail.nsh for the expected failure path.

The SHA-256 digest and target metadata are derived from the artifact and build
configuration instead of being maintained by hand.

The generated scripts seed both the original /etc/nxpkg/index.json catalog
location and #3642's default /var/lib/nxpkg/index.jsn location. This keeps
the standalone fixtures usable before and after the storage migration.

Impact

  • New feature: YES, limited to generated nxpkg validation fixtures.
  • User adaptation: NO.
  • Build: YES for examples/elf; a C host helper is used by Make and CMake.
  • Hardware: NO architecture, board, or driver change.
  • Documentation: NO user documentation is required for this validation
    fixture.
  • Security: NO runtime trust change.
  • Compatibility: The loader and nxpkg runtime behavior are unchanged.
  • Other: The valid and failing cases are generated from the same target
    artifact and build metadata.

Testing

Host:

  • macOS 26.5, arm64
  • xtensa-esp-elf-gcc 14.2.0
    (esp-14.2.0_20241119)

Target:

  • esp32s3-xiao:elf
  • Test configuration additionally requires
    CONFIG_EXAMPLES_ELF_ROMFS=y and CONFIG_SYSTEM_NXPKG=y. The related
    NuttX branch's current esp32s3-xiao:elf defconfig does not enable
    CONFIG_SYSTEM_NXPKG, so that prerequisite must be added before collecting
    the final target log.

Verification:

  • clean Make build;
  • repaired CMake loadable-hello target and verified elf_artifacts path;
  • all four generated fixture files; and
  • full nuttx.bin link.

Testing logs before change:

N/A: the generated validation scripts do not exist on the PR base.

Testing logs after change:

Real hardware (ESP32-S3):
  The complete examples/elf suite finished with end memory delta 0.
  pkgtest.nsh installed hello 1.0.0 and nxpkg list showed its target metadata.
  pkgfail.nsh selected hello-missing, failed artifact acquisition with -2,
  and left the installed list empty.

CMake fixture result:
  hello             1820 bytes
  index.json         413 bytes
  bad-index.json     230 bytes
  pkgtest.nsh        229 bytes
  pkgfail.nsh        234 bytes

  The index SHA-256 exactly matches the CMake-built hello ELF.
  The complete firmware target reached the final nuttx link, where this
  host's Homebrew ARM toolchain lacks nosys.specs. The elf_artifacts target,
  including the generated ROMFS, completed successfully.

The full retained serial transcript should be attached with the signed update.

PR verification Self-Check

  • This PR introduces one focused validation-fixture change.
  • I have updated all required description fields above.
  • I have reviewed and signed the rewritten commit.
  • I have collected before/after real-hardware runtime logs.
  • This PR adheres to the current contribution and coding guidelines.
  • My PR is still work in progress.
  • My PR is ready for review and can be safely merged.

@aviralgarg05
aviralgarg05 force-pushed the gsoc/elf-validation-fixtures-pr4 branch 2 times, most recently from c180ecf to 04e17b4 Compare June 13, 2026 16:32
@aviralgarg05
aviralgarg05 marked this pull request as ready for review June 21, 2026 14:55
@aviralgarg05
aviralgarg05 force-pushed the gsoc/elf-validation-fixtures-pr4 branch from 04e17b4 to 4429450 Compare June 21, 2026 14:57

@acassis acassis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aviralgarg05 please test CMake as well, NuttX used both building systems

@aviralgarg05
aviralgarg05 force-pushed the gsoc/elf-validation-fixtures-pr4 branch 2 times, most recently from bf8be7a to 7f20e7e Compare June 22, 2026 13:52
acassis
acassis previously approved these changes Jun 22, 2026
@acassis

acassis commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

@xiaoxiang781216 PTAL

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

This PR is ready to merge :)

@linguini1 linguini1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change enforces that the user has Python as part of the build system, which I remember was a contentious issue.

Also, is the elf example not different from the nxpkg application? I don't think we should be generating nxpkg test artifacts from the elf example. It would be better to include those under some test/ directory.

@acassis

acassis commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

@aviralgarg05 please convert these tools to C, I think it is safer

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

I pushed an update for this.

The Python generators added by this PR are gone now. The fixture helper is now a small C host tool, and the CMake path uses the existing in-tree tooling style instead of introducing new Python build steps.

I kept the fixture under examples/elf because it is validating the hello artifact produced by that example itself, but if you still want that split into a dedicated test location, I can do that as a follow-up.

@linguini1 linguini1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Squash your commits.

Extend the examples/elf ROMFS path so nxpkg validation can use
generated package fixtures instead of hand-managed metadata.

Generate index.json, bad-index.json, pkgtest.nsh, and pkgfail.nsh
from the built hello ELF, recording the correct target arch/compat
metadata and SHA-256 digest for the valid fixture. Keep mismatched-
target and missing-artifact cases alongside the valid fixture so
nxpkg target selection and failure handling can be exercised from
the same example tree. Group the paired fixture outputs in one make
step so parallel builds do not re-enter the generator independently.

This keeps the fixture metadata tied to the actual built hello
artifact instead of relying on manually maintained hashes or ad hoc
shell setup, making follow-up review and later test reruns more
predictable. There is no intended loader or board-behavior change;
this is limited to fixture generation inside examples/elf.

Assisted-by: Claude:claude-sonnet-5
Assisted-by: OpenAI Codex:gpt-5.6-sol
Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05
aviralgarg05 force-pushed the gsoc/elf-validation-fixtures-pr4 branch from 8ef5065 to 98a74df Compare July 24, 2026 10:01
@aviralgarg05 aviralgarg05 changed the title examples/elf: extend nxpkg validation coverage examples/elf: Extend nxpkg validation coverage. Jul 24, 2026
@aviralgarg05

Copy link
Copy Markdown
Contributor Author

Updated as one signed commit. The C host generator is shared by the Make and CMake paths, and the CMake graph now defines the loadable hello target and complete fixture outputs. On the ESP32-S3 target, the full ELF suite ended with zero memory delta; the valid package installed and listed, while the missing artifact was rejected without installed state. Fresh CI is queued on the new head.

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

One additional scope clarification: the loadable fixture itself now lives under examples/elf/tests, while the host helper is under examples/elf/main/host. The generated nxpkg metadata remains part of this example because it exercises the same ELF loader through the installed-package path; it is test-only output rather than a general package repository feature. A shared package-fixture generator can still be considered separately without coupling this validation PR to that larger design.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants