|
| 1 | +nxpkg - a package manager for NuttX application images |
| 2 | +======================================================== |
| 3 | + |
| 4 | + nxpkg installs, updates, uninstalls, and rolls back standalone loadable |
| 5 | + ELF applications (MODULE=m in an app's Makefile - see games/NXDoom or |
| 6 | + examples/calculator for real ports) from a package repository served |
| 7 | + over HTTP, onto local storage (an SD card on this board's config). |
| 8 | + system/nxstore is an LVGL touchscreen front-end for nxpkg; this |
| 9 | + document covers the repository/server side, which nxstore and the |
| 10 | + nxpkg CLI both consume identically. |
| 11 | + |
| 12 | + |
| 13 | +Repository layout |
| 14 | +================== |
| 15 | + |
| 16 | + A repository is nothing more than a directory of static files: |
| 17 | + |
| 18 | + <repo-root>/ |
| 19 | + index.json - the package catalog |
| 20 | + artifacts/<arch>/<chip>/<compat>/<name>/<version>/<artifact-file> |
| 21 | + icons/<arch>/<chip>/<compat>/<name>.bin - optional, see below |
| 22 | + |
| 23 | + index.json is a single JSON object with one "packages" array. Each |
| 24 | + entry is one installable version of one package: |
| 25 | + |
| 26 | + { |
| 27 | + "packages": [ |
| 28 | + { |
| 29 | + "name": "calc", |
| 30 | + "version": "6", |
| 31 | + "arch": "xtensa", |
| 32 | + "compat": "esp32s3-touch-lcd-7", |
| 33 | + "artifact": "artifacts/xtensa/esp32s3/esp32s3-touch-lcd-7/calc/6/calc", |
| 34 | + "sha256": "<64 hex chars>", |
| 35 | + "type": "elf", |
| 36 | + "description": "Simple 4-function calculator", |
| 37 | + "category": "Utilities", |
| 38 | + "icon": "icons/xtensa/esp32s3/esp32s3-touch-lcd-7/calc.bin" |
| 39 | + } |
| 40 | + ] |
| 41 | + } |
| 42 | + |
| 43 | + "artifact" and "icon" (both optional except "artifact") are resolved |
| 44 | + relative to the repository's own base URL (the URL index.json itself |
| 45 | + was fetched from, with the filename stripped) unless they're already |
| 46 | + a full http(s):// URL - see pkg_resolve_artifact_source()/ |
| 47 | + pkg_resolve_icon_source() in pkg_repo.c. That means the whole |
| 48 | + directory tree above can be moved, mirrored, or served from a |
| 49 | + completely different host without editing a single path in |
| 50 | + index.json, as long as the *relative* structure between index.json |
| 51 | + and artifacts/icons/ stays the same. |
| 52 | + |
| 53 | + "arch"/"compat" must match this board's CONFIG_ARCH ("xtensa") and |
| 54 | + CONFIG_ARCH_BOARD ("esp32s3-touch-lcd-7") exactly - see |
| 55 | + pkg_compat_check() in pkg_compat.c - so one repository can host |
| 56 | + packages for several different boards side by side; each board only |
| 57 | + ever installs the entries that match its own identity. |
| 58 | + |
| 59 | + |
| 60 | +What actually serves this - any static file host works |
| 61 | +======================================================== |
| 62 | + |
| 63 | + pkg_repo_fetch_url() (pkg_repo.c) does a plain HTTP GET with no auth, |
| 64 | + no custom headers, and no server-side logic required - it just needs |
| 65 | + a 2xx response. This means literally any static file server works: |
| 66 | + nginx or Apache with a DocumentRoot pointed at the repo directory, a |
| 67 | + one-line Python http.server, GitHub Pages, S3/R2/Cloudflare Pages, or |
| 68 | + a NAS's built-in web server. There is nothing nxpkg-specific to |
| 69 | + install on the server side. |
| 70 | + |
| 71 | + For local development/testing (what this project actually used while |
| 72 | + building and testing every package in this series on real hardware): |
| 73 | + |
| 74 | + cd /path/to/repo-root |
| 75 | + python3 -m http.server 8000 |
| 76 | + |
| 77 | + That's the entire "server setup." Confirm it's reachable from your |
| 78 | + own machine first: |
| 79 | + |
| 80 | + curl -sI http://<your-machine-ip>:8000/index.json |
| 81 | + |
| 82 | + sha256 verification (pkg_hash_file_sha256(), checked against the |
| 83 | + manifest's "sha256" field before an artifact is ever activated) |
| 84 | + already protects payload integrity in transit even over plain HTTP. |
| 85 | + Only confidentiality/availability would need HTTPS on top of this if |
| 86 | + that matters for a given deployment - nothing in the protocol assumes |
| 87 | + HTTP specifically, an https:// URL works exactly the same way. |
| 88 | + |
| 89 | + |
| 90 | +Populating a repository: tools/export_pkg_repo.py |
| 91 | +================================================== |
| 92 | + |
| 93 | + Building the JSON by hand is error-prone (sha256 digests, exact |
| 94 | + relative paths); tools/export_pkg_repo.py does it for you from a |
| 95 | + built binary: |
| 96 | + |
| 97 | + python3 tools/export_pkg_repo.py \ |
| 98 | + --arch xtensa --chip esp32s3 --compat esp32s3-touch-lcd-7 \ |
| 99 | + --package "calc:6:elf:/path/to/apps/bin/calc:Simple 4-function calculator:Utilities" \ |
| 100 | + /path/to/repo-root |
| 101 | + |
| 102 | + --package can be repeated to export several packages/versions in one |
| 103 | + call. The colon-separated spec is: |
| 104 | + |
| 105 | + <name>:<version>:<elf|shared-lib>:<source>[:<description>:<category>[:<icon>]] |
| 106 | + |
| 107 | + <source> is the built artifact on your machine (e.g. apps/bin/calc |
| 108 | + after a normal `make`). <icon> is optional: any image Pillow can open |
| 109 | + (PNG, JPEG, ...) - the tool resizes it to a fixed 48x48 and encodes it |
| 110 | + into the small raw RGB565 format system/nxstore's icon rendering |
| 111 | + expects (see nxstore_load_icon() in nxstore_main.c for the exact |
| 112 | + on-wire layout). Re-running the script is idempotent: it loads |
| 113 | + whatever index.json already exists in the target directory, merges in |
| 114 | + the packages you just passed (matched on name+version+arch+compat+ |
| 115 | + type), and rewrites the file - existing unrelated entries are left |
| 116 | + alone. |
| 117 | + |
| 118 | + This board's FAT-formatted SD card only supports short (8.3) file/ |
| 119 | + directory names with no long-name extension - any path component |
| 120 | + (package name, artifact leaf filename) over 8 characters before its |
| 121 | + extension fails to install with errno 22, not a clear error message. |
| 122 | + Keep names short (this is why this series' packages are named "calc", |
| 123 | + "brick", "cgol" rather than "calculator", "brickmatch", "conway") - |
| 124 | + this is a board/filesystem constraint, not an export_pkg_repo.py or |
| 125 | + nxpkg one, and would not apply to a board with a different storage |
| 126 | + filesystem. |
| 127 | + |
| 128 | + |
| 129 | +Pointing the board at your repository |
| 130 | +====================================== |
| 131 | + |
| 132 | + Two ways to get an index synced onto the board's local storage, |
| 133 | + useful in different situations: |
| 134 | + |
| 135 | + 1. One-shot from NSH, useful for iterating on a package during |
| 136 | + development (this is what building and testing every package in |
| 137 | + this series actually looked like): |
| 138 | + |
| 139 | + nxpkg sync http://<your-machine-ip>:8000/index.json |
| 140 | + nxpkg install <name> |
| 141 | + nxpkg update <name> # re-check for/install a newer version |
| 142 | + nxpkg list # show what's installed, current vs previous |
| 143 | + nxpkg rollback <name> # swap back to the previous version |
| 144 | + nxpkg remove <name> |
| 145 | + |
| 146 | + 2. Automatically at boot, via system/nxstore: the repo URL nxstore |
| 147 | + syncs from at startup is currently a single hardcoded string in |
| 148 | + board_nxstore_autostart() (boards/xtensa/esp32s3/ |
| 149 | + esp32s3-touch-lcd-7/src/esp32s3_bringup.c) - change that one line |
| 150 | + to point at a different host, or a public one, and every boot |
| 151 | + re-syncs the catalog automatically (falling back to whatever was |
| 152 | + last synced to local storage if the network/server is unreachable |
| 153 | + at boot - nxstore never blocks the app list on a failed sync). |
| 154 | + |
| 155 | + Development-network note: if your development machine's IP address |
| 156 | + changes (switching Wi-Fi networks, a new DHCP lease, a hotspot), |
| 157 | + re-run `nxpkg sync` with the new address - the board and the machine |
| 158 | + serving the repository just need to be able to reach each other over |
| 159 | + IP; nothing else about the setup changes. |
| 160 | + |
| 161 | + |
| 162 | +Concurrency note |
| 163 | +================= |
| 164 | + |
| 165 | + nxpkg protects a single package's own install/update/rollback against |
| 166 | + running twice at once (a per-package lock file), and separately |
| 167 | + protects the shared installed-packages database against being |
| 168 | + corrupted by two *different* packages' installs racing each other |
| 169 | + (see pkg_install_acquire_installed_lock() in pkg_install.c) - so it is |
| 170 | + safe to have, for example, system/nxstore's own background install |
| 171 | + worker and a directly-invoked `nxpkg install` running at the same |
| 172 | + time for different packages. Two operations on the *same* package |
| 173 | + name at the same time will have the second one fail with -EBUSY |
| 174 | + (surfaced as "package has an install/update in progress") rather than |
| 175 | + either one silently clobbering the other's work. |
0 commit comments