Skip to content

[feature,test:gsoc26] Reproducible Test Infrastructure #415

@atif09

Description

@atif09

Is your feature request related to a problem? Please describe.

The extraction pipeline introduced across the previous phases depend on real firmware images to be tested meaningfully. Mocked tests alone cannot verify that fwtool succeeds on a real sysupgrade image, that the DTB fallback works on a real sunxi image, or that x86 and armsr fail cleanly. Without a reproducible way to acquire and cache these images across CI and developer machines, the test suite is either unreliable or introduces a heavy network dependency on every run

Describe the solution you'd like
I would like to finalize the test infrastructure for the extraction pipeline

  1. Commit SHA256 checksums for a representative set of real OpenWrt builds to the repository:
  • sysupgrade images for fwtool happy path verification (ath79, mediatek)
  • sunxi for DTB fallback verification
  • x86 and armsr for graceful failure verification
  1. Implement 2 dedicated unit tests to verify that ValidationError (for invalid file headers or Rootfs-only images) is correctly applied and surfaced across these two entry points:
  • Admin UI: Ensuring the clean() method or save_model() returns a user-friendly error in the Django change-form
  • REST API: Ensuring the FirmwareImageSerializer catches the error and returns a structured and relevant response.
  1. Implement dedicated test cases for OOM enforcement during decompression and task timeouts to ensure the Celery worker handles stalled extractions gracefully

  2. Add tests to verify the state machine transitions and field unlock logic when the 'enable manual edit' admin action is triggered and during the failed cases where the fields are unlocked automatically

  3. Implement setUpClass download and caching logic: on first run, images are downloaded from downloads.openwisp.io , verified against committed checksums, and cached locally. Subsequent run skip the download if the checksum matches. Network failure skips the test gracefully

  4. Work towards replacing downloaded image with minimal fake images committed directly to the repository, same format and metadata placement as real images but near-zero content, eliminating the network dependency entirely.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestgsoc-ideaIssues part of Google Summer of Code project

Type

Projects

Status

ToDo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions