Skip to content

⭐ Add ZFS resources to OS provider#6784

Merged
tas50 merged 1 commit intomainfrom
tas50/zfs
Mar 11, 2026
Merged

⭐ Add ZFS resources to OS provider#6784
tas50 merged 1 commit intomainfrom
tas50/zfs

Conversation

@tas50
Copy link
Copy Markdown
Member

@tas50 tas50 commented Mar 4, 2026

Summary

  • Add zfs, zfs.pool, and zfs.dataset resources to the OS provider
  • Pools expose health, sizeBytes, capacity, dedup ratio, and auto-expand/replace/trim settings
  • Datasets cover filesystems, volumes, snapshots, and bookmarks with compression, quotas, encryption, and mount info
  • Both zfs.pool and zfs.dataset support init(name) for targeted lookup and lazy properties() for the full property set
  • zfs.dataset.snapshots() filters child snapshots from the cached dataset list
  • All byte-valued fields use a Bytes suffix for clarity (e.g., sizeBytes, usedBytes, quotaBytes)

Example queries

zfs.pools { name health sizeBytes freeBytes }
zfs.datasets.where(type == "filesystem") { name usedBytes mountpoint }
zfs.pool(name: "rpool").properties
zfs.dataset(name: "rpool/data").snapshots { name creation }

Test plan

  • Parser unit tests (13 tests covering pool list, dataset list, properties, edge cases)
  • Interactive verification on a ZFS system: mql run local -c "zfs.pools { name health sizeBytes }"
  • Verify zfs.pool(name: "...") init lookup works
  • Verify zfs.dataset(name: "...").snapshots filters correctly

@github-actions

This comment has been minimized.

@tas50 tas50 force-pushed the tas50/zfs branch 2 times, most recently from b122013 to eb11565 Compare March 4, 2026 05:26
@tas50
Copy link
Copy Markdown
Member Author

tas50 commented Mar 4, 2026

> zfs.version
zfs.version: "zfs-2.4.0-pve1"
> zfs.datasets{*}
zfs.datasets: [
  0: {
    properties: {
      aclinherit: "restricted"
      aclmode: "discard"
      acltype: "off"
      atime: "on"
      available: "24637811257600"
      canmount: "on"
      casesensitivity: "sensitive"
      checksum: "on"
      compression: "on"
      compressratio: "1.00"
      context: "none"
      copies: "1"
      createtxg: "1"
      creation: "1601010375"
      dedup: "off"
      defcontext: "none"
      devices: "on"
      direct: "standard"
      dnodesize: "legacy"
      encryption: "off"
      exec: "on"
      filesystem_count: "none"
      filesystem_limit: "none"
      fscontext: "none"
      guid: "18418074912290193914"
      keyformat: "none"
      keylocation: "none"
      logbias: "latency"
      logicalreferenced: "13986708672000"
      logicalused: "13987104441344"
      longname: "off"
      mlslabel: "none"
      mounted: "yes"
      mountpoint: "/storage"
      nbmand: "off"
      normalization: "none"
      objsetid: "54"
      omvzfsplugin:uuid: "ead3f27b-aa30-4b89-a4d7-1ecc0f46c312"
      overlay: "on"
      pbkdf2iters: "0"
      prefetch: "all"
      primarycache: "all"
      quota: "0"
      readonly: "off"
      recordsize: "131072"
      redundant_metadata: "all"
      refcompressratio: "1.00"
      referenced: "13968403506624"
      refquota: "0"
      refreservation: "0"
      relatime: "on"
      reservation: "0"
      rootcontext: "none"
      secondarycache: "all"
      setuid: "on"
      sharenfs: "off"
      sharesmb: "off"
      snapdev: "hidden"
      snapdir: "hidden"
      snapshot_count: "none"
      snapshot_limit: "none"
      special_small_blocks: "0"
      sync: "standard"
      type: "filesystem"
      used: "13969649766144"
      usedbychildren: "1246259520"
      usedbydataset: "13968403506624"
      usedbyrefreservation: "0"
      usedbysnapshots: "0"
      utf8only: "off"
      version: "5"
      volmode: "default"
      vscan: "off"
      written: "13968403506624"
      xattr: "sa"
      zoned: "off"
    }
    quotaBytes: 0
    referencedBytes: 13968403506624
    type: "filesystem"
    reservationBytes: 0
    usedBytes: 13969649766144
    mounted: true
    origin: ""
    mountpoint: "/storage"
    name: "storage"
    snapshots: []
    recordsizeBytes: 131072
    encryption: "off"
    availableBytes: 24637811257600
    compression: "on"
    creation: 2020-09-24 22:06:15 -0700 PDT
    compressratio: 1
  }
]
> zfs.pools{*}
zfs.pools: [
  0: {
    readonly: false
    vdevs: [
      0: zfs.pool.vdev name="raidz2-0" type="raidz" state="ONLINE" numDevices=4
    ]
    autoexpand: true
    autotrim: false
    health: "ONLINE"
    sizeBytes: 79989470920704
    percentUsed: 36
    properties: {
      allocated: "28840567271424"
      altroot: "-"
      ashift: "12"
      autoexpand: "on"
      autoreplace: "off"
      autotrim: "off"
      bcloneratio: "1.00"
      bclonesaved: "0"
      bcloneused: "0"
      bootfs: "-"
      cachefile: "-"
      capacity: "36"
      checkpoint: "-"
      comment: "-"
      compatibility: "off"
      dedup_table_quota: "auto"
      dedup_table_size: "0"
      dedupratio: "1.00"
      delegation: "on"
      expandsize: "-"
      failmode: "wait"
      feature@allocation_classes: "enabled"
      feature@async_destroy: "enabled"
      feature@blake3: "enabled"
      feature@block_cloning: "enabled"
      feature@bookmark_v2: "enabled"
      feature@bookmark_written: "enabled"
      feature@bookmarks: "enabled"
      feature@device_rebuild: "enabled"
      feature@device_removal: "enabled"
      feature@draid: "enabled"
      feature@edonr: "enabled"
      feature@embedded_data: "active"
      feature@empty_bpobj: "enabled"
      feature@enabled_txg: "active"
      feature@encryption: "enabled"
      feature@extensible_dataset: "active"
      feature@fast_dedup: "enabled"
      feature@filesystem_limits: "enabled"
      feature@head_errlog: "active"
      feature@hole_birth: "active"
      feature@large_blocks: "enabled"
      feature@large_dnode: "enabled"
      feature@large_microzap: "enabled"
      feature@livelist: "enabled"
      feature@log_spacemap: "active"
      feature@longname: "enabled"
      feature@lz4_compress: "active"
      feature@multi_vdev_crash_dump: "enabled"
      feature@obsolete_counts: "enabled"
      feature@project_quota: "active"
      feature@raidz_expansion: "enabled"
      feature@redacted_datasets: "enabled"
      feature@redaction_bookmarks: "enabled"
      feature@redaction_list_spill: "enabled"
      feature@resilver_defer: "enabled"
      feature@sha512: "enabled"
      feature@skein: "enabled"
      feature@spacemap_histogram: "active"
      feature@spacemap_v2: "active"
      feature@userobj_accounting: "active"
      feature@vdev_zaps_v2: "active"
      feature@zilsaxattr: "active"
      feature@zpool_checkpoint: "enabled"
      feature@zstd_compress: "enabled"
      fragmentation: "4"
      free: "51148903649280"
      freeing: "0"
      guid: "5096186678844629762"
      health: "ONLINE"
      last_scrubbed_txg: "34958344"
      leaked: "0"
      listsnapshots: "off"
      load_guid: "12824753509653635113"
      multihost: "off"
      readonly: "off"
      size: "79989470920704"
      version: "-"
    }
    fragmentation: 4
    autoreplace: false
    dedupratio: 1
    allocatedBytes: 28840567271424
    guid: "5096186678844629762"
    name: "storage"
    freeBytes: 51148903649280
  }
]
> zfs.pools.first.vdevs.first{*}
zfs.pools.first.vdevs.first: {
  writeErrors: 0
  numDevices: 4
  devices: [
    0: zfs.pool.vdev name="ata-ST20000NM004E-3HR103_ZX210AAB" type="disk" state="ONLINE" numDevices=0
    1: zfs.pool.vdev name="ata-ST20000NM002C-3X6103_ZXA04ZF7" type="disk" state="ONLINE" numDevices=0
    2: zfs.pool.vdev name="ata-ST20000NM004E-3HR103_ZX22FLCW" type="disk" state="ONLINE" numDevices=0
    3: zfs.pool.vdev name="ata-ST20000NM004E-3HR103_ZX22EJJC" type="disk" state="ONLINE" numDevices=0
  ]
  readErrors: 0
  path: ""
  name: "raidz2-0"
  slowIos: 0
  checksumErrors: 0
  type: "raidz"
  state: "ONLINE"
}
zfs.pools.first.vdevs.first.devices.first: {
  path: "/dev/disk/by-id/ata-ST20000NM004E-3HR103_ZX210AAB-part1"
  checksumErrors: 0
  writeErrors: 0
  name: "ata-ST20000NM004E-3HR103_ZX210AAB"
  devices: []
  state: "ONLINE"
  slowIos: 13
  readErrors: 0
  type: "disk"
  numDevices: 0
}

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ZFS resource caching is broken because generated code was not re-run after id() methods were added, causing all pools/datasets to share an empty cache key.

Comment thread providers/os/resources/os.lr.go
Comment thread providers/os/resources/zfs.go Outdated
Comment thread providers/os/resources/zfs.go Outdated
@mondoo-code-review mondoo-code-review Bot dismissed their stale review March 4, 2026 05:28

Superseded by new review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resource queries will work correctly; parsing and caching logic are solid.

Comment thread providers/os/resources/zfs.go Outdated
Comment thread providers/os/resources/zfs.go Outdated
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS pool/dataset querying works correctly but command strings with variable names should use shell quoting consistent with codebase conventions.

Comment thread providers/os/resources/zfs.go Outdated
Comment thread providers/os/resources/zfs.go Outdated
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resource queries will be vulnerable to shell injection when fetching pool/dataset properties.

Comment thread providers/os/resources/zfs.go Outdated
Comment thread providers/os/resources/zfs.go Outdated
@mondoo-code-review mondoo-code-review Bot dismissed their stale review March 4, 2026 05:29

Superseded by new review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resources for pools and datasets are well-implemented; previous critical finding about missing id() calls is resolved.

Comment thread providers/os/resources/zfs.go
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 4, 2026

Test Results

5 418 tests  +15   5 414 ✅ +15   2m 17s ⏱️ +10s
  412 suites + 1       4 💤 ± 0 
   31 files   ± 0       0 ❌ ± 0 

Results for commit 726875b. ± Comparison against base commit 540cb1b.

♻️ This comment has been updated with latest results.

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vdev resources from different pools will collide in cache due to non-unique IDs

Comment thread providers/os/resources/zfs.go
@tas50 tas50 changed the title Add ZFS resources to OS provider ⭐ WIP: Add ZFS resources to OS provider Mar 4, 2026
@mondoo-code-review mondoo-code-review Bot dismissed their stale review March 5, 2026 21:20

Superseded by new review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resource queries may fail on multi-pool systems due to non-unique vdev IDs (already flagged), but otherwise the implementation is solid.

Comment thread providers/os/resources/zfs.go
Comment thread providers/os/resources/zfs.go
Comment thread providers/os/resources/zfs/zfs.go
@github-actions

This comment has been minimized.

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resource implementation for querying ZFS pools, datasets, and vdevs via MQL.

Comment thread providers/os/resources/zfs.go
Comment thread providers/os/resources/zfs.go Outdated
@github-actions

This comment has been minimized.

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resources for querying pools, datasets, vdevs, and snapshots via the OS provider

Comment thread providers/os/resources/zfs/zfs.go
Comment thread providers/os/resources/zfs/zfs.go
Comment thread providers/os/resources/zfs.go
@github-actions

This comment has been minimized.

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spell-check CI will break because required dictionary words were accidentally deleted

Comment thread .github/actions/spelling/expect.txt
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Add zfs, zfs.pool, and zfs.dataset resources for querying ZFS storage
pools and datasets (filesystems, volumes, snapshots, bookmarks).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling dictionary removes words still used in the codebase, which will break the spell-check CI job.

Comment thread .github/actions/spelling/expect.txt
Comment thread .github/actions/spelling/expect.txt
@tas50
Copy link
Copy Markdown
Member Author

tas50 commented Mar 11, 2026

/review

@mondoo-code-review mondoo-code-review Bot dismissed their stale review March 11, 2026 21:57

Superseded by new review

Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New ZFS resource definitions, parsing library, and generated code for querying ZFS pools, datasets, and vdevs via MQL.

@tas50 tas50 merged commit 29877db into main Mar 11, 2026
23 checks passed
@tas50 tas50 deleted the tas50/zfs branch March 11, 2026 21:59
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 11, 2026
@tas50 tas50 changed the title ⭐ WIP: Add ZFS resources to OS provider ⭐ Add ZFS resources to OS provider Mar 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant