Skip to content

storage-configuration.md

Latest commit

 

History

History
170 lines (112 loc) · 7.65 KB

storage-configuration.md

File metadata and controls

170 lines (112 loc) · 7.65 KB
Raw
description This guide describes how to attach and configure sealing and permanent storage for Curio nodes

Storage Configuration

Each Curio node keeps track of defined storage locations in ~/.curio/storage.json (or $CURIO_REPO_PATH/storage.json) and uses ~/.curio path as default.

Upon initialization of a storage location, a <path-to-storage>/sectorstore.json file is created that contains the UUID assigned to this location, along with whether it can be used for sealing or storing.

Adding sealing storage location

Before adding your sealing storage location you will need to consider where the sealing tasks are going to be performed. This command must be run locally from the Curio node where you want to attach the storage.

curio cli storage attach --init --seal <PATH_FOR_SEALING_STORAGE>

OR

curio cli --machine <Machine IP:Port> storage attach --init --seal <PATH_FOR_SEALING_STORAGE>

{% hint style="info" %} The --machine flag accepts an <IP:Port> input, specifying the machine on which the command should be executed. The IP refers to the address of the machine where the Curio process is running, and the Port indicates the port on which the Curio process is listening. By default, Curio operates on port 12300. This flag enables Curio commands to be executed on remote nodes, facilitating centralized administration from a single node. {% endhint %}

Adding long-term storage location

Custom location for storing: After the sealing process is completed, sealed sectors are moved to the store location, which can be specified as follows:

curio cli storage attach --init --store <PATH_FOR_LONG_TERM_STORAGE>

OR

curio cli --machine <Machine IP:Port> storage attach --init --store <PATH_FOR_LONG_TERM_STORAGE>

This command must be run locally from the Curio node where you want to attach the storage. This location can be made of large capacity, albeit slower, spinning-disks.

Attach existing storage to Curio

The storage location used by lotus-miner or lotus-worker can be reused by the Curio cluster. It can be attached once the migrated lotus-miner or lotus-worker is already running a Curio service.

Common errors (from support)

permission denied when attaching storage

Example error:

  • mkdir '<path>/key': permission denied

Plain-English meaning:

  • Curio writes small metadata into the storage path when attaching/initializing it (for example sectorstorage.json and a key/ directory). If Curio can’t write there, attach will fail.

Fix checklist:

  1. Confirm the filesystem is mounted read-write.
  2. Ensure the Curio service user owns (or can write to) the directory.
  3. If you use curio cli --machine ... storage attach ..., the operation is executed on the remote machine, so permissions must be correct there.

Stale/ghost storage endpoints in storage list

Symptom:

  • curio cli storage list shows IDs/paths with errors like “all endpoints failed for remote storage ”.

Meaning:

  • Curio still remembers a storage entry, but the endpoint is no longer reachable (retired node, IP change, firewall).

Safe cleanup options:

  • Preferred: run curio cli storage detach --really-do-it <path> on the machine where that path was attached.
  • If the path is gone and detach can’t work: last resort is editing ~/.curio/storage.json (or $CURIO_REPO_PATH/storage.json).

Warning:

  • Removing the wrong entry can make sectors look “missing” until you re-attach the correct storage path.
curio cli storage attach <PATH_FOR_LONG_TERM_STORAGE>

OR

curio cli --machine <Machine IP:Port> storage attach <PATH_FOR_LONG_TERM_STORAGE>

Filter sector types

You can filter for what sector file types are allowed in each path by adjusting <path-to-storage>/sectorstore.json. You can also set these at attach time with --allow-types and --deny-types.

{
  "ID": "1626519a-5e05-493b-aa7a-0af71612010b",
  "Weight": 10,
  "CanSeal": false,
  "CanStore": true,
  "MaxStorage": 0,
  "Groups": [],
  "AllowTo": [],
  "AllowTypes": null,
  "DenyTypes": null
}

Valid values for AllowTypes and DenyTypes are:

"unsealed"      // unsealed sector files
"sealed"        // sealed sector replicas
"cache"         // sealing cache
"update"        // snap-deal update replica
"update-cache"  // snap-deal update cache
"piece"         // parked pieces: PDP payloads and market piece parking
"key"           // unseal keys

These values must be put in an array to be valid (e.g "AllowTypes": ["unsealed", "update-cache"]). Unknown values generate a warning and are ignored. A restart of the Curio node where this storage is attached is needed for sectorstore.json edits to take effect.

Separate sealed and unsealed

A very basic setup where you want to separate unsealed and sealed sectors could be achieved by:

  • Add "DenyTypes": ["unsealed"] to long-term storage path(s) where you want to store the sealed sectors.
  • Add "AllowTypes": ["unsealed"] to long-term storage path(s) where you want to store the unsealed sectors.

Setting only unsealed for AllowTypes will still allow cache and update-cache files to be placed in this storage path. If you want to completely deny all other types of sectors in this path, you can add additional valid values to the "DenyTypes" field.

{% hint style="info" %} If there are existing files with disallowed types in a storage path, those files will remain readable for PoSt/Retrieval. So the worst that can happen in case of misconfiguration in the storage path is that sealing tasks will get stuck waiting for storage to become available. {% endhint %}

Separate PDP / parked pieces from sealed storage

PDP payloads and market parked pieces are stored as file type "piece", not as sealed/unsealed sector files. They also use miner ID f00 (actor 0) internally; they do not use the SP miner ID. Miner AllowMiners / DenyMiners filters are for splitting disks between real miners, not for keeping PDP off sealed storage.

To keep parked pieces on their own disks:

  • Attach PDP / piece-park store paths with "AllowTypes": ["piece"] (and CanStore: true).
  • Attach sealed long-term paths with "DenyTypes": ["piece"].
curio cli storage attach --init --store --allow-types piece /mnt/pdp
curio cli storage attach --init --store --deny-types piece /mnt/sealed

Long-term PDP pieces allocate on store paths (--store / CanStore). A --seal-only path will not take PDP uploads or pulls.

This filter applies to all parked pieces. MK12/MK20 piece parking uses the same "piece" type, so those files share the piece-only disks. There is no path filter that selects PDP pieces and excludes market parking.

Segregating long-term storage per miner

Users can allocate long-term storage to specific miner IDs by specifying miner address strings in the sectorstore.json file. This configuration allows precise control over which miners can use the storage.

  • To allow specific miners, include their addresses in the AllowMiners array:

    "AllowMiners": ["t01000", "t01002"]

    This configuration permits only the listed miners (t01000 and t01002) to access the storage.

  • Similarly, to deny specific miners access to the storage, include their addresses in the DenyMiners array:

    "DenyMiners": ["t01003", "t01004"]

    In this example, miners with addresses t01003 and t01004 are explicitly denied access to the storage.

This dual configuration approach allows for flexible and secure management of storage access based on miner IDs.