You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+233-3Lines changed: 233 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,7 @@ Simple Tape File System (STFS), a file system for tapes and tar files.
5
5
⚠️ STFS has not yet been audited! While we try to make it as secure as possible, it has not yet undergone a formal security audit by a third party. Please keep this in mind if you use it for security-critical applications. ⚠️
@@ -42,6 +43,8 @@ You can find binaries for more operating systems and architectures on [GitHub re
42
43
43
44
## Usage
44
45
46
+
> Please note that this is only a short overview and does not explain all configuration options. To get more info on available commands or options, use `--help`.
47
+
45
48
### 1. Generating Keys with `stfs keygen`
46
49
47
50
While not strictly required, it is recommended to generate keys to sign and encrypt your data on tape. There are multiple methods available; `PGP` and [age](https://github.com/FiloSottile/age) for encryption, and `PGP` as well as [Minisign](https://github.com/aead/minisign) for signatures. In most cases, using age for encryption and PGP for signatures is the best option. To generate the appropriate keys, run the following; make sure to save the keys in a secure location and use a secure password:
@@ -119,15 +122,242 @@ For more information, see the [servers reference](#servers).
119
122
120
123
### 4. Using Optimized Operations with `stfs operation`
121
124
125
+
While the file system API is convenient because of its similarity to most filesystems, it also can't be used without a write cache. While this isn't an issue for most applications, it requires you to have a disk that is at least as large as the largest file you want to add to the tape. To get around these limitations, STFS also provides a `tar`-like interface for interacting with the tape. Please note that these operations should be used carefully, as the usual checks (such as checking if a parent directory exists before adding files to it) don't apply.
126
+
127
+
First, initialize an empty tape:
128
+
129
+
```shell
130
+
# Use `-d /dev/nst0` for your primary tape drive instead
Full CRUD support is implemented, so you can `delete`, `move`, `restore` and `update` files like this as well. For example, to restore `pkg/tape/write.go`, run the following:
166
+
167
+
```shell
168
+
# Use `-d /dev/nst0` for your primary tape drive instead
For more information, see the [operations reference](#operations).
188
+
122
189
### 5. Managing the Index with `stfs inventory`
123
190
191
+
For similar reasons as described in [Using Optimized Operations with `stfs operation`](#4-using-optimized-operations-with-stfs-operation), it can make sense to take advantage of the index in order to quickly find a file or directory. For example, to list all files in the `pkg` directory, run the following:
It is also possible to get information on a single file or directory using `stfs inventory stat`. For more information, see the [inventory reference](#inventory).
214
+
124
215
### 6. Recovering Data with `stfs recovery`
125
216
217
+
In case of unfinished write operations, sudden power losses or other forms of data corruption, the integrated recovery tools can help. For example, to query a tape starting from a specific record and block, use `stfs query`:
218
+
219
+
```shell
220
+
# Use `-d /dev/nst0` for your primary tape drive instead
It is also possible to restore a broken index from scratch with `stfs recovery index`. For more information, see the [recovery reference](#recovery).
258
+
126
259
### 7. Managing the Drive with `stfs drive`
127
260
261
+
STFS can also manage the physical tape drive directly without having to rely on external tools. For example, to eject a tape from the drive, use `stfs drive eject`:
262
+
263
+
```shell
264
+
$ stfs drive eject \
265
+
-d /dev/nst0
266
+
```
267
+
268
+
It is also possible to get the current tape position with `stfs drive tell`. For more information, see the [drive management reference](#drive-management).
269
+
128
270
### 8. Embedding STFS with `fs.STFS`
129
271
130
-
🚀 **That's it!** We hope you enjoy using stfs.
272
+
STFS at its core provides quite a few public APIs, but the easiest way to embed it is to use it's provided [`afero.FS implementation`](https://github.com/spf13/afero). This makes it possible to easily swap out the filesystem implementation with a native one, layer caching implementations and decouple your storage layer.
273
+
274
+
Using this API is fairly straightforward:
275
+
276
+
```go
277
+
// ...
278
+
stfs:= fs.NewSTFS(
279
+
readOps,
280
+
writeOps,
281
+
282
+
config.MetadataConfig{
283
+
Metadata: metadataPersister,
284
+
},
285
+
286
+
config.CompressionLevelFastestKey,
287
+
func() (cache.WriteCache, func() error, error) {
288
+
return cache.NewCacheWrite(
289
+
*writeCacheFlag,
290
+
config.WriteCacheTypeFile,
291
+
)
292
+
},
293
+
false,
294
+
false,
295
+
296
+
func(hdr *config.Header) {
297
+
l.Trace("Header transform", hdr)
298
+
},
299
+
l,
300
+
)
301
+
302
+
root, err:= stfs.Initialize("/", os.ModePerm)
303
+
if err != nil {
304
+
panic(err)
305
+
}
306
+
307
+
fs, err:= cache.NewCacheFilesystem(
308
+
stfs,
309
+
root,
310
+
config.NoneKey,
311
+
0,
312
+
"",
313
+
)
314
+
if err != nil {
315
+
panic(err)
316
+
}
317
+
```
318
+
319
+
You can now use the Afero APIs to interact with the filesystem; if you've worked with Go's `fs` package before, they should be very familiar:
320
+
321
+
```go
322
+
log.Println("stat /")
323
+
324
+
stat, err:= fs.Stat("/")
325
+
if err != nil {
326
+
panic(err)
327
+
}
328
+
329
+
log.Println("Result of stat /:", stat)
330
+
331
+
log.Println("create /test.txt")
332
+
333
+
file, err:= fs.Create("/test.txt")
334
+
if err != nil {
335
+
panic(err)
336
+
}
337
+
338
+
log.Println("Result of create /test.txt:", file)
339
+
340
+
log.Println("writeString /test.txt")
341
+
342
+
n, err:= file.WriteString("Hello, world!")
343
+
if err != nil {
344
+
panic(err)
345
+
}
346
+
347
+
log.Println("Result of writeString /test.txt:", n)
348
+
349
+
iferr:= file.Close(); err != nil {
350
+
panic(err)
351
+
}
352
+
353
+
// ...
354
+
```
355
+
356
+
Note that STFS also implements `afero.Symlinker`, so symlinks are available as well.
357
+
358
+
For more information, check out the [Go API](https://pkg.go.dev/github.com/pojntfx/stfs) and take a look at the provided [examples](./examples), utilities, services and tests in the package for examples.
359
+
360
+
🚀 **That's it!** We hope you enjoy using STFS.
131
361
132
362
## Reference
133
363
@@ -370,14 +600,14 @@ All command line arguments described above can also be set using environment var
370
600
371
601
To contribute, please use the [GitHub flow](https://guides.github.com/introduction/flow/) and follow our [Code of Conduct](./CODE_OF_CONDUCT.md).
372
602
373
-
To build and start a development version of stfs locally, run the following:
603
+
To build and start a development version of STFS locally, run the following:
374
604
375
605
```shell
376
606
$ git clone https://github.com/pojntfx/stfs.git
377
607
$ cd stfs
378
608
$ make depend
379
609
$ make && sudo make install
380
-
$ stfs serve ftp -d /tmp/drive.tar -m /tmp/dev.sqlite # Now point Nautilus to `ftp://localhost:1337`
610
+
$ stfs serve ftp -d /tmp/drive.tar -m /tmp/dev.sqlite # Now point your file explorer to `ftp://localhost:1337`
381
611
```
382
612
383
613
Have any questions or need help? Chat with us [on Matrix](https://matrix.to/#/#stfs:matrix.org?via=matrix.org)!
0 commit comments