Skip to content

Commit bd4a38d

Browse files
authored
Port Fuzz tests (#78)
1 parent 655936b commit bd4a38d

5 files changed

Lines changed: 474 additions & 63 deletions

File tree

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +0,0 @@
1-
testdata/*

README.md

Lines changed: 66 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,18 @@ Compared to Golang's standard package `encoding/json`, `simdjson-go` is about 10
1414
## Features
1515

1616
`simdjson-go` is a validating parser, meaning that it amongst others validates and checks numerical values, booleans etc.
17-
Therefore these values are available as the appropriate `int` and `float64` representations after parsing.
17+
Therefore, these values are available as the appropriate `int` and `float64` representations after parsing.
1818

1919
Additionally `simdjson-go` has the following features:
2020

2121
- No 4 GB object limit
2222
- Support for [ndjson](http://ndjson.org/) (newline delimited json)
2323
- Pure Go (no need for cgo)
24+
- Object search/traversal.
25+
- In-place value replacement.
26+
- Remove object/array members.
27+
- Serialize parsed JSONas binary data.
28+
- Re-serialize parts as JSON.
2429

2530
## Requirements
2631

@@ -265,6 +270,65 @@ func findHondas(r io.Reader) {
265270

266271
More examples can be found in the examples subdirectory and further documentation can be found at [godoc](https://pkg.go.dev/github.com/minio/simdjson-go?tab=doc).
267272

273+
274+
### In-place Value Replacement
275+
276+
It is possible to replace a few, basic internal values.
277+
This means that when re-parsing or re-serializing the parsed JSON these values will be output.
278+
279+
Boolean (true/false) and null values can be freely exchanged.
280+
281+
Numeric values (float, int, uint) can be exchanged freely.
282+
283+
Strings can also be exchanged with different values.
284+
285+
Strings and numbers can be exchanged. However, note that there is no checks for numbers inserted as object keys,
286+
so if used for this invalid JSON is possible.
287+
288+
There is no way to modify objects, arrays, other than value types above inside each.
289+
It is not possible to remove or add elements.
290+
291+
To replace a value, of value referenced by an `Iter` simply call `SetNull`, `SetBool`, `SetFloat`, `SetInt`, `SetUInt`,
292+
`SetString` or `SetStringBytes`.
293+
294+
### Object & Array Element Deletion
295+
296+
It is possible to delete one or more elements in an object.
297+
298+
`(*Object).DeleteElems(fn, onlyKeys)` will call back fn for each key+ value.
299+
300+
If true is returned, the key+value is deleted. A key filter can be provided for optional filtering.
301+
If the callback function is nil all elements matching the filter will be deleted.
302+
If both are nil all elements are deleted.
303+
304+
Example:
305+
306+
```Go
307+
// The object we are modifying
308+
var obj *simdjson.Object
309+
310+
// Delete all entries where the key is "unwanted":
311+
err = obj.DeleteElems(func(key []byte, i Iter) bool {
312+
return string(key) == "unwanted")
313+
}, nil)
314+
315+
// Alternative version with prefiltered keys:
316+
err = obj.DeleteElems(nil, map[string]struct{}{"unwanted": {}})
317+
```
318+
319+
`(*Array).DeleteElems(fn func(i Iter) bool)` will call back fn for each array value.
320+
If the function returns true the element is deleted in the array.
321+
322+
```Go
323+
// The array we are modifying
324+
var array *simdjson.Array
325+
326+
// Delete all entries that are strings.
327+
array.DeleteElems(func(i Iter) bool {
328+
return i.Type() == TypeString
329+
})
330+
```
331+
268332
## Serializing parsed json
269333

270334
It is possible to serialize parsed JSON for more compact storage and faster load time.
@@ -493,64 +557,6 @@ BenchmarkUpdate_center/copy-32 1665 708717 ns/op 752.31
493557
BenchmarkUpdate_center/nocopy-32 2241 536027 ns/op 994.68 MB/s 2130 B/op 58 allocs/op
494558
```
495559

496-
### In-place Value Replacement
497-
498-
It is possible to replace a few, basic internal values.
499-
This means that when re-parsing or re-serializing the parsed JSON these values will be output.
500-
501-
Boolean (true/false) and null values can be freely exchanged.
502-
503-
Numeric values (float, int, uint) can be exchanged freely.
504-
505-
Strings can also be exchanged with different values.
506-
507-
Strings and numbers can be exchanged. However, note that there is no checks for numbers inserted as object keys,
508-
so if used for this invalid JSON is possible.
509-
510-
There is no way to modify objects, arrays, other than value types above inside each.
511-
It is not possible to remove or add elements.
512-
513-
To replace a value, of value referenced by an `Iter` simply call `SetNull`, `SetBool`, `SetFloat`, `SetInt`, `SetUInt`,
514-
`SetString` or `SetStringBytes`.
515-
516-
### Object & Array Element Deletion
517-
518-
It is possible to delete one or more elements in an object.
519-
520-
`(*Object).DeleteElems(fn, onlyKeys)` will call back fn for each key+ value.
521-
522-
If true is returned, the key+value is deleted. A key filter can be provided for optional filtering.
523-
If the callback function is nil all elements matching the filter will be deleted.
524-
If both are nil all elements are deleted.
525-
526-
Example:
527-
528-
```Go
529-
// The object we are modifying
530-
var obj *simdjson.Object
531-
532-
// Delete all entries where the key is "unwanted":
533-
err = obj.DeleteElems(func(key []byte, i Iter) bool {
534-
return string(key) == "unwanted")
535-
}, nil)
536-
537-
// Alternative version with prefiltered keys:
538-
err = obj.DeleteElems(nil, map[string]struct{}{"unwanted": {}})
539-
```
540-
541-
`(*Array).DeleteElems(fn func(i Iter) bool)` will call back fn for each array value.
542-
If the function returns true the element is deleted in the array.
543-
544-
```Go
545-
// The array we are modifying
546-
var array *simdjson.Array
547-
548-
// Delete all entries that are strings.
549-
array.DeleteElems(func(i Iter) bool {
550-
return i.Type() == TypeString
551-
})
552-
```
553-
554560
## Design
555561

556562
`simdjson-go` follows the same two stage design as `simdjson`.
@@ -619,9 +625,7 @@ For more information, see `TestStage2BuildTape` in `stage2_build_tape_test.go`.
619625
`simdjson-go` has been extensively fuzz tested to ensure that input cannot generate crashes and that output matches
620626
the standard library.
621627

622-
The fuzzers and corpus are contained in a separate repository at [github.com/minio/simdjson-fuzz](https://github.com/minio/simdjson-fuzz)
623-
624-
The repo contains information on how to run them.
628+
The fuzz tests are included as Go 1.18+ compatible tests.
625629

626630
## License
627631

0 commit comments

Comments
 (0)