Skip to content

Releases: DataDog/lading

0.4.0

Choose a tag to compare

@github-actions github-actions released this 12 Jul 21:22
40a2b63
Introduce `http_gen` (#52)

* Implement http_gen

This commit introduces a new program into the project, http_gen. It's similar to
file_gen except it works over HTTP. I've pretty substantially reworked the
configuration of file_gen and extracted common code from file_gen / http_gen,
though the two are still quite similar. I haven't tested http_gen against vector
yet and the configuration documentation must be updated.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

* Slightly trim depedency list

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

* Work on http_gen + vector bug

When I apply http_gen with DatadogLogs as variant vector will sometimes complain
about sheered payloads. I've added tests to confirm that lading_common generates
blocks that deserialize in this regard.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

* No longer coordinate omission

As of this commit http_gen will no longer coordinate its throughput with the
target. Total successful throughput is still limited by target capacity but
request generation now is a combination of internal clock and maximum connection
saturation.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

* correct test failure

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

* Also release http_gen binary

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.8

Choose a tag to compare

@github-actions github-actions released this 07 Jul 15:41
777f075
multi-artifact release

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.7

Choose a tag to compare

@github-actions github-actions released this 07 Jul 03:20
84262d3
Correct FoundationDB building (#44)

This commit addresses important mistakes in the FoundationDB generator. Firstly,
the size of the serialized FoundationDB json lines will cause some block sizes
to exceed the maximum bytes per second, a problem. Second, the names of fields
were not correct. Numbers are now quoted as in the FDB logs.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.6

Choose a tag to compare

@github-actions github-actions released this 06 Jul 23:24
e2b6bac
Add the ability to generate limited FoundationDB logs

This commit adds a limited ability to generate FoundationDB logs. There's more
variants to be added and any fields in the existing variants that have an
internal complex structure have been dropped.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.5

Choose a tag to compare

@github-actions github-actions released this 01 Jul 21:24
45511f1
Crash when a worker crashes (#38)

* Crash when a worker crashes

We have discovered situations where file_gen runs out of disk space and then
quietly stop. This happens when testing vector once vector falls behind.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

* Always truncate files on startup

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.4

Choose a tag to compare

@github-actions github-actions released this 01 Jul 19:34
cdd9dc0
Make generated data less blank (#37)

This commit ensures that the data we generate from Ascii and Json has a definite
size to it, primarily by selecting from a fixed array of sizes. This is... a
little less arbitrary than I would have liked but it's better than a file full
of blank lines.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.3

Choose a tag to compare

@github-actions github-actions released this 01 Jul 18:25
ed568b2
Use less CPU for slow targets (#36)

This commit re-introduces buffered writing, using less CPU when we have slow
targets. Once we have a need for GB/s range writing we'll need to figure
something else out but for now this is good enough. I'm more interested in
addressing generated data quality.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.2

Choose a tag to compare

@github-actions github-actions released this 01 Jul 17:41
88f4947
Use rayon to push faster byte block builds (#35)

This commit breaks up the block building into iterators so that we can bring
rayon to bear in their construction. This significantly reduces startup time.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.1

Choose a tag to compare

@github-actions github-actions released this 01 Jul 15:59
bf2cd3a
Fix file writing (#34)

I worked quite hard yesterday on the previous commit and convinced myself of a
non-thing. This resulted in there being no logs written to disk.

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>

0.3.0

Choose a tag to compare

@github-actions github-actions released this 01 Jul 15:03
286db2b
Pivot to the use of Arbitrary (#33)

This commit contains some interesting work. Firstly, I have replaced the
`buffer` code with the use of `Arbitrary`, reducing the project's dependence on
wonky custom generation code. I have also removed any `BufWriter` in the spin
loop, pivoting instead to the use of fixed MiB sized "blocks" that are
constructed ahead of time before entering the spin loop. This slows down
construction some -- hence the introduction of rayon -- but does significantly
improve on file_gen's throughput capability.

The config file has changed somewhat. We'll need to address the loss of line
length control but this work feels overlong as is.

Resolves #32
Resolves #31
REF #10

Signed-off-by: Brian L. Troutwine <brian@troutwine.us>