Enable mirror hammer to use an existing log as a source.#1062
Conversation
4bcece9 to
6bd2c30
Compare
7ff34d8 to
8dbbd65
Compare
8dbbd65 to
a6990d7
Compare
| c.hammer.randomReaders.Shrink(ctx) | ||
| c.hammer.fullReaders.Shrink(ctx) | ||
| c.hammer.writers.Shrink(ctx) | ||
| if c.hammer != nil { |
There was a problem hiding this comment.
Does it mean there will be no way to control the QPS copying entries from the existing log to the mirror server?
There was a problem hiding this comment.
Correct - AFAIK the mirror client has been implemented to send requests as quickly as possible :)
These rate controls never controlled the client send rate, they only ever controlled the rate at which the hammer adds things to the source log; if the mirroring process pauses for a while, then the moment it resumes it'll go as fast as possible.
| return nil, nil, nil, fmt.Errorf("failed to create HTTP fetcher: %v", err) | ||
| } | ||
| default: | ||
| lr = client.FileFetcher{Root: logURL} |
There was a problem hiding this comment.
Would the logURL := strings.ToLower(logURL) on L392 cause any issue if the logURL contains some uppercase value?
There was a problem hiding this comment.
Good spot, restructured.
| mirrorURL multiStringFlag | ||
|
|
||
| sourceLogURL = flag.String("source_log_url", "", "URL of the log to mirror, or empty if using a locally created log.") | ||
| sourceLogPubKeyPath = flag.String("source_log_pubkey_path", "", "Path to the public key of the log to mirror. Required if --source_log_url is set.") |
There was a problem hiding this comment.
Shall we align the naming of this flag with the log_private_key flag below? Maybe call it source_log_public_key?
There was a problem hiding this comment.
I've made them consistent, and also renamed the local log flags such that it's a little clearer they relate to this and not some other usage.
a6990d7 to
bfc1f23
Compare
bfc1f23 to
8cde6e5
Compare
| if appender != nil { | ||
| hammer, ha = newHammer(ctx, appender, logReader, tracker) | ||
| ha.Run(ctx) | ||
| hammer.Run(ctx) |
There was a problem hiding this comment.
Only "Standalone Hammer Log mode" will call hammer.Run, so the tracker checkpoint size is never updated in the "Replication mode".
If --leaf_write_goal flag is provided in the replication mode, the hammer will never stop.
The TUI tracker information will not be updated in the replication mode. Is that intended? If so, we can add the logic to enforce replication mode to non-TUI.
There was a problem hiding this comment.
I've added a workaround to the TUI mode, it can probably be improved, but at least it'll show the checkpoint size now.
2 participants
This PR enables an existing log to optionally be used as the source of entries when hammering a mirror.
Towards #945