Skip to content

Commit e3b2ed2

Browse files
nabonnabon
authored
Merge pull request #27 from dist-sys/fine-grained-latency
Change timeunit in latency calculation.
2 parents 72da733 + 7381dc8 commit e3b2ed2

12 files changed

+147
-122
lines changed

‎README.md

+8-8
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,18 @@ Below is an execution result sample.
88

99
```
1010
-----Publisher-----
11-
Maximum throughput[msg/s]: 53068
12-
Average throughput[msg/s]: 49894.57
11+
Maximum throughput [msg/s]: 53068
12+
Average throughput [msg/s]: 49894.571
1313
Number of published messages: 349262
14-
Per second throughput[msg/s]: 44460, 47558, 52569, 53068, 51041, 51583, 48983
14+
Per second throughput [msg/s]: 44460, 47558, 52569, 53068, 51041, 51583, 48983
1515
1616
-----Subscriber-----
17-
Maximum throughput[msg/s]: 53050
18-
Average throughput[msg/s]: 49891.14
17+
Maximum throughput [msg/s]: 53050
18+
Average throughput [msg/s]: 49891.142
1919
Number of received messages: 349238
20-
Per second throughput[msg/s]: 44399, 47587, 52566, 53050, 51078, 51575, 48983
21-
Maximum latency[ms]: 24
22-
Average latency[ms]: 1.39
20+
Per second throughput [msg/s]: 44399, 47587, 52566, 53050, 51078, 51575, 48983
21+
Maximum latency [ms]: 24.812
22+
Average latency [ms]: 1.396
2323
```
2424

2525
MQTTLoader is licensed under the Apache License, Version2.0.

‎doc/usage_en.md

+17-16
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# MQTTLoader usage (v0.8.2)
1+
# MQTTLoader usage (v0.8.3)
22
MQTTLoader is a load testing tool (client tool) for MQTT.
33
It supports both MQTT v5.0 and v3.1.1.
44
From v0.8.0, it supports TLS authentication.
@@ -13,7 +13,7 @@ Download the archive file (zip or tar) from: https://github.com/dist-sys/mqttloa
1313
Below is an example of downloading by using curl command.
1414

1515
```
16-
$ curl -OL https://github.com/dist-sys/mqttloader/releases/download/v0.8.2/mqttloader-0.8.2.zip
16+
$ curl -OL https://github.com/dist-sys/mqttloader/releases/download/v0.8.3/mqttloader-0.8.3.zip
1717
```
1818

1919
By extracting it, you can get the following files.
@@ -197,24 +197,24 @@ MQTTLoader displays results like the following on standard output.
197197

198198
```
199199
-----Publisher-----
200-
Maximum throughput[msg/s]: 18622
201-
Average throughput[msg/s]: 16666.666666666668
200+
Maximum throughput [msg/s]: 18622
201+
Average throughput [msg/s]: 16666.666
202202
Number of published messages: 100000
203-
Per second throughput[msg/s]: 11955, 16427, 18430, 18030, 18622, 16536
203+
Per second throughput [msg/s]: 11955, 16427, 18430, 18030, 18622, 16536
204204
205205
-----Subscriber-----
206-
Maximum throughput[msg/s]: 18620
207-
Average throughput[msg/s]: 16666.666666666668
206+
Maximum throughput [msg/s]: 18620
207+
Average throughput [msg/s]: 16666.666
208208
Number of received messages: 100000
209-
Per second throughput[msg/s]: 11218, 16414, 18426, 18026, 18620, 17296
210-
Maximum latency[ms]: 81
211-
Average latency[ms]: 42.23691
209+
Per second throughput [msg/s]: 11218, 16414, 18426, 18026, 18620, 17296
210+
Maximum latency [ms]: 81.838
211+
Average latency [ms]: 42.236
212212
```
213213
MQTTLoader counts the number of messages sent by publishers.
214214
If QoS level is set to 1 or 2, counting is done when receiving PUBACK or PUBCOMP respectively.
215215

216216
After completion, MQTTLoader calculates the maximum throughput, the average throughput, and the number of published messages.
217-
`Per second throughput[msg/s]` is the time series of throughputs per second.
217+
`Per second throughput [msg/s]` is the time series of throughputs per second.
218218

219219
By using the parameterse `ramp_up` and `ramp_down`, you can exclude the beginning and trailing data.
220220
If you set the following parameter settings for example, the beginning one second and the trailing one second are excluded.
@@ -242,10 +242,10 @@ Note that if the specified directory doesn't exist, it is newly created.
242242
The file `mqttloader_xxxxxxxx-xxxxxx.csv` has records like the following:
243243

244244
```
245-
1599643916416,ml-EeiE-p-00001,S,
246-
1599643916416,ml-EeiE-p-00000,S,
247-
1599643916419,ml-EeiE-s-00000,R,3
248-
1599643916422,ml-EeiE-p-00001,S,
245+
1599643916416823,ml-EeiE-p-00001,S,
246+
1599643916416882,ml-EeiE-p-00000,S,
247+
1599643916419123,ml-EeiE-s-00000,R,3165
248+
1599643916422982,ml-EeiE-p-00001,S,
249249
:
250250
:
251251
```
@@ -254,10 +254,11 @@ Each line, consists of comma-separeted values, indicates the following data.
254254
In the case that the event type is `R`, latency data follows.
255255

256256
```
257-
timestamp (Unix time in milliseconds), client ID, event type (S: send, R: receive), latency (in milliseconds)
257+
timestamp (Unix time in microseconds), client ID, event type (S: send, R: receive), latency (in microseconds)
258258
```
259259

260260
Although MQTTLoader outputs the measurement result to the console, you can use the above .csv file for further analysis.
261+
Note that the latency in the above file is in microseconds, whereas that in the console is in milliseconds with three digits after the decimal point.
261262

262263
---
263264
---

‎doc/usage_jp.md

+17-16
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# MQTTLoader 利用方法 (v0.8.2)
1+
# MQTTLoader 利用方法 (v0.8.3)
22
MQTTLoaderは、MQTT v5.0とv3.1.1に対応した負荷テストツール（クライアントツール）です。
33
v0.8.0から、ブローカとのTLS接続にも対応しました。
44

@@ -14,7 +14,7 @@ https://github.com/dist-sys/mqttloader/releases
1414
以下は、Curlコマンドを使ってダウンロードする場合の例です。
1515

1616
```
17-
$ curl -OL https://github.com/dist-sys/mqttloader/releases/download/v0.8.2/mqttloader-0.8.2.zip
17+
$ curl -OL https://github.com/dist-sys/mqttloader/releases/download/v0.8.3/mqttloader-0.8.3.zip
1818
```
1919

2020
ダウンロードしたファイルを解凍すると、以下のディレクトリ構造が得られます。
@@ -198,25 +198,25 @@ MQTTLoadは標準出力に以下のような測定結果の情報を出力しま
198198

199199
```
200200
-----Publisher-----
201-
Maximum throughput[msg/s]: 18622
202-
Average throughput[msg/s]: 16666.666666666668
201+
Maximum throughput [msg/s]: 18622
202+
Average throughput [msg/s]: 16666.666
203203
Number of published messages: 100000
204-
Per second throughput[msg/s]: 11955, 16427, 18430, 18030, 18622, 16536
204+
Per second throughput [msg/s]: 11955, 16427, 18430, 18030, 18622, 16536
205205
206206
-----Subscriber-----
207-
Maximum throughput[msg/s]: 18620
208-
Average throughput[msg/s]: 16666.666666666668
207+
Maximum throughput [msg/s]: 18620
208+
Average throughput [msg/s]: 16666.666
209209
Number of received messages: 100000
210-
Per second throughput[msg/s]: 11218, 16414, 18426, 18026, 18620, 17296
211-
Maximum latency[ms]: 81
212-
Average latency[ms]: 42.23691
210+
Per second throughput [msg/s]: 11218, 16414, 18426, 18026, 18620, 17296
211+
Maximum latency [ms]: 81.838
212+
Average latency [ms]: 42.236
213213
```
214214

215215
MQTTLoaderは、各publisherによるメッセージの送信をカウントします。
216216
QoSレベルが1または2の場合は、それぞれ、PUBACKおよびPUBCOMPを受信したタイミングでカウントされます。
217217

218218
測定が終了したら、MQTTLoaderはカウントしたメッセージ数を集計し、最大スループット、平均スループット、送信メッセージ数を計算します。
219-
`Per second throughput[msg/s]`は、スループット値の時間変化を秒単位で列挙したものです。
219+
`Per second throughput [msg/s]`は、スループット値の時間変化を秒単位で列挙したものです。
220220

221221
パラメータ`ramp_up`と`ramp_down`を用いると、測定開始直後と終了直前の一定秒数分を、集計対象データから除外することができます。
222222
例えば以下のように設定した場合、最初と最後の1秒間のデータは集計対象外となります。
@@ -244,10 +244,10 @@ subscriberに関しても、上記と同様にして、受信メッセージの
244244
このcsvファイルには、以下のようなデータが記録されます。
245245

246246
```
247-
1599643916416,ml-EeiE-p-00001,S,
248-
1599643916416,ml-EeiE-p-00000,S,
249-
1599643916419,ml-EeiE-s-00000,R,3
250-
1599643916422,ml-EeiE-p-00001,S,
247+
1599643916416823,ml-EeiE-p-00001,S,
248+
1599643916416882,ml-EeiE-p-00000,S,
249+
1599643916419123,ml-EeiE-s-00000,R,3165
250+
1599643916422982,ml-EeiE-p-00001,S,
251251
:
252252
:
253253
```
@@ -256,10 +256,11 @@ subscriberに関しても、上記と同様にして、受信メッセージの
256256
送受信種別が `R` の場合のみ、レイテンシも記載されます。
257257

258258
```
259-
タイムスタンプ（ミリ秒単位Unix時間）, クライアントID, 送受信種別（S: 送信, R: 受信）, レイテンシ（ミリ秒単位）
259+
タイムスタンプ（マイクロ秒単位Unix時間）, クライアントID, 送受信種別（S: 送信, R: 受信）, レイテンシ（マイクロ秒単位）
260260
```
261261

262262
MQTTLoaderは、測定結果のサマリをコンソールに出力しますが、追加の集計・分析を行いたい場合には上記のファイルを使ってください。
263+
なお、コンソールに出力されるレイテンシはミリ秒単位（小数点以下3桁まで）であるのに対し、上記ファイルのレイテンシはマイクロ秒単位である点に注意してください。
263264

264265
---
265266
---

‎src/main/java/mqttloader/Constants.java

+4-4
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
import java.text.SimpleDateFormat;
2020

2121
public class Constants {
22-
public static final String VERSION = "0.8.2";
22+
public static final String VERSION = "0.8.3";
2323
public static final String BROKER_PREFIX_TCP = "tcp://";
2424
public static final String BROKER_PREFIX_TLS = "ssl://";
2525
public static final String BROKER_PORT_TCP = "1883";
@@ -29,10 +29,10 @@ public class Constants {
2929
public static final String SUB_CLIENT_ID_PREFIX = "ml-"+HOST_ID+"-s-";
3030
public static final String PUB_CLIENT_ID_PREFIX = "ml-"+HOST_ID+"-p-";
3131
public static final Record STOP_SIGNAL = new Record();
32-
public static final int MILLISECOND_IN_NANO = 1000000;
32+
public static final int MICROSECOND_IN_NANO = 1000;
3333
public static final int MILLISECOND_IN_MICRO = 1000;
34-
public static final int SECOND_IN_NANO = 1000000000;
35-
public static final int SECOND_IN_MILLI = 1000;
34+
public static final long SECOND_IN_MILLI = 1000L;
35+
public static final long SECOND_IN_MICRO = 1000000L;
3636
public static final SimpleDateFormat DATE_FORMAT_FOR_LOG = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS z");
3737
public static final SimpleDateFormat DATE_FORMAT_FOR_FILENAME = new SimpleDateFormat("yyyyMMdd-HHmmss");
3838

‎src/main/java/mqttloader/Loader.java

+21-18
Original file line numberDiff line numberDiff line change
@@ -398,7 +398,7 @@ private void waitForMeasurement() {
398398
if(Util.getPropValueInt(Prop.NUM_SUB) > 0){
399399
timer = new Timer();
400400
int subTimeout = Util.getPropValueInt(Prop.SUB_TIMEOUT);
401-
timer.schedule(new RecvTimeoutTask(timer, subTimeout), subTimeout*1000);
401+
timer.schedule(new RecvTimeoutTask(timer, subTimeout), subTimeout*Constants.SECOND_IN_MILLI);
402402
}
403403

404404
int execTime = Util.getPropValueInt(Prop.EXEC_TIME);
@@ -451,14 +451,14 @@ private void calcResult() {
451451
while ((str = br.readLine()) != null) {
452452
StringTokenizer st = new StringTokenizer(str, ",");
453453
long timestamp = Long.valueOf(st.nextToken());
454-
String clientId = st.nextToken(); //client ID
454+
int elapsedTime = (int)((timestamp - Util.getEpochMicros(Loader.measurementStartTime))/Constants.SECOND_IN_MICRO);
455+
st.nextToken(); //client ID
455456
boolean isSend = st.nextToken().equals("S") ? true : false;
456-
int latency = -1;
457-
if (st.hasMoreTokens()) {
458-
latency = Integer.valueOf(st.nextToken());
457+
if(isSend){
458+
recorder.recordSendInMemory(elapsedTime);
459+
} else {
460+
recorder.recordReceiveInMemory(elapsedTime, Long.valueOf(st.nextToken()));
459461
}
460-
461-
recorder.recordInMemory(new Record(timestamp, clientId, isSend, latency));
462462
}
463463

464464
br.close();
@@ -481,7 +481,7 @@ private void calcResult() {
481481
TreeMap<Integer, Integer> sendThroughputs = recorder.getSendThroughputs();
482482
TreeMap<Integer, Integer> recvThroughputs = recorder.getRecvThroughputs();
483483
TreeMap<Integer, Long> latencySums = recorder.getLatencySums();
484-
TreeMap<Integer, Integer> latencyMaxs = recorder.getLatencyMaxs();
484+
TreeMap<Integer, Long> latencyMaxs = recorder.getLatencyMaxs();
485485

486486
int rampup = Util.getPropValueInt(Prop.RAMP_UP);
487487
int rampdown = Util.getPropValueInt(Prop.RAMP_DOWN);
@@ -504,21 +504,24 @@ private void calcResult() {
504504
System.out.println("-----Subscriber-----");
505505
printThroughput(recvThroughputs, false);
506506

507-
int maxLt = 0;
508-
double aveLt = 0;
507+
long maxLtMicros = 0;
508+
double aveLtMicros = 0;
509509
long numMsg = 0;
510510
for(int elapsedSecond: latencySums.keySet()) {
511-
if(latencyMaxs.get(elapsedSecond) > maxLt) {
512-
maxLt = latencyMaxs.get(elapsedSecond);
511+
if(latencyMaxs.get(elapsedSecond) > maxLtMicros) {
512+
maxLtMicros = latencyMaxs.get(elapsedSecond);
513513
}
514514
int numInSec = recvThroughputs.get(elapsedSecond);
515515
numMsg += numInSec;
516516
double aveInSec = (double)latencySums.get(elapsedSecond)/numInSec;
517-
aveLt = aveLt + ((aveInSec-aveLt)*numInSec)/numMsg;
517+
aveLtMicros = aveLtMicros + ((aveInSec-aveLtMicros)*numInSec)/numMsg;
518518
}
519519

520-
System.out.println("Maximum latency[ms]: "+maxLt);
521-
System.out.println("Average latency[ms]: "+String.format("%.2f", aveLt));
520+
double maxLtMillis = (double)maxLtMicros/Constants.MILLISECOND_IN_MICRO;
521+
double aveLtMillis = aveLtMicros/Constants.MILLISECOND_IN_MICRO;
522+
523+
System.out.println("Maximum latency [ms]: "+String.format("%.3f", maxLtMillis));
524+
System.out.println("Average latency [ms]: "+String.format("%.3f", aveLtMillis));
522525
}
523526

524527
/**
@@ -538,15 +541,15 @@ private void printThroughput(TreeMap<Integer, Integer> throughputs, boolean forP
538541
}
539542

540543
double aveTh = throughputs.size()>0 ? (double)sumMsg/throughputs.size() : 0;
541-
System.out.println("Maximum throughput[msg/s]: "+maxTh);
542-
System.out.println("Average throughput[msg/s]: "+String.format("%.2f", aveTh));
544+
System.out.println("Maximum throughput [msg/s]: "+maxTh);
545+
System.out.println("Average throughput [msg/s]: "+String.format("%.3f", aveTh));
543546
if(forPublisher){
544547
System.out.println("Number of published messages: "+sumMsg);
545548
}else{
546549
System.out.println("Number of received messages: "+sumMsg);
547550
}
548551

549-
System.out.print("Per second throughput[msg/s]: ");
552+
System.out.print("Per second throughput [msg/s]: ");
550553
for(int elapsedSecond: throughputs.keySet()){
551554
System.out.print(throughputs.get(elapsedSecond));
552555
if(elapsedSecond<throughputs.lastKey()){

‎src/main/java/mqttloader/Record.java

+28-13
Original file line numberDiff line numberDiff line change
@@ -16,31 +16,39 @@
1616

1717
package mqttloader;
1818

19+
import java.time.Duration;
20+
import java.time.Instant;
21+
import java.time.temporal.ChronoUnit;
22+
1923
public class Record {
20-
private long timestamp;
24+
private long sentEpochMicros;
25+
private Instant receivedTime;
2126
private String clientId;
2227
private boolean isSend;
23-
private int latency;
2428

2529
private boolean isStopSignal = false;
2630

27-
public Record(long timestamp, String clientId, boolean isSend, int latency) {
28-
this.timestamp = timestamp;
31+
public Record(long sentEpochMicros, Instant receivedTime, String clientId, boolean isSend) {
32+
this.sentEpochMicros = sentEpochMicros;
33+
this.receivedTime = receivedTime;
2934
this.clientId = clientId;
3035
this.isSend = isSend;
31-
this.latency = latency;
3236
}
3337

34-
public Record(long timestamp, String clientId, boolean isSend) {
35-
this(timestamp, clientId, isSend, -1);
38+
public Record(long sentEpochMicros, String clientId, boolean isSend) {
39+
this(sentEpochMicros, null, clientId, isSend);
3640
}
3741

3842
public Record() {
3943
this.isStopSignal = true;
4044
}
4145

42-
public long getTimestamp() {
43-
return timestamp;
46+
public long getSentEpochMicros() {
47+
return sentEpochMicros;
48+
}
49+
50+
public Instant getReceivedTime() {
51+
return receivedTime;
4452
}
4553

4654
public String getClientId() {
@@ -51,11 +59,18 @@ public boolean isSend() {
5159
return isSend;
5260
}
5361

54-
public int getLatency() {
55-
return latency;
56-
}
57-
5862
public boolean isStopSignal() {
5963
return isStopSignal;
6064
}
65+
66+
public long getLatency() {
67+
long latency = Util.getEpochMicros(receivedTime) - sentEpochMicros;
68+
if(latency < 0) {
69+
// If running MQTTLoader on multiple machines, a slight time error may cause a negative value of latency.
70+
Loader.LOGGER.fine("Negative value of latency is converted to zero.");
71+
return 0;
72+
} else {
73+
return latency;
74+
}
75+
}
6176
}

0 commit comments

Comments
 (0)