Feat/txpool latency qps (#6)

* Refactor tx_pool_latency_analysis configuration to include QPS parameter and reorder fields for improved clarity

* change task config

* Enhance transaction latency analysis by implementing synchronized arrays for precise latency measurement and improving transaction sending logic. Added QPS-based transaction distribution and refined logging for better monitoring of transaction flow.

* Revert "Enhance transaction latency analysis by implementing synchronized arrays for precise latency measurement and improving transaction sending logic. Added QPS-based transaction distribution and refined logging for better monitoring of transaction flow."

This reverts commit 31355d6.

* refactoring, check p2p message vs tx

* initial refactoring (to complete)

* Refactor tx_pool_latency_analysis task configuration to use QPS and adjust parameters for improved performance and clarity

* solve concurrency problem

* small fixes

* check tx index range

* fix(get_task_result_api)

* fix(server)

* fix(execution-spec-tests-execute.yaml)

* fix(build-release)

* fix(build-release)

* feat(res/execution-spec-tests-styles.css)

* feat(test_run)

* refactor(build-master.yml): update Docker repository and improve wiki generation steps

* fix p2p event loop error condition

* fix(run_shell)

* lint

* lint(server)

* lint

* lint

* Update tx-pool-check-short.yaml: modify transaction pool analysis tasks to use tps instead of qps and adjust parameters for improved performance

* replace QPS with TPS

* fix loop boundary

* Update transaction generation logic to extend test duration and improve logging for negative sleep time

* Reduce transaction duration in tx-pool-check-short.yaml from 30 seconds to 10 seconds for improved testing efficiency

* Swap names and titles of transaction pool analysis tasks in tx-pool-check-short.yaml for clarity and consistency

* complete throughput task

* Rename transaction variable in tx pool throughput analysis task for clarity and consistency

* rename mean_throughput

* Add duplicated p2p event logging and counting in tx pool latency analysis

* Implement coordinated omission events counting in tx pool latency analysis

* Remove logging of duplicated p2p events in tx pool latency analysis

* Add TODO for timeout implementation in duplicated p2p event handling in tx pool latency analysis

* Implement coordinated omission events logging and output in tx pool throughput analysis

* Swap names and titles of latency and throughput analysis tasks in tx-pool-check-short.yaml for clarity.

* fix

* fix tx gen and parsing

* move exit check

* Swap names and titles of throughput and latency analysis tasks in tx-pool-check-short.yaml for consistency.

* add logs; more uniform code

* fix termination condition

* Enhance readUntil function to support context for timeout handling and update ReadTransactionMessages to accept an optional timeout parameter.

* Update error message in readUntil function to use a simplified format for timeout handling.

* Refactor tx-pool-check.yaml to update transaction pool latency and throughput configurations, changing parameters to tps and duration_s for consistency and clarity.

---------

Co-authored-by: Michelangelo Riccobene <michelangelo.riccobene@gmail.com>

Author: tosettil-polimi

.github/workflows/build-master.yml

@@ -1,3 +1,4 @@
+
 name: Build Release
 
 on:
@@ -18,133 +19,133 @@ jobs:
       ref: ${{ github.sha }}
       release: "v${{ inputs.version }}"
       docker: true
-      docker_repository: "ethpandaops/assertoor"
+      docker_repository: "noku-team/assertoor"
       docker_tag_prefix: "v${{ inputs.version }}"
       additional_tags: "['v${{ inputs.version }}','latest']"
     secrets:
       DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
       DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
-
+  
   create_release:
     name: Create Release
     needs: [build_binaries]
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
-        with:
-          fetch-depth: 100
-          ref: ${{ github.sha }}
-      - name: "Generate release changelog"
-        id: changelog
-        run: |
-          git fetch --tags
-          prev_tag=$(git tag --sort=-version:refname | grep -e "^v[0-9.]*$" | head -n 1)
-          echo "previous release: $prev_tag"
-          if [ "$prev_tag" ]; then
-            changelog=$(git log --oneline --no-decorate $prev_tag..HEAD)
-          else
-            changelog=$(git log --oneline --no-decorate)
-          fi
-          echo "changelog<<EOF" >> $GITHUB_OUTPUT
-          echo " - ${changelog//$'\n'/$'\n' - }" >> $GITHUB_OUTPUT
-          echo "EOF" >> $GITHUB_OUTPUT
+    - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      with:
+        fetch-depth: 100
+        ref: ${{ github.sha }}
+    - name: "Generate release changelog"
+      id: changelog
+      run: |
+        git fetch --tags
+        prev_tag=$(git tag --sort=-version:refname | grep -e "^v[0-9.]*$" | head -n 1)
+        echo "previous release: $prev_tag"
+        if [ "$prev_tag" ]; then
+          changelog=$(git log --oneline --no-decorate $prev_tag..HEAD)
+        else
+          changelog=$(git log --oneline --no-decorate)
+        fi
+        echo "changelog<<EOF" >> $GITHUB_OUTPUT
+        echo " - ${changelog//$'\n'/$'\n' - }" >> $GITHUB_OUTPUT
+        echo "EOF" >> $GITHUB_OUTPUT
 
     # download build artifacts
     - name: "Download build artifacts"
       uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
 
-      # create draft release
-      - name: Create latest release
-        uses: actions/create-release@0cb9c9b65d5d1901c1f53e5e66eaf4afd303e70e # v1.1.4
-        id: create_release
-        with:
-          draft: true
-          prerelease: false
-          release_name: "v${{ inputs.version }}"
-          tag_name: "v${{ inputs.version }}"
-          body: |
-            ### Changes
-            ${{ steps.changelog.outputs.changelog }}
-
-              ### Release Artifacts
-              Please read through the [wiki](https://github.com/ethpandaops/assertoor/wiki) for setup & configuration instructions.
-              | Release File  | Description |
-              | ------------- | ------------- |
-              | [assertoor_${{ inputs.version }}_windows_amd64.zip](https://github.com/ethpandaops/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_windows_amd64.zip) | assertoor executables for windows/amd64 |
-              | [assertoor_${{ inputs.version }}_linux_amd64.tar.gz](https://github.com/ethpandaops/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_linux_amd64.tar.gz) | assertoor executables for linux/amd64 |
-              | [assertoor_${{ inputs.version }}_linux_arm64.tar.gz](https://github.com/ethpandaops/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_linux_arm64.tar.gz) | assertoor executables for linux/arm64 |
-              | [assertoor_${{ inputs.version }}_darwin_amd64.tar.gz](https://github.com/ethpandaops/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_darwin_amd64.tar.gz) | assertoor executable for macos/amd64 |
-              | [assertoor_${{ inputs.version }}_darwin_arm64.tar.gz](https://github.com/ethpandaops/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_darwin_arm64.tar.gz) | assertoor executable for macos/arm64 |
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
-
-      # generate & upload release artifacts
-      - name: "Generate release package: assertoor_${{ inputs.version }}_windows_amd64.zip"
-        run: |
-          cd assertoor_windows_amd64
-          zip -r -q ../assertoor_${{ inputs.version }}_windows_amd64.zip .
-      - name: "Upload release artifact: assertoor_${{ inputs.version }}_windows_amd64.zip"
-        uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
-        with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }}
-          asset_path: ./assertoor_${{ inputs.version }}_windows_amd64.zip
-          asset_name: assertoor_${{ inputs.version }}_windows_amd64.zip
-          asset_content_type: application/octet-stream
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
-
-      - name: "Generate release package: assertoor_${{ inputs.version }}_linux_amd64.tar.gz"
-        run: |
-          cd assertoor_linux_amd64
-          tar -czf ../assertoor_${{ inputs.version }}_linux_amd64.tar.gz .
-      - name: "Upload release artifact: assertoor_${{ inputs.version }}_linux_amd64.tar.gz"
-        uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
-        with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }}
-          asset_path: ./assertoor_${{ inputs.version }}_linux_amd64.tar.gz
-          asset_name: assertoor_${{ inputs.version }}_linux_amd64.tar.gz
-          asset_content_type: application/octet-stream
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
+    # create draft release
+    - name: Create latest release
+      uses: actions/create-release@0cb9c9b65d5d1901c1f53e5e66eaf4afd303e70e # v1.1.4
+      id: create_release
+      with:
+        draft: true
+        prerelease: false
+        release_name: "v${{ inputs.version }}"
+        tag_name: "v${{ inputs.version }}"
+        body: |
+          ### Changes
+          ${{ steps.changelog.outputs.changelog }}
 
-      - name: "Generate release package: assertoor_${{ inputs.version }}_linux_arm64.tar.gz"
-        run: |
-          cd assertoor_linux_arm64
-          tar -czf ../assertoor_${{ inputs.version }}_linux_arm64.tar.gz .
-      - name: "Upload release artifact: assertoor_${{ inputs.version }}_linux_arm64.tar.gz"
-        uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
-        with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }}
-          asset_path: ./assertoor_${{ inputs.version }}_linux_arm64.tar.gz
-          asset_name: assertoor_${{ inputs.version }}_linux_arm64.tar.gz
-          asset_content_type: application/octet-stream
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
+          ### Release Artifacts
+          Please read through the [wiki](https://github.com/noku-team/assertoor/wiki) for setup & configuration instructions.
+          | Release File  | Description |
+          | ------------- | ------------- |
+          | [assertoor_${{ inputs.version }}_windows_amd64.zip](https://github.com/noku-team/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_windows_amd64.zip) | assertoor executables for windows/amd64 |
+          | [assertoor_${{ inputs.version }}_linux_amd64.tar.gz](https://github.com/noku-team/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_linux_amd64.tar.gz) | assertoor executables for linux/amd64 |
+          | [assertoor_${{ inputs.version }}_linux_arm64.tar.gz](https://github.com/noku-team/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_linux_arm64.tar.gz) | assertoor executables for linux/arm64 |
+          | [assertoor_${{ inputs.version }}_darwin_amd64.tar.gz](https://github.com/noku-team/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_darwin_amd64.tar.gz) | assertoor executable for macos/amd64 |
+          | [assertoor_${{ inputs.version }}_darwin_arm64.tar.gz](https://github.com/noku-team/assertoor/releases/download/v${{ inputs.version }}/assertoor_${{ inputs.version }}_darwin_arm64.tar.gz) | assertoor executable for macos/arm64 |
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
 
-      - name: "Generate release package: assertoor_${{ inputs.version }}_darwin_amd64.tar.gz"
-        run: |
-          cd assertoor_darwin_amd64
-          tar -czf ../assertoor_${{ inputs.version }}_darwin_amd64.tar.gz .
-      - name: "Upload release artifact: assertoor_${{ inputs.version }}_darwin_amd64.tar.gz"
-        uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
-        with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }}
-          asset_path: ./assertoor_${{ inputs.version }}_darwin_amd64.tar.gz
-          asset_name: assertoor_${{ inputs.version }}_darwin_amd64.tar.gz
-          asset_content_type: application/octet-stream
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
+    # generate & upload release artifacts
+    - name: "Generate release package: assertoor_${{ inputs.version }}_windows_amd64.zip"
+      run: |
+        cd assertoor_windows_amd64
+        zip -r -q ../assertoor_${{ inputs.version }}_windows_amd64.zip .
+    - name: "Upload release artifact: assertoor_${{ inputs.version }}_windows_amd64.zip"
+      uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./assertoor_${{ inputs.version }}_windows_amd64.zip
+        asset_name: assertoor_${{ inputs.version }}_windows_amd64.zip
+        asset_content_type: application/octet-stream
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
+    
+    - name: "Generate release package: assertoor_${{ inputs.version }}_linux_amd64.tar.gz"
+      run: |
+        cd assertoor_linux_amd64
+        tar -czf ../assertoor_${{ inputs.version }}_linux_amd64.tar.gz .
+    - name: "Upload release artifact: assertoor_${{ inputs.version }}_linux_amd64.tar.gz"
+      uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./assertoor_${{ inputs.version }}_linux_amd64.tar.gz
+        asset_name: assertoor_${{ inputs.version }}_linux_amd64.tar.gz
+        asset_content_type: application/octet-stream
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
+    
+    - name: "Generate release package: assertoor_${{ inputs.version }}_linux_arm64.tar.gz"
+      run: |
+        cd assertoor_linux_arm64
+        tar -czf ../assertoor_${{ inputs.version }}_linux_arm64.tar.gz .
+    - name: "Upload release artifact: assertoor_${{ inputs.version }}_linux_arm64.tar.gz"
+      uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./assertoor_${{ inputs.version }}_linux_arm64.tar.gz
+        asset_name: assertoor_${{ inputs.version }}_linux_arm64.tar.gz
+        asset_content_type: application/octet-stream
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
+    
+    - name: "Generate release package: assertoor_${{ inputs.version }}_darwin_amd64.tar.gz"
+      run: |
+        cd assertoor_darwin_amd64
+        tar -czf ../assertoor_${{ inputs.version }}_darwin_amd64.tar.gz .
+    - name: "Upload release artifact: assertoor_${{ inputs.version }}_darwin_amd64.tar.gz"
+      uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./assertoor_${{ inputs.version }}_darwin_amd64.tar.gz
+        asset_name: assertoor_${{ inputs.version }}_darwin_amd64.tar.gz
+        asset_content_type: application/octet-stream
+      env:
+        GITHUB_TOKEN: ${{ github.token }}
 
-      - name: "Generate release package: assertoor_${{ inputs.version }}_darwin_arm64.tar.gz"
-        run: |
-          cd assertoor_darwin_arm64
-          tar -czf ../assertoor_${{ inputs.version }}_darwin_arm64.tar.gz .
-      - name: "Upload release artifact: assertoor_${{ inputs.version }}_darwin_arm64.tar.gz"
-        uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
-        with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }}
-          asset_path: ./assertoor_${{ inputs.version }}_darwin_arm64.tar.gz
-          asset_name: assertoor_${{ inputs.version }}_darwin_arm64.tar.gz
-          asset_content_type: application/octet-stream
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
+    - name: "Generate release package: assertoor_${{ inputs.version }}_darwin_arm64.tar.gz"
+      run: |
+        cd assertoor_darwin_arm64
+        tar -czf ../assertoor_${{ inputs.version }}_darwin_arm64.tar.gz .
+    - name: "Upload release artifact: assertoor_${{ inputs.version }}_darwin_arm64.tar.gz"
+      uses: actions/upload-release-asset@e8f9f06c4b078e705bd2ea027f0926603fc9b4d5 # v1.0.2
+      with:
+        upload_url: ${{ steps.create_release.outputs.upload_url }}
+        asset_path: ./assertoor_${{ inputs.version }}_darwin_arm64.tar.gz
+        asset_name: assertoor_${{ inputs.version }}_darwin_arm64.tar.gz
+        asset_content_type: application/octet-stream
+      env:
+        GITHUB_TOKEN: ${{ github.token }}

.github/workflows/build-release.yml

@@ -396,7 +396,7 @@ func (t *Task) storeTaskResults(summaryFile *resultFile, resultDir string) {
 						TaskID: uint64(t.ctx.Index),
 						Type:   "result",
 						Index:  fileIdx,
-						Name:   file.Name(),
+						Name:   fmt.Sprintf("%v%v", prefix, file.Name()),
 						Size:   uint64(len(data)),
 						Data:   data,
 					}); err3 != nil {

pkg/coordinator/tasks/run_shell/task.go

@@ -9,36 +9,39 @@ The `tx_pool_latency_analysis` task evaluates latency of transaction processing
 - **`privateKey`**:
   The private key of the account to use for sending transactions.
 
-- **`txCount`**:
-  The total number of transactions to send.
+- **`tps`**:
+  The total number of transactions to send in one second.
+
+- **`duration_s`**:
+  The test duration (the number of transactions to send is calculated as `tps * duration_s`).
 
 - **`measureInterval`**:
   The interval at which the script logs progress (e.g., every 100 transactions).
 
-- **`highLatency`**:
-  The expected average transaction latency in milliseconds.
-
-- **`failOnHighLatency`**:
-  Whether the task should fail if the measured latency exceeds `highLatency`.
-
-
 ### Outputs
 
 - **`tx_count`**:
   The total number of transactions sent.
 
-- **`avg_latency_ms`**:
-  The average latency of the transactions in milliseconds.
+- **`min_latency_mus`**:
+  The min latency of the transactions in microseconds.
+
+- **`max_latency_mus`**:
+  The max latency of the transactions in microseconds.
+
+- **`tx_pool_latency_hdr_plot`**:
+  The HDR plot of the transaction pool latency.
 
 ### Defaults
 
 ```yaml
 - name: tx_pool_latency_analysis
   config:
-    txCount: 15000
+    tps: 100
+    duration_s: 10  
     measureInterval: 1000
-    highLatency: 5000
-    failOnHighLatency: false
   configVars:
-    privateKey: "tx_pool_latency_analysis"
+    privateKey: "walletPrivkey"
 ```
+
+

pkg/coordinator/tasks/tx_pool_latency_analysis/README.md

@@ -3,19 +3,17 @@ package txpoollatencyanalysis
 type Config struct {
 	PrivateKey string `yaml:"privateKey" json:"privateKey"`
 
-	TxCount							 int   `yaml:"txCount" json:"txCount"`
+	TPS                  int   `yaml:"tps" json:"tps"`
+	Duration_s           int   `yaml:"duration_s" json:"duration_s"`
 	MeasureInterval      int   `yaml:"measureInterval" json:"measureInterval"`
-	HighLatency		       int64 `yaml:"highLatency" json:"highLatency"`
-	FailOnHighLatency    bool  `yaml:"failOnHighLatency" json:"failOnHighLatency"`
 	SecondsBeforeRunning int64 `yaml:"secondsBeforeRunning" json:"secondsBeforeRunning"`
 }
 
 func DefaultConfig() Config {
 	return Config{
-		TxCount: 							1000,
+		TPS:                  100,
+		Duration_s:           60,
 		MeasureInterval:      100,
-		HighLatency:					5000, // in microseconds
-		FailOnHighLatency:    true,
 		SecondsBeforeRunning: 0,
 	}
 }