Merge development into main (#1511)

* Enhance MexTokenService to calculate and store token volumes for base and quote tokens in getAllMexTokensRaw method

* extend aws s3 config

* update configs

* update s3 configs

* fix transfers count with relayed txs (#1496)

* add support for new NFT events

* add maximum process retries

* add unit tests

* ferch token price from highest liquidity pool

* emit also deleteCacheKeys

* undo process reries

* fix tests

* Proposer fix (#1498)

* proposer fix

* proposer fix

* proposer fix

* fix getBlock

* fixes after review

Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>

* add execution order sorting using miniBlocksDetails from Elasticsearch (#1504)

* add execution order sorting using miniBlocksDetails from Elasticsearch

* fixes after review

* fixes after review

* fix userUndelegatedList field (#1507)

* fix userUndelegatedList field

* use ?? instead of [] to handle undefined case

* collections count subType required false fix (#1509)

* tokens supply format from plugin (#1505)

* fix collection set fields (#1510)

* fix collection set fields

* fixes after review

* add configurable response compression with gzip deflate support + update package json (#1512)

* add configurable response compresion

* set true compression for testnet env

* refactor processNFtCollections method

* update package

* update version of node

* update unit test yaml

* getNfts parallel execution

* compression flag false by default

* improve getSingleNFt

* getNftsCollection increase performance

* undo last commit

* Integrate last sdk nestjs (#1516)

* update to latest sdk version

* fixes for latest sdk version

* add package-lock

* fix unit tests

* Packages security issues 2 (#1517)

* fix packages

* add package-lock

* add missing packages

* remove unneeded package

* fix mongo connection

* Api 672 add timestampMs field into transaction response (#1518)

* Add timestampMs field to Transaction interface and class

* update specs

* update accounts spec

* Add normalizeTimestampMs method for timestamp conversion (#1519)

* Enhance account filtering by adding 'withBalance' query option to account endpoints and update ElasticIndexerHelper to support balance filtering. This allows users to filter accounts based on their balance status. (#1521)

* add events filtered by order (#1523)

* Refactor content type validation in NftMediaService to handle media types correctly by stripping parameters from the content type string. (#1527)

* use events index instead of logs (#1514)

* use events index instead of logs

* configurable migrated indices

* added protection for empty fields

* fix log address

* add transaction get unit tests (#1525)

---------

Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>

* fix ESDTTransfer duplicated events (#1526)

* use events index instead of logs

* configurable migrated indices

* added protection for empty fields

* fix log address

* add transaction get unit tests (#1525)

* fix duplicated events

---------

Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>

* update MetaESDT tokens (#1520)

* update MetaESDT tokens

* update imports

* nft collections es improvements (#1530)

* ES collections improvements

* getNfts improvements

* improve bulkProcessTransactions

* fixes

* fixes

* fixes

* refactor getSmartContractResultsRaw()

* improve nftCollection processing

* eslint fix

---------

Co-authored-by: cfaur09 <catalinfaurpaul@gmail.com>
Co-authored-by: Gabriel Matei <gabrielvmatei@gmail.com>
Co-authored-by: bogdan-rosianu <51945539+bogdan-rosianu@users.noreply.github.com>
Co-authored-by: bogdan-rosianu <bogdan.rosianu@yahoo.com>
Co-authored-by: Catalin Faur <52102171+cfaur09@users.noreply.github.com>
Co-authored-by: Gutica Stefan <123564494+GuticaStefan@users.noreply.github.com>

Author: 5 people

.github/workflows/lint.yml

@@ -15,7 +15,7 @@ jobs:
 
     strategy:
       matrix:
-        node-version: [16.x]
+        node-version: [18.x]
         # See supported Node.js release schedule at https://nodejs.org/en/about/releases/
 
     steps:

.github/workflows/unit.tests.yml

@@ -15,7 +15,7 @@ jobs:
 
     strategy:
       matrix:
-        node-version: [16.x]
+        node-version: [18.x]
         # See supported Node.js release schedule at https://nodejs.org/en/about/releases/
 
     steps:

config/config.devnet-old.yaml

@@ -70,6 +70,7 @@ aws:
   s3Secret: ''
   s3Bucket: 'devnet-old-media.elrond.com'
   s3Region: ''
+  s3Endpoint: ''
 urls:
   self: 'https://devnet-old-api.multiversx.com'
   elastic:

config/config.devnet.yaml

@@ -84,6 +84,8 @@ features:
     durationThresholdMs: 5000
     failureCountThreshold: 5
     resetTimeoutMs: 30000
+  elasticMigratedIndices:
+    logs: 'events'
   statusChecker:
     enabled: false
     thresholds:
@@ -122,6 +124,7 @@ aws:
   s3Secret: ''
   s3Bucket: 'devnet-media.elrond.com'
   s3Region: ''
+  s3Endpoint: ''
 urls:
   self: 'https://devnet-api.multiversx.com'
   elastic:
@@ -176,3 +179,8 @@ inflation:
 nftProcess:
   parallelism: 1
   maxRetries: 3
+compression:
+  enabled: true
+  level: 6
+  threshold: 1024
+  chunkSize: 16384

config/config.e2e.mainnet.yaml

@@ -85,6 +85,8 @@ features:
     durationThresholdMs: 5000
     failureCountThreshold: 5
     resetTimeoutMs: 30000
+  elasticMigratedIndices:
+    logs: 'events'
   statusChecker:
     enabled: false
     thresholds:
@@ -126,6 +128,7 @@ aws:
   s3Secret: ''
   s3Bucket: 'media.elrond.com'
   s3Region: ''
+  s3Endpoint: ''
 urls:
   self: 'http://localhost:3001'
   elastic:

config/config.mainnet.yaml

@@ -85,6 +85,8 @@ features:
     durationThresholdMs: 5000
     failureCountThreshold: 5
     resetTimeoutMs: 30000
+  elasticMigratedIndices:
+    logs: 'events'
   statusChecker:
     enabled: false
     thresholds:
@@ -126,6 +128,7 @@ aws:
   s3Secret: ''
   s3Bucket: 'media.elrond.com'
   s3Region: ''
+  s3Endpoint: ''
 urls:
   self: 'https://api.multiversx.com'
   elastic:
@@ -180,3 +183,8 @@ inflation:
 nftProcess:
   parallelism: 1
   maxRetries: 3
+compression:
+  enabled: true
+  level: 6
+  threshold: 1024
+  chunkSize: 16384

config/config.testnet.yaml

@@ -84,6 +84,8 @@ features:
     durationThresholdMs: 5000
     failureCountThreshold: 5
     resetTimeoutMs: 30000
+  elasticMigratedIndices:
+    logs: 'events'
   statusChecker:
     enabled: false
     thresholds:
@@ -125,6 +127,7 @@ aws:
   s3Secret: ''
   s3Bucket: 'testnet-media.elrond.com'
   s3Region: ''
+  s3Endpoint: ''
 urls:
   self: 'https://testnet-api.multiversx.com'
   elastic:
@@ -179,3 +182,8 @@ inflation:
 nftProcess:
   parallelism: 1
   maxRetries: 3
+compression:
+  enabled: true
+  level: 6
+  threshold: 1024
+  chunkSize: 16384