add mexc exchange to the libx package #3262
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s, periodic background refreshes
…ndling
Corrects the total USD balance calculation displayed for MEXC accounts
on the /mm page. The previous implementation yielded near-zero values
when CEX balances were large due to two issues:
1. **Incorrect Precision:** The `getCoinPrecision` helper function
incorrectly inferred asset precision from unreliable API fields
(e.g., `WithdrawMin`), leading to wrong conversion factors when
converting API balance strings to atomic units (`uint64`).
2. **JS Precision Loss:** Large `uint64` atomic balances could exceed
JavaScript's `Number.MAX_SAFE_INTEGER`, causing precision loss when
the data was transferred to the frontend and parsed as standard numbers.
This commit addresses these issues by:
- Replacing the flawed `getCoinPrecision` call in `mexc.go` with logic
that uses the asset's `UnitInfo` (via `asset.UnitInfo` and
`inferDecimals`) to determine the correct precision factor.
- Changing the Go `libxc.ExchangeBalance` struct fields (`Available`,
`Locked`) to `string` type (removing the `,string` json tag as it's
no longer needed and caused double-quoting).
- Updating Go code (`mexc.go`, `binance.go`, `mm.go`) to correctly
convert between `uint64` and `string` for these fields using `strconv`.
- Updating the corresponding TypeScript `ExchangeBalance` interface in
`registry.ts` to expect strings.
- Updating the frontend calculation (`mm.ts`) to use `parseFloat` on
the received balance strings.
- Fixing TypeScript compilation errors in `mmsettings.ts`, `mmutil.ts`,
and `tsconfig.json` related to the type changes and ES version features.
These changes ensure that large atomic balances are correctly calculated
in Go, transferred as strings to avoid JS precision issues, and parsed
correctly in the frontend for accurate USD value summation.
This commit addresses several issues related to the MEXC CEX
implementation, focusing on improving
order book synchronization reliability and WebSocket connection stability.
Problem Sequence & Fixes:
1. **Asset Info Failure for Tokens:** The initial error involved
`SubscribeMarket` failing with "failed to get asset info for factors"
when dealing with token assets (e.g., USDT.POLYGON). This occurred
because `asset.Info` was incorrectly used instead of `asset.UnitInfo`,
as `asset.Info` only checks base asset drivers.
- Fixed by changing the calls in `SubscribeMarket` to use `asset.UnitInfo`.
2. **Market Stream Connection Race:** After fixing the asset info issue,
a "market stream not connected" error appeared during market
subscription. This was likely a race condition where the subscription
attempt occurred before the WebSocket connection (lazily started on
first subscribe) was fully established.
- Fixed by proactively starting the market stream connection goroutine
within the main `Connect` method, similar to the user data stream.
- Simplified `SubscribeMarket` to assume the connection process was
already initiated.
3. **Order Book Sync Failures (Initial & Reconnect):** The "orderbook not
synced" error persisted. Analysis showed the initial snapshot fetch
could fail, or the book could fail to resync after a WebSocket
reconnection because the reconnect event didn't trigger a new
snapshot fetch.
- Added retry logic with exponential backoff to the initial
`syncOrderbook` call in the `mexcOrderBook.run` goroutine.
- Modified `handleMarketConnectEvent` to signal the `run` goroutine
(via `marketSubResyncChan`) to perform a resync whenever the
WebSocket status becomes `Connected`.
- Added logic to the `mexcOrderBook.run` select loop to listen for
resync signals and call `syncOrderbook`.
4. **Order Book Sync Timing:** The "orderbook not synced" error still
occurred intermittently, particularly after restarts. This was traced
to the arbitrage logic potentially calling `VWAP` *before* the initial
or resync `syncOrderbook` process could complete.
- Modified `VWAP` to wait on the `mexcOrderBook.syncChan` for up to
20 seconds if the book is found to be initially unsynced, giving the
sync process time to finish.
5. **User Data Stream Stability:** Frequent disconnects (`close 1005`)
were observed on the User Data stream (`UserWS`), likely due to
server-side idle timeouts.
- Decreased the `PingWait` configuration for the UserWS connection
from 60s to 30s to trigger library-level PINGs sooner.
- Implemented proactive client-side PINGs sent every 25 seconds on
the UserWS within the `connectUserStream` loop to ensure constant
activity from the client's perspective. Problem reduced, but persists (todo)
The `Trade` function previously used `asset.Info` to retrieve conversion factors for the base and quote assets. However, `asset.Info` only supports base-chain assets and fails when called with a token asset ID (e.g., USDT.POLYGON ID 966004). This resulted in a "failed to get asset info for factors" error, preventing trades involving tokens on MEXC. This commit replaces the `asset.Info` calls within the `Trade` function with `asset.UnitInfo`, which correctly retrieves information for both base assets and registered tokens. This enables successful trade execution for markets involving token assets on the MEXC CEX adapter.
Refactors the `cancelExpiredCEXTrades` function in the arbitrage market maker (`mm_arb_market_maker.go`) to prevent repeated cancellation attempts on CEX orders that have already been filled or cancelled. Previously, when the bot attempted to cancel an expired CEX order that had already reached a terminal state (filled/cancelled), the CEX adapter (`mexc.go` or `binance.go`) would correctly handle the API error (e.g., MEXC's -2011) and return `nil` error to the caller. However, the `cancelExpiredCEXTrades` function incorrectly interpreted this `nil` error solely as a successful cancellation and would prematurely `return` from its loop without removing the inactive trade ID from its internal `a.cexTrades` tracking map. This caused the bot to repeatedly attempt to cancel the same inactive order in subsequent epochs. This commit refactors `cancelExpiredCEXTrades`: - It now first identifies all potentially expired trades based on epoch. - It then iterates through these identified trades, attempting cancellation. - If `CancelTrade` returns `nil` (indicating successful cancellation OR that the order was already inactive), the trade ID is now correctly removed from the `a.cexTrades` map (using proper locking). - The inaccurate log message "was cancelled before it was filled" is replaced with a more accurate message reflecting the outcome. - The premature `return` within the loop is removed, ensuring all identified expired trades are processed. Additionally, a related change was made in the MEXC adapter (`mexc.go`) to log the specific API error for already-filled/cancelled orders (-2011) at the Debug level instead of Warning, reducing log noise for this expected race condition scenario.
Decouples WebSocket connection lifecycle from bot lifecycle to ensure market data continues to be processed even when bots are stopped. Key changes: - Add active flag to track books with no subscribers - Modify UnsubscribeMarket to keep books in memory when inactive - Update SubscribeMarket to reactivate existing books - Ensure book methods support inactive but synced books - Maintain proper reconnection handling for inactive books
Added comprehensive shutdown mechanism to ensure clean resource cleanup when context is cancelled. This includes proper closing of WebSocket connections, stopping timers, cancelling contexts, and deleting listen keys to prevent resource leaks during application shutdown.
Prevent orderbooks from being incorrectly marked as unsynced during WebSocket disconnections by: - Removing automatic unsyncing on disconnect in handleMarketConnectEvent - Adding timestamp tracking for updates to detect actual stale books - Only triggering resyncs after prolonged periods without updates - Increasing buffer sizes for event notification channels - Improving connection state handling during reconnections
Implemented the handleDepthUpdate function in mexc.go to process market depth updates from the MEXC WebSocket API. The implementation: - Extracts market symbols from incoming messages - Parses depth update data (bids/asks) from the JSON payload - Verifies subscription status for the market - Routes updates to the appropriate order book with thread safety - Handles edge cases like missing symbols and invalid data Added a comprehensive unit test that verifies correct functionality and error handling. This implementation maintains accurate order book data for market making bots across multiple subscribed markets.
Fixed the MEXC order book synchronization to properly handle depth updates according to MEXC documentation. Key changes: - Added update buffering during initial sync - Relaxed strict sequence requirements to handle out-of-order updates - Enhanced order book sync process to follow MEXC's recommended approach - Added proper processing of buffered updates after fetching a snapshot
The WsConn interface in client/comms/wsconn.go only provides IsDown() method and not IsConnected(). This change fixes build errors by replacing all instances of m.marketStream.IsConnected() with !m.marketStream.IsDown() to properly check connection status throughout the MEXC exchange implementation.
Token Asset ID Handling: - Add special handling for USDT tokens (including USDT.polygon) - Allow the bot to properly connect to MEXC markets with variant tokens - Use a consistent 10^6 conversion factor for USDT regardless of network - Implement the fix across SubscribeMarket, Book, and other key functions These changes ensure proper communication between the DEX and MEXC despite their different token identification schemes, particularly for network-specific stablecoins like USDT.polygon.
This commit fixes a nil interface conversion error that occurred when accessing the tokenIDs map in the MEXC integration. Specifically: - Implemented a robust getCoinInfo function that properly fetches and processes coin information from MEXC API, building accurate token ID mappings and withdrawal minimums - Improved the getMarkets function to require proper tokenIDs data before proceeding with market matching, adding better validation and error handling - Added a mapMEXCCoinNetworkToDEXSymbol helper function to correctly map MEXC coins and networks to DEX symbols These changes ensure the MEXC exchange integration properly initializes all required data structures before attempting to use them, preventing the nil pointer dereference that was causing application crashes during startup. The implementation was adapted from a working reference implementation, ensuring compatibility with the MEXC API.
implementing correct protobuf message parsing for market depth data.
Improve MEXC exchange integration by implementing correct protobuf message parsing for market depth data. This replaces the previous manual binary parsing with proper protobuf unmarshaling. Changes include: - Use protobuf-generated Go code in the mexctypes package - Handle various websocket message formats/framing - Add appropriate scaling factors for values: * Price scaling: 10^6 (e.g., 12913000 → 12.913 USDT) * Quantity scaling: 10^8 (e.g., 986000000 → 9.86 coins) - Support both protobuf and JSON message types - Provide better error handling and reporting
|
Thanks for this! I will be reviewing soon. Regarding the ping issue, I've experienced the same thing with Coinbase. For the fixes you added in the UI, the libxc interface, and the Binance code, would you mind putting those in a separate PR? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add libx/mexc adapter for the market making functionality
Protobuf is used for the orderbook depth updates since they do not deliver any orderbook depth updates trough their json websocket and are going to retire that endpoint in august 2025, see: https://www.mexc.com/support/articles/17827791522393
The repository for the mexc proto files can be found at https://github.com/mexcdevelop/websocket-proto
The changes on the underlying core client code, especially the front-end files were made to ensure a correct display of the CEX balance on the mm site exchange list. See ec428c9
Since i have no access to a binance api tests are needed to unesure the correct USD balance for binance is shown on the exchange list on the mm page. The changes from this specific commit break the current test harness.
wsconn manager expects websockets to receive ping/pongs in order to determine if a websocket connection is alive. The mexc protobuf endpoint sends ~4/s orderbook update messages but no pings, leading to unneccessary reconnect/resubscribes even on a stable data delivering connection.