Skip to content

re-running failed operations with TzKT backend is so slow, it hangs #674

@nicolasochem

Description

@nicolasochem

This was described on slack and I've seen it as well:

When TRD fails, the next run attempts to run the failed payments again. But this results in a lot of calls to tzkt api. Recently, it seems that this endpoint has been rate-limited, causing failure like the example below (taken on ghostnet TRD).

A workaround is to delete the failed payment folder (assuming it's a solid failure and not a partial payment). But it would be good to look into why it's querying every

│ 2023-06-22 21:39:11,754 - producer  - INFO - Summary 4 paid, 0 done, 0 injected, 9 fail, 1 avoided                                                                                                               │
│ Exception in thread producer:                                                                                                                                                                                    │
│ Traceback (most recent call last):                                                                                                                                                                               │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn                                                                                                                  │
│     conn = connection.create_connection(                                                                                                                                                                         │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection                                                                                                      │
│     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):                                                                                                                                       │
│   File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo                                                                                                                                           │
│     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):                                                                                                                                      │
│ socket.gaierror: [Errno -3] Try again                                                                                                                                                                            │
│                                                                                                                                                                                                                  │
│ During handling of the above exception, another exception occurred:                                                                                                                                              │
│                                                                                                                                                                                                                  │
│ Traceback (most recent call last):                                                                                                                                                                               │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen                                                                                                                │
│     httplib_response = self._make_request(                                                                                                                                                                       │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request                                                                                                          │
│     self._validate_conn(conn)                                                                                                                                                                                    │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn                                                                                                        │
│     conn.connect()                                                                                                                                                                                               │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/connection.py", line 363, in connect                                                                                                                    │
│     self.sock = conn = self._new_conn()                                                                                                                                                                          │
│   File "/app/.local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn                                                                                                                  │
│     raise NewConnectionError(                                                                                                                                                                                    │
│ urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7ff7a2064460>: Failed to establish a new connection: [Errno -3] Try again                                                 │
│                                                                                                                                                                                                                  │
│ During handling of the above exception, another exception occurred:          

Link to baker slack discussion: https://tezos-baking.slack.com/archives/CQ35AM8KE/p1685978794781209

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions