aync write read - more than 20000 read not supported

[dwarikap]

Hi ,

my sample code objective is asyn write and read (aiocassandra). Both the cases it could be millions. But failing to do so. Especial if i go for selection and looping back large set of data say 100000.

testCRUDasync.txt

Also it gives me -
1.
cqlsh> select * from test.dummytbl;

u'key' | u'col1' | u'col2'
--------+---------+---------

(0 rows)
Failed to format value u'key' : 'unicode' object has no attribute 'formatter'
Failed to format value u'col1' : 'unicode' object has no attribute 'formatter'
1 more decoding errors suppressed.

Current time - End - session : 1551876594703
2019-03-06 18:19:55,200 [DEBUG] aiocassandra: Paginator is closed, cleared in-memory 0 records

[root@vm-9 cassandra_poc]# cqlsh
Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})

your help is really appreciated.

I have attached the file here.

Regards,
Dwarika

[FedirAlifirenko]

Hi @dwarikap
I see that there are no errors neither with aiocassandra nor cassandra packages.
You aren't wait the result of future object (returned by session.execute_async(batch)) in your example and as result you aren't see any output from aiocassandra paginator query.

If you'll wait the result of future like session.execute_async(batch).result() you'll get results from aiocassandra paginator query (actually not with batch_size=60000 as in your example, see info below ).
Batch query with provided size 60000 exceeds the limit and triggers the following error (I've tested with cassandra:3.11 docker image with 5000 as batch size):

ERROR [Native-Transport-Requests-1] 2019-05-01 19:02:19,926 BatchStatement.java:295 - Batch for [test.dummytbl] is of size 108.386KiB, exceeding specified threshold of 50.000KiB by 58.386KiB. (see batch_size_fail_threshold_in_kb)

Try to set this value to 1000 or less and you will get the results.