Start packaging latte

[vponomaryov]

For the moment we cook only docker images in the scope of our test automation framework.

Need to start packaging our latte fork.
It will be useful for people easily start using it without a need to use docker'ized solution.

[mykaul]

I actually think a container is a wonderful packaging and delivery mechanism. I just don't see the instructions on how to use it with docker/podman (or the Dockerfile, probably did not look for it hard enough).

[fruch]

in other stress tools we have binaries or RPM/DEB, I don't know if we need them all.

[mykaul]

in other stress tools we have binaries or RPM/DEB, I don't know if we need them all.

I don't see a need to. I do see a need for a container based solution for latte (I could have used one yesterday if I had it, had to use c-s instead - which worked fine)

[vponomaryov]

in other stress tools we have binaries or RPM/DEB, I don't know if we need them all.

I don't see a need to. I do see a need for a container based solution for latte (I could have used one yesterday if I had it, had to use c-s instead - which worked fine)

https://hub.docker.com/r/scylladb/hydra-loaders/tags?page=1&name=latte

[fruch]

in other stress tools we have binaries or RPM/DEB, I don't know if we need them all.

I don't see a need to. I do see a need for a container based solution for latte (I could have used one yesterday if I had it, had to use c-s instead - which worked fine)

the learning curve for using it is a bit more steep than, one need to write a rune script (there are examples, but it not a cli command you paste it, and as in c-s)

[mykaul]

in other stress tools we have binaries or RPM/DEB, I don't know if we need them all.

I don't see a need to. I do see a need for a container based solution for latte (I could have used one yesterday if I had it, had to use c-s instead - which worked fine)

the learning curve for using it is a bit more steep than, one need to write a rune script (there are examples, but it not a cli command you paste it, and as in c-s)

I know, was hoping to reuse an existing example. I only needed a simple 'fill a table'.

[mykaul]

If only I had the examples (and nano) in the container...

I think the Dockerfile is wrong - the entrypoint should be latte, and everything else should be external.
But anyway, success:

SUMMARY STATS ══════════════════════════════════════════════════════════════════════════════════════════════════════════════
    Elapsed time       [s]     60.003                                                                 
        CPU time       [s]     21.350                                                                 
 CPU utilisation       [%]        2.2                                                                 
          Cycles      [op]    3042862                                                                 
          Errors      [op]          0                                                                 
              └─       [%]        0.0                                                                 
        Requests     [req]    3042862                                                                 
              └─  [req/op]        1.0                                                                 
         Retries     [ret]          0                                                                 
              └─ [ret/req]    0.00000                                                                 
            Rows     [row]          0                                                                 
              └─ [row/req]        0.0                                                                 
     Concurrency     [req]         65 ± 1                                                             
              └─       [%]         51                                                                 
      Throughput    [op/s]      50712 ± 1027                                                          
              ├─   [req/s]      50712 ± 1027                                                          
              └─   [row/s]          0 ± 0                                                             
   Cycle latency      [ms]      2.144 ± 0.017                                                         
 Request latency      [ms]      2.142 ± 0.017                                                         

CYCLE LATENCY for run [ms]  ════════════════════════════════════════════════════════════════════════════════════════════════
          Min                   0.551 ± 0.086                                                         
           25                   1.948 ± 0.011                                                         
           50                   2.082 ± 0.015                                                         
           75                   2.238 ± 0.013                                                         
           90                   2.404 ± 0.046                                                         
           95                   2.802 ± 0.095                                                         
           98                   3.297 ± 0.217                                                         
           99                   3.879 ± 0.360                                                         
           99.9                 5.308 ± 0.664                                                         
           99.99                6.394 ± 1.568                                                         
          Max                   7.410 ± 1.568                                                         

[mykaul]

I need to re-create the container if I wish to just use latte, and turn it into ENTRYPOINT, then something like:

podman run --network host -v ./write.rn:/bin/write.rn -it scylladb/hydra-loaders:latte-0.28.1-scylladb  "/bin/latte load /bin/write.rn"

should work (I think).

[vponomaryov]

I need to re-create the container if I wish to just use latte, and turn it into ENTRYPOINT, then something like:

podman run --network host -v ./write.rn:/bin/write.rn -it scylladb/hydra-loaders:latte-0.28.1-scylladb  "/bin/latte load /bin/write.rn"

should work (I think).

The approach current docker image is designed to be used with is starting it with bash:

$ docker run --rm -it --net=host --cpus=2 \
-v %path-to%/rune-scripts:/rune-scripts:rw \
scylladb/hydra-loaders:latte-0.28.1-scylladb \
/bin/bash

And then use it the following way:

$ latte schema /rune-scripts/some-script.rn %IP%

$ latte run /rune-scripts/some-script.rn %IP% -q \
-P row_count=1000000 -P offset=0
-d 1000000 \
-r 2000 \
--retry-number=0 \
--retry-interval=1s,5s \
--request-timeout=5 \
--tag my-write
-f %function-name%
--generate-report

And then view/compare stress run results (based on the generated files triggered by --generate-report parameter) if needed:

$ latte list --tag my-write

$ latte list --tag my-read

[mykaul]

Thanks @vponomaryov . I understand the current approach, I don't think it's the right approach for either automation nor manual invocation - as the tool is NOT interactive.
I hope to find some time to play with it some more to show the benefit of ENTRYPOINT.

[mykaul]

Hmm, why is 'rows' 0?
Rows [row] 0
└─ [row/req] 0.0

[vponomaryov]

Hmm, why is 'rows' 0? Rows [row] 0 └─ [row/req] 0.0

It is number of rows returned in request responses.
So, in your case you posted results of the write queries.
Reading partitions which have more than 1 row will affect that counter.

Example for the read function when there were 2-4 rows per partition:

SUMMARY STATS ══════════════════════════════════════════════════════════════════════════════════════════════════════════════
    Elapsed time       [s]     10.003                                                                 
        CPU time       [s]      3.426                                                                 
 CPU utilisation       [%]        2.9                                                                 
          Cycles      [op]      20000                                                                 
          Errors      [op]          0                                                                 
              └─       [%]        0.0                                                                 
        Requests     [req]      20000                                                                 
              └─  [req/op]        1.0                                                                 
         Retries     [ret]          0                                                                 
              └─ [ret/req]    0.00000                                                                 
            Rows     [row]      71566                                                                 
              └─ [row/req]        3.6                                                                 
     Concurrency     [req]          1 ± 0                                                             
              └─       [%]          1                                                                 
      Throughput    [op/s]       1999 ± 3                                                             
              ├─   [req/s]       1999 ± 3                                                             
              └─   [row/s]       7154 ± 11                                                            
   Cycle latency      [ms]      0.960 ± 0.103                                                         
 Request latency      [ms]      0.942 ± 0.101