Improve hurl test script with arguments

[JonBendtsen]

QUAL: Improve hurl test script, some arguments now available
This PR introduces arguments to the hurl test script:

• --noapi will skip the API tests (that require authentication)
• --nogui weill skip the GUI tests (that require authentication
• --nopublic Preparing for in the future being able to skip the tests that do not require autentication
• --cookiefile= if a file exists and it has a non zero size, then the contents are used as cookies, if it supplied, but either empty or does not exist, this is the filename that will be used
• --just= will find all hurl files and do a simple grep to select just those few tests one might need

Example:
./run.sh --noapi --cookiefile=/tmp/cookie.jar

[mdeweerd]

$(grep -c -- '--cookiefile=' <<< "$arg") saves an echo.

You can even write the following, bu tI think that may confuse.
$(<<< "$arg" grep -c -- '--cookiefile=' )

[JonBendtsen]

JonSweet16:hurl jonbendtsen$ ./run.sh --noapi --cookiefile=/tmp/cookie.jar
----- Run hurl test on APIs ---
First we run tests that do not require authentication
Success gui/00_HOME.hurl (1 request(s) in 341 ms)
Success api/00_foobar.hurl (1 request(s) in 345 ms)
Success api/00_explorer.hurl (1 request(s) in 373 ms)
Success api/status/00_status.hurl (1 request(s) in 377 ms)
Success public/payment/00_payment_newpayment.hurl (1 request(s) in 402 ms)
Success public/onlinesign/00_onlinesign_newonlinesign.hurl (1 request(s) in 411 ms)
Success api/login/00_login_POST.hurl (1 request(s) in 1383 ms)
Success api/login/00_login_GET.hurl (1 request(s) in 1384 ms)
--------------------------------------------------------------------------------
Executed files:    8
Executed requests: 8 (5.8/s)
Succeeded files:   8 (100.0%)
Failed files:      0 (0.0%)
Duration:          1390 ms

Now we are ready to run GUI tests that do require authentication
Success gui/10_HOME.hurl (1 request(s) in 625 ms)
Success gui/admin/10_mails_templates.hurl (1 request(s) in 879 ms)
Success gui/comm/mailing/10_mailing.hurl (1 request(s) in 1095 ms)
--------------------------------------------------------------------------------
Executed files:    3
Executed requests: 3 (2.7/s)
Succeeded files:   3 (100.0%)
Failed files:      0 (0.0%)
Duration:          1096 ms

[JonBendtsen]

examples with --just=

  546  ./run.sh --just=mail ; echo $?
  547  ./run.sh --just=mailings ; echo $?
  551  ./run.sh --just= ; echo $?
  554  ./run.sh --just=api/comm
  556  ./run.sh --just=api/users

[JonBendtsen]

@mdeweerd I suppose that with your PR #36279 then this PR is obsolete?

[mdeweerd]

@mdeweerd I suppose that with your PR #36279 then this PR is obsolete?

No because you add options starting with "--" - apparently some options overlap.

I prefer to select tests without a -- option.

Basically, I think we could do just with what I proposed and --cookie option.

Select api and public in my PR is test/hurl/run.sh api/ public/
Select public is test/hurl/run.sh public/ .
Run setup_modules: test/hurl/run.sh setup_modules.
Wildcards can be used, but not grep like must be "find" compatible

I did not mean to overlap though...

[JonBendtsen]

@mdeweerd I suppose that with your PR #36279 then this PR is obsolete?

No because you add options starting with "--" - apparently some options overlap.

I prefer to select tests without a -- option.

Basically, I think we could do just with what I proposed and --cookie option.

Select api and public in my PR is test/hurl/run.sh api/ public/ Select public is test/hurl/run.sh public/ . Run setup_modules: test/hurl/run.sh setup_modules. Wildcards can be used, but not grep like must be "find" compatible

I did not mean to overlap though...

It happens, and I think your approach is better. Will you add possibility for specifying where the cookie should be saved/reused?

I could see some value in having it possible to skip one or some files in a subdir that you included - for when people run it by hand during development

• you know it doesn't work
• it does a destructive test AND/OR it expects some demo data to already be inside Dolibarr

[mdeweerd]

I added the cookie option and some other ones.
The exclusion option is something else a bit more complex.

[JonBendtsen]

I added the cookie option and some other ones. The exclusion option is something else a bit more complex.

yeah that's why i ended up using grep and xargs. I even though about using grep -e for regex or perhaps grep -E for lines like "abc|foo|..."

[mdeweerd]

Using git ls-files would be more interesting in a way - faster because using the git database, only running on git files, and multiple patterns just provided as multiple arguments.

But I went with what was already done.

[JonBendtsen]

replaced with #36291