Improvement/s3 utils 191

[benzekrimaha]

Issue: S3UTILS-191

[benzekrimaha]

Please note that this will be switched to the arsenal adequate version once the current in review PR with a fix is merged: scality/Arsenal#2371

[bert-e]

Hello benzekrimaha,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request TBA
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[francoisferrand]

just mocking getIsTransient means we basically do not execute this code...

• Best to have some test exercising the "whole" code
• If needed, we may use a mock (in some tests, ideally not all) to simulate specific conditions/results/...
• In any case, need extra tests cases to verify the new functions behave somewhat correctly (esp. do not crash) and the information ("is transient") is used appropriately

[benzekrimaha]

done here : 8f20ab6#diff-c0f7a781647ac82835f751f3f2acc5767cca6b22e6f41de1539ec22c29ffcb32R220

[francoisferrand]

Looking at the tests, it does not cover all branches of the code.

In addition, these tests are overly complicated to tests 2 very simple functions : would seem much easier (both to write and to review) to have "simpler" tests to exercise all code path in getIsTransient() (and the nested pensieveLocationIsTransient) by just calling this function.

[francoisferrand]

should have at least 2 scenarios, depending on this.client.isLocationTransient (since there are multiple code path).

the point is not just to see that countItems is calling getIsTransient, but really to execute the code in getIsTransient to ensure it will not fail in production... (it may be done though other tests, calling the function directly!)

(alternatively, since we use a wrapper of MongoClient in here: if you add the function there they can be tested in mongoclient tests, and you don't need to change the countWorker tests)

[francoisferrand]

Looking at the tests, it does not cover all branches of the code.

In addition, these tests are overly complicated to tests 2 very simple functions : would seem much easier (both to write and to review) to have "simpler" tests to exercise all code path in getIsTransient() (and the nested pensieveLocationIsTransient) by just calling this function.

[francoisferrand]

test name feels weird : there is no "feature" of the code to use a "real getIsTransient", it is really just that getIsTransient is mocked in all tests...
--> should find a name that better matches what "feature" of the code this test exercises (or take this as a hint that indeed we are mixing approaches here, and we shoudl either not use a mock in the tests -focus on e2e function, covering all cases in data-, or keep the mock to test the "counting" part and create other tests to verify getIsTransient works as expected)

[francoisferrand]

in the previous code, we used to have different getIsTransient() mock results depending on the test case : i.e. either [] (i.e. no error, not transient) or [null, true] (i.e. no error, transient)

However, with your change the mock always returns true : and we don't actually test the "usual" scenario we are in...

[bert-e]

Conflict

There is a conflict between your branch improvement/S3UTILS-191 and the
destination branch development/1.16.

Please resolve the conflict on the feature branch (improvement/S3UTILS-191).

git fetch && \
git checkout origin/improvement/S3UTILS-191 && \
git merge origin/development/1.16

Resolve merge conflicts and commit

git push origin HEAD:improvement/S3UTILS-191

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[bert-e]

Conflict

There is a conflict between your branch improvement/S3UTILS-191 and the
destination branch development/1.16.

Please resolve the conflict on the feature branch (improvement/S3UTILS-191).

git fetch && \
git checkout origin/improvement/S3UTILS-191 && \
git merge origin/development/1.16

Resolve merge conflicts and commit

git push origin HEAD:improvement/S3UTILS-191

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[benzekrimaha]

/approve

[bert-e]

I have successfully merged the changeset of this pull request
into targetted development branches:

• ✔️ development/1.16

The following branches have NOT changed:

• development/1.13
• development/1.14
• development/1.15
• development/1.4

Please check the status of the associated issue S3UTILS-191.

Goodbye benzekrimaha.

The following options are set: approve