Skip to content

TDL-16314 added request timeout#96

Open
namrata270998 wants to merge 10 commits into
masterfrom
TDL-16314-implement-request-timeout
Open

TDL-16314 added request timeout#96
namrata270998 wants to merge 10 commits into
masterfrom
TDL-16314-implement-request-timeout

Conversation

@namrata270998

@namrata270998 namrata270998 commented Nov 12, 2021

Copy link
Copy Markdown

Description of change

Implemented request timeout with parameter provided from config

Manual QA steps

  • The request backoff for 5 times in case it is timed out for sync mode.
  • The request backoff for 120 seconds in case it is timed out for discovery mode
  • Error should be thrown in case of retries not successful after all the tries
  • Checked the request timeout works for each stream and backoff

Risks

Rollback steps

  • revert this branch

@namrata270998 namrata270998 changed the title addd request timeout TDL-16314 added request timeout Nov 12, 2021
@namrata270998

Copy link
Copy Markdown
Author

Disabled pylint for consider-using-f-string when used format.

Comment thread tap_xero/client.py Outdated
# backoff for 1 minute when the Timeout error occurs as the request will again backoff
# when timeout occurs in `refresh_credentials()`, hence instead of setting max_tries as 5
# setting the max_time of 60 seconds
@backoff.on_exception(backoff.expo, Timeout, max_time=60, factor=2)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we don't add ConnectTimeout here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we don't add ConnectTimeout here?

We didn't get ConnectTimeout Error here, so haven't included it

Comment thread tap_xero/client.py Outdated
import pytz
import backoff
import singer
from requests.exceptions import Timeout, ConnectTimeout

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any specific reason to import ConnectTimeout?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any specific reason to import ConnectTimeout?

It was not used anywhere as ConnectTimeout inherits Timeout, hence removed it

@savan-chovatiya
savan-chovatiya self-requested a review November 26, 2021 10:45

@kspeer825 kspeer825 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a suggestion for the test implementation.

mock_send.side_effect = Timeout
mock_post.side_effect = Timeout
before_time = datetime.datetime.now()
with self.assertRaises(Timeout):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these with assertRaises contexts should contain only the method that raises the exception, if the XeroClient init had a request in it and that timed out unexpectedly, this would never get to the check_platform_access step. But it would still pass. While I think it is performing functionally the way it's intended, it is difficult to see exactly what is being tested.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@namrata270998

Copy link
Copy Markdown
Author

The build is failing because the token is expired on the circleci side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants