request_id key not found causes unlogged error

[jgrevich]

Summary

Similar in nature to #24, errors occur during load test runs that are not logged as request failures, skewing the results, at least in regard to request success/failure. In this case there is an error parsing the request params where it expects there to be a request_id key. Similar to #24 it appears that the failure is not logged and the request is retried.

Detail

I started the SP based sign in test using the following configuration:

NUM_USERS=10000 SP_HOST=https://sp-oidc-sinatra.pt.identitysandbox.gov locust --locustfile load_testing/sp_sign_in.locustfile.py --host https://idp.pt.identitysandbox.gov  --users 50 --hatch-rate 10 --run-time 30m --headless

While running I noticed repeating errors caused by the lack of the request_id key in the parameters:

[2020-10-05 15:05:38,215] j12/ERROR/locust.user.task: 'request_id'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/locust/user/task.py", line 284, in run
    self.execute_next_task()
  File "/usr/local/lib/python3.8/site-packages/locust/user/task.py", line 309, in execute_next_task
    self.execute_task(self._task_queue.pop(0))
  File "/usr/local/lib/python3.8/site-packages/locust/user/task.py", line 321, in execute_task
    task(self)
  File "/Users/jjg/Documents/sources/login/identity-loadtest/load_testing/sp_sign_in.locustfile.py", line 9, in sp_sign_in_load_test
    flow_sp_sign_in.do_sp_sign_in(self)
  File "/Users/jjg/Documents/sources/login/identity-loadtest/load_testing/common_flows/flow_sp_sign_in.py", line 47, in do_sp_sign_in
    request_id = querystring_value(resp.url, "request_id")
  File "/Users/jjg/Documents/sources/login/identity-loadtest/load_testing/common_flows/flow_helper.py", line 86, in querystring_value
    return parse_qs(parsed.query)[key][0]
KeyError: 'request_id'

https://github.com/18F/identity-loadtest/blob/master/load_testing/common_flows/flow_helper.py#L85

Steps To Reproduce

See above, note it was run from a home internet connection and resulted in 5-30s response times.

Expected Behavior

The error should be logged as a failed request and included in the failed request summary at the completion of a test run.

Actual Behavior

The errors are not logged and it appears that the request is retried.

Suggested Actions

We should make sure these errors (and any others) are logged and counted as request failures. Perhaps we should have some way of confirming the request counts by type using a separate system.

Acceptance Criteria

• the missing param/unexpected response errors should be logged and included in the results summary