Query args not supported while sending to client

[bertvw]

Dear all,
I get the following error: ngx_http_aws_auth_canonical_request: query args not supported while sending to client, client: 10.69.96.38, server: msd-workflow-dev.proxyserver.msd, request: "GET /?list-type=2&prefix=Sites%2FBlaBla HTTP/1.1", host: "10.113.149.48:8080"
Other GET requests are ok. It looks like the nginx-aws-auth-module cannot handle query parameters in the URL.
Is there a solution for this issue?

[kaltura-hooks]

Hi @bertvw,

Thank for you reporting an issue and helping improve Kaltura!

To get the fastest response time, and help the maintainers review and test your reported issues or suggestions, please ensure that your issue includes the following (please comment with more info if you have not included all this info in your original issue):

• Is the issue you're experiencing consistent and across platforms? or does it only happens on certain conditions?
  please provide as much details as possible.
• Which Kaltura deployment you're using: Kaltura SaaS, or self-hosted?
  If self hosted, are you using the RPM or deb install?
• Packages installed.
  When using RPM, paste the output for:

	# rpm -qa \"kaltura*\"

For deb based systems:

	# dpkg -l \"kaltura-*\"

• If running a self hosted ENV - provide the MySQL server version used
• If running a self hosted ENV - is this a single all in 1 server or a cluster?
• If running a self hosted ENV, while making the problematic request, run:

	# tail -f /opt/kaltura/log/*.log /opt/kaltura/log/batch/*.log | grep -A 1 -B 1 --color \"ERR:\|PHP\|trace\|CRIT\|\[error\]\"

and paste the output.

• When relevant, provide any screenshots or screen recordings showing the issue you're experiencing.

For general troubleshooting see:
https://github.com/kaltura/platform-install-packages/blob/Jupiter-10.13.0/doc/kaltura-packages-faq.md#troubleshooting-help

If you only have a general question rather than a bug report, please close this issue and post at:
http://forum.kaltura.org

Thank you in advance,

[erankor]

What is the use case that requires query parameters? For simple reading of files it's not needed...

[bertvw]

Hi, Thanks for your reaction. We have a mobile app that uses AWS .NET SDK to access the AWS S3 bucket. This is not secure, because the app contains the AWS credentials. We want to use a nginx based reverse proxy server in front of the S3 bucket that will have the AWS credentials instead. We discovered that the AWS .NET SDK issues requests containing query parameters e.g. (nginx error log): 2020/03/20 09:00:12 [error] 17188#0: *25 ngx_http_aws_auth_canonical_request: query args not supported while sending to client, client: 10.69.96.38, server: msd-workflow-dev.proxyserver.msd, request: "GET /?list-type=2&prefix=Sites%2FBlablalala HTTP/1.1", host: "10.113.149.48:8080" The AWS S3 bucket issues then an access denied error. We think that then the AWS credentials in the HTTP request are not calculated correctly. I would be very happy if you see a solution to this problem or could help in pointing to another nginx based solution that can handle query parameters correctly. Currently, our project has a strict deadline. The app is a legacy application which we cannot easily redevelop. Thank you very much for your support! Kind regards, Bert van Willigen Van: erankor [mailto:notifications@github.com] Verzonden: vrijdag 20 maart 2020 11:29 Aan: kaltura/nginx-aws-auth-module <nginx-aws-auth-module@noreply.github.com> CC: bertvw <bert.vanwilligen@onsnet.nu>; Mention <mention@noreply.github.com> Onderwerp: Re: [kaltura/nginx-aws-auth-module] Query args not supported while sending to client (#8) What is the use case that requires query parameters? For simple reading of files it's not needed... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7T4B75URMFGSTBAAWCP7DRINAOPANCNFSM4LQGKF7Q> . <https://github.com/notifications/beacon/AA7T4B5FG6UG4KT73Q37GITRINAOPA5CNFSM4LQGKF72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEPOCDNY.gif>

[erankor]

So, if I understand correctly, you don't really want to send query params to S3...
If that is the case, you can probably make nginx's proxy_pass ignore the query params.
If you currently use something like this:

location /proxy/ {
   proxy_pass http://mybucket.s3.eu-central-1.amazonaws.com/;
   ...
}

Try changing to:

location ~ ^/proxy/(.*) {
  proxy_pass http://mybucket.s3.eu-central-1.amazonaws.com/$1;
  ...
}

[bertvw]

Thank you very much for your response! Actually I am not sure, but I think you are right. If the S3 bucket is only providing static information and just ignores query parameters then yes the query parameters are redundant. So, what I have now is: location / { proxy_pass http://mybucket.s3.eu-central-1.amazonaws.com/$request_uri; ... So, this must be then: location ~ ^/(.*) { proxy_pass http://mybucket.s3.eu-central-1.amazonaws.com/$1; ... } Sorry for my ignorance but I am a newbie (not a nginx expert yet….;-)) Thank you very much! Best regards, Bert Van: erankor [mailto:notifications@github.com] Verzonden: vrijdag 20 maart 2020 14:24 Aan: kaltura/nginx-aws-auth-module <nginx-aws-auth-module@noreply.github.com> CC: bertvw <bert.vanwilligen@onsnet.nu>; Mention <mention@noreply.github.com> Onderwerp: Re: [kaltura/nginx-aws-auth-module] Query args not supported while sending to client (#8) So, if I understand correctly, you don't really want to send query params to S3... If that is the case, you can probably make nginx's proxy_pass ignore the query params. If you currently use something like this: location /proxy/ { proxy_pass http://mybucket.s3.eu-central-1.amazonaws.com/; ... } Try changing to: location ~ ^/proxy/(.*) { proxy_pass http://mybucket.s3.eu-central-1.amazonaws.com/$1; ... } — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7T4B2MIP4IK76J6ACZU53RINU5LANCNFSM4LQGKF7Q> . <https://github.com/notifications/beacon/AA7T4B3LLRFTKYFN4TPRKKTRINU5LA5CNFSM4LQGKF72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEPOS65Q.gif>

[erankor]

If you are using location /, you can probably just replace $request_uri with $uri and that's it, no need for the regex location I suggested.

[bertvw]

Hi, Thanks it is working! I only get now an access denied with a HTTP PUT. I assume your module is also supporting HTTP POST/PUT/DELETE? Thank you very much for your time/attention! Kind regards Bert van Willigen Van: erankor [mailto:notifications@github.com] Verzonden: vrijdag 20 maart 2020 14:46 Aan: kaltura/nginx-aws-auth-module <nginx-aws-auth-module@noreply.github.com> CC: bertvw <bert.vanwilligen@onsnet.nu>; Mention <mention@noreply.github.com> Onderwerp: Re: [kaltura/nginx-aws-auth-module] Query args not supported while sending to client (#8) If you are using location /, you can probably just replace $request_uri with $uri and that's it, no need for the regex location I suggested. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7T4B7WM2C3VEA7VOOB3HDRINXRTANCNFSM4LQGKF7Q> . <https://github.com/notifications/beacon/AA7T4B6HGORRUV3MPRRQ363RINXRTA5CNFSM4LQGKF72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEPOVSRY.gif>

[erankor]

No, GET is supported, HEAD/DELETE will probably work as well, but PUT/POST are much more problematic and not supported. In order to support these, the module would need to buffer the entire file, so that it could calculate the content SHA256, I have no plans to do it...

[bertvw]

Clear. Thank you very much for your answer! I will check whether I can redirect PUT/POST HTTP to another server i.e. not to the S3 bucket. Any suggestions? Ps.: Where are you located? Kind regards, Bert Van: erankor [mailto:notifications@github.com] Verzonden: maandag 23 maart 2020 12:22 Aan: kaltura/nginx-aws-auth-module <nginx-aws-auth-module@noreply.github.com> CC: bertvw <bert.vanwilligen@onsnet.nu>; Mention <mention@noreply.github.com> Onderwerp: Re: [kaltura/nginx-aws-auth-module] Query args not supported while sending to client (#8) No, GET is supported, HEAD/DELETE will probably work as well, but PUT/POST are much more problematic and not supported. In order to support these, the module would need to buffer the entire file, so that it could calculate the content SHA256, I have no plans to do it... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7T4B2GH2ZBWBLPBFAY2RTRI5A65ANCNFSM4LQGKF7Q> . <https://github.com/notifications/beacon/AA7T4B6SP3DO7GD6ZKSDRX3RI5A65A5CNFSM4LQGKF72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEPU6ZGI.gif>

[erankor]

Sorry, not familiar with something that does it, but didn't really check...