feat: SelectObjectContentRequest #236
Conversation
…essage, headers, and payload parsing, Added new functions to parse EventStream messages
SelectObjectContents struct and add XML serialization helpers
options for input_serialization, output_serialization, and scan_range
improve code organization
|
ping @bernardd |
modules to improve readability and consistency.
performance.
select object contents test
| describe "integration test" do | ||
| setup [:start_bypass] | ||
|
|
||
| test "stream SelectObjectContent results", %{bypass: bypass} do |
There was a problem hiding this comment.
ex-aws/ex_aws#1012 has to be merged for this test to pass
… ExAws.SParsers.EventStream to buffer streams
|
This is great work, would love to see this merged. |
| def to_xml(value) when is_atom(value), do: Atom.to_string(value) |> String.upcase() | ||
| def to_xml(value) when is_binary(value), do: value | ||
| def to_xml(value) when is_integer(value), do: Integer.to_string(value) | ||
| def to_xml(value) when is_float(value), do: Float.to_string(value) |
There was a problem hiding this comment.
It's probably best not to roll our own XML encoder here - it misses some basic stuff like escaping of control characters. Maybe add something like https://github.com/joshnuss/xml_builder as an optional dependency to be used here?
|
Thanks very much for your work on this and the corresponding PR over in ex_aws_s3, @avinayak. Please also accept my apologies for not getting to it sooner. It's been quite a busy end of the year over here. |
3 participants
This PR Adds support for SelectObjectContentRequest in S3. This let's users query contents of S3 Objects if the yare of type CSV, JSON or Parquet.
Explanation for some design choices:
Rangeheader for this request. So, we cannot use multiple http requests to stream this.@bernardd Can you please check this?