Add support for partial file retrieve

[kool79]

Add native support for 'Range' header;
Add support for "If-Range" header
Add detection if server support partial file download.

[lookfirst]

Yea, the original intent (oh so long ago now) for this library was that it was an "all or nothing" type of operation. It simplifies things greatly. This is something that can be handled on the client using this library, but would require a full download again. Maybe first discuss what you think the API should look like.

[kool79]

Add detection if server support partial file download.

The easiest part. sardine.isRangesSupported(url) or sardine.supportsPartialDownload(url)

Add native support for 'Range' header;

In can be few new method like:

getPartial(URL, InputStream, long fromByte, long toByte)
getPartial(URL, InputStream, long position)

in 2nd case position can be:

• positive, then it should be interpretted as "give me all bytes after that position". It shoud be converted to: Range: bytes=<position>-.
  For example, for position = 100: Range: bytes=100-
• negative: position will be counted from end of file and will be the same as "give me last 100 bytes". Range: bytes=-<(-position)> or (for position = -100): Range: bytes=-100.

But with that approach we cannot handle multipart downloads for Range: bytes=0-99, 200-299.
In that case we should supply something like List<Pair<Long, Long>>. But better will be to have RangeSpecification object with methods: .addRange(start, end), .last(position), .after(position):

  // Range: bytes=0-99, 200-299, -100
  var rangeSpecification = new RangeSpecification();
  range.addRange(0, 99).addRange(200,299).last(100);
  getPartial(URL, InputStream, rangeSpecification);

  // Range: bytes=0-99, 200-299, -100
  var rangeSpecification = new RangeSpecification();
  range.addRange(0, 99).addRange(200,299).last(100);
  getPartial(URL, InputStream, rangeSpecification);
 //  or 
    getPartial(URL, InputStream, new RangeSpecification(0, 100));

Also we need to supply additional headers like current method get() does For each of above methods... So API will be very ugly: +6 methods in total ((

getPartial(URL, InputStream, fromByte, toByte)
getPartial(URL, InputStream, size)
getPartial(URL, InputStream, RangeSpecification)
getPartial(URL, InputStream, fromByte, toByte, headers)
getPartial(URL, InputStream, size, headers)
getPartial(URL, InputStream, RangeSpecification , headers)

But... if RangeSpecification will be flexible enougth then we can leave only 3 methods:

getPartial(URL, InputStream, RangeSpecification)
getPartial(URL, InputStream, RangeSpecification , headers)

Note: getPartial should handle 200-responce in special way. Because it means server does not supportpartial download and can return empty body (empty file) (in my case it actually do that).

Add support for "If-Range" header

dont know... Plus one more parameter increases overrided methods count.
Maybe we can omit this 'feature request' and user should supply this header inside "headers" field.

There is more ellegant solution with Builder pattern. But this will require a lot of work. Something like:

var inputStream = Sardine.builder()
   .url("http://myserver.webDav.com/myfile.log")
   .auth(user, password)
   .enableCompression()
   .addHeader("x-header", "value")
   .addHeader("If-Range", "123456789abcd")
   .addRange(0, 99)
   .addRange(200, 299)
   .get();

It will be good if every method will return new immutable object: i.e it should not modify current instance (and returns 'this') but instead it should returns new instance every time. In such case it will be thread safe and can do:

var myServerRequest = Sardine.builder()
   .baseUrl("http://myserver.webDav.com/")
   .addHeader("x-header", "value")
   .auth(user, password)
   .enableCompression()
   .;enablePreemptiveAuthentication();
 
// reuse saved config
var fileStream1 = myServerRequest.path("myfile1.log").get();
var fileStream2 = myServerRequest.path("myfile2.log").range(0, 100).get();
var files = myServerRequest.path("").list();
var files2 = myServerRequest.path("subfolder/").list();

[kool79]

just for example
https://gist.github.com/kool79/0dff6c471620948670022f81bc20efcf

[lookfirst]

Wow, excellent response. I like the builder idea, but yea, that is like a whole new version and a lot of work. Obviously, not my itch to scratch anymore. I haven't had a use for this project in probably 10 years. =( Crazy it is (and I am) this old!

[3c71]

Based on kool79's answer, I just did this to open a file starting at a specific position (to build a random input stream):

HashMap<String, String> headers = new HashMap<>();
headers.put("Range", "bytes=" + start_offset + "-");
sardine.get(getInternalPath(), headers);

Worked like a charm!