release-2023-05-23: mkbucket

new command mkbucket

Creates a bucket.

Usage

Usage: mkbucket <region> <S3A path>

hadoop jar cloudstore-1.0.jar mkbucket us-east-2 s3a://new-bucket-name/

release-2023-05-15

• tlsinfo to print out x509 certs in the default certificate store
• HADOOP-18637. s3a diag prints out new multipart options.
• storediag looks for X509 class removed from java17 (which breaks shipping wildfly jars)
• ABFS diagnostics fail fast if account is unknown. Handy as abfs connector itself doesn't.

release-2023-05-03

ABFS diagnostics fail fast if account is unknown

See HADOOP-18728. ABFS handles nonexistent/non-auth accounts badly

release-2023-04-28

better diagnostics of privatelink endpoints

• adds a trailing / at the end of the url in probes for vpce endpoints (and amazon.cn)
• doesn't warn about https or highlight risk of config forgetting to change endpoint for internal deployment

release-2023-04-27: Add a -bfs option to the dux command for breadth-first-treewalk

The -bfs option for dux does a breadth-first treewalk of the target filesystem, scanning subdirectories in parallel.

This is faster for high-latency object stores which don't have an O(1) deep tree list command (which S3 does; this option will be slower there)

release-2023-04-20

storediag + dux enhancements

storediag

• more hints on s3a endpoint diagnostics.
• configuration option reports which configs are final
• abfs diag: print java network sysprops

dux

• same M/G/T printing as hadoop, not the commons-utils call which rounds down
• prints less, unless the -verbose option is set

release-2023-03-31

storediag -h option to hide all chars in secret strings

release-2023-03-30

print "javax.net.ssl.trustStorePassword" in TLS diags

release-2023-03-29

release of 2023-03-29: better jceks resilience

when a getPassword() raises an IOE, the first failure is logged with a stack trace, for subsequent ones swallowed
the logged value is the error string.

thus jceks retrieval problems are not immediately fatal

release-2023-03-27

release of 2023-03-27

work in progress: multipart listing

• outputs to CSV
• includes size
• next step would be to to commit incomplete uploads.
  note, calculating size is slow as it requires each upload to be scanned, so takes parts/1000 #of calls per outstanding upload.

+some better reporting of file io on storediag