update documentation for cloud storage and 0.27.4

Author: rferraton

README.md

@@ -2,7 +2,7 @@
 
 FastBCP is a command line that export data from a database to CSV , JSON,  Parquet or XLSX file(s) (or even binary file(s))
 
-![FastBCP](images/FastBCP.png)
+![FastBCP](images/FastBCP_V2.png)
 
 ## Table of Contents
 
@@ -232,15 +232,17 @@ This field is mandatory if some parallel method is used like Rowid and Ctid
 
 ### Directory (`-D` or `--directory`)
 Specify where the output file will be generated. it could be : 
- - a local directory `/data/db/schema/tname/` or `d:\data\db\schema\tname\`v
+ - a local directory `/data/db/schema/tname/` or `d:\data\db\schema\tname\`
+ - a remote directory `\\uncpath\db\schema\tname\`
  - an aws s3 or s3 compatible s3 path : `s3://bucket/db/schema/tname/`
- - a gcs localtion : `gcs://bucket/db/schema/tname/`
- - *coming soon* : an azure abfss location : `abfss://<file_system>.dfs.core.windows.net/<path>/`
+ - a gcs localtion : `gs://bucket/db/schema/tname/`
+ - an azure blob storage location : `abs://<storage_account>.blob.core.windows.net/<container>/<path>/`
+ - an azure datalake gen2 location : `abfss://<file_system>.dfs.core.windows.net/<path>/`
 
 ### Output file (`-o` or `--fileoutput`)
 The name of the final file generated by FastBCP.
-**IMPORTANT : the file extension will define the output format : csv, tsv, json, bson, parquet, xlsx or binary (postgresql source only for binary)**
- For parallel extraction without merge, the file name (without file extension) will be used as a prefix for the distributed files. 
+<br> **IMPORTANT : the file extension will define the output format : csv, tsv, json, bson, parquet, xlsx or binary (postgresql source only for binary)**
+
 
 ### Timestamped output (`-x` or `--timestamped` switch)
 Will add a timestamp to the output file name if the switch is used
@@ -274,7 +276,7 @@ How boolean are formatted (`true/false`, `t/f` or `1/0`)
 ### cloud profile (--cloudprofile)
 specify the name of the cloud profile 
 - for aws and s3 compatible aws cli : `aws configure --profile profilename`
-- 
+- for gcs it should be the location of the json key file
 
 **Nota1** : for **S3 compatible** target you must define the S3_ENDPOINT environment variable to define the proper S3 endpoint :
 - exemple bash : `export S3_ENDPOINT = "https://play.min.io:9000"`
@@ -285,7 +287,11 @@ specify the name of the cloud profile
 - `s3:AbortMultipartUpload`
 - `s3:ListMultipartUploadParts`
 
-
+**Nota 3**
+For S3, Azure and GCS, without a cloud profile, FastBCP will try to use the default chain authentication mecanism of the cloud provider.
+- For AWS/S3 : [AWS Default Credential Provider Chain](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html#credentials-default)
+- For Azure : [Azure Default Credential Provider Chain](https://learn.microsoft.com/en-us/dotnet/api/azure.identity.defaultazurecredential?view=azure-dotnet)
+- For GCS : [GCS Default Credential Provider Chain](https://cloud.google.com/docs/authentication/production)
 
 ## Advanced Parameters 
 
@@ -486,5 +492,76 @@ You can have a lookp on the live [FastBCP Performance dashboard](https://public.
 ```
 
 
+#### Parallel using RangeId from mssql to parquet on GCS bucket:
+``` powershell
+.\FastBCP.exe `
+--connectiontype mssql `
+--server localhost `
+--database "tpch10_collation_bin2" `
+--trusted `
+--query "SELECT * FROM dbo.orders" `
+--directory "gs://targetbucket/bronze/tpch" `
+--cloudprofile "fastbcp_profile" `
+--fileoutput "mssql_orders.parquet" `
+--method RangeId `
+--distributekeycolumn "o_orderkey" `
+--paralleldegree 12 `
+--merge false `
+--runid "mssql_to_parquet_aws_s3"
+```
+
+#### Parallel using DataDriven from mssql to parquet on GCS bucket with datadriven query:
+``` powershell
+.\FastBCP.exe `
+--connectiontype "mssql" `
+--server "localhost" `
+--database "tpch10_collation_bin2" `
+--trusted `
+--query "select * from orders" `
+--directory "gs://aetpftoutput/test/" `
+--fileoutput "mssql_orders.parquet" `
+--method "DataDriven" `
+--distributekeycolumn "o_orderdate" `
+--datadrivenquery " select top 100 * from (select distinct o_orderdate from orders WHERE o_orderdate between '19950101' and '19950630') a order by 1desc" `
+--cloudprofile "C:\app\fastbcp\gcs\fastbcp_arpeio_key.json"
+--paralleldegree -2
+```
+
+#### Parallel using DataDriven from mssql to parquet on Azure Datalake Gen2 or Azure Blob Storage with datadriven query:
+
+```powershell
+# For Azure you can define the following environment variables for authentication
+$env:AZURE_CLIENT_ID="<ClientID>"
+$env:AZURE_TENANT_ID="<TenantId>"
+$env:AZURE_CLIENT_SECRET="<SecretID>"
+
+# Azure Datalake Gen 2 
+ .\FastBCP.exe `
+--connectiontype "mssql" `
+--server "localhost" `
+--database "tpch10_collation_bin2" `
+--trusted `
+--query "select * from orders" `
+--directory "abfss://aetpadlseu.dfs.core.windows.net/fastbcpoutput/testdfs/orders/" `
+--fileoutput "mssql_orders.parquet" `
+--method "DataDriven" `
+--distributekeycolumn "o_orderdate" `
+--datadrivenquery " select top 100 * from (select distinct o_orderdate from orders WHERE o_orderdate between '19950101' and '19950630') a order by 1 desc"
+
+
+# Azure Blob Storage
+ .\FastBCP.exe `
+--connectiontype "mssql" `
+--server "localhost" `
+--database "tpch10_collation_bin2" `
+--trusted `
+--query "select * from orders" `
+--directory "abs://aetpadlseu.blob.core.windows.net/fastbcpoutput/testblob/orders/" `
+--fileoutput "mssql_orders.parquet" `
+--method "DataDriven" `
+--distributekeycolumn "o_orderdate" `
+--datadrivenquery " select top 100 * from (select distinct o_orderdate from orders WHERE o_orderdate between '19950101' and '19950630') a order by 1 desc"
+```
+
 You can find more exemples in the [samples.ps1](samples.ps1.md) file