Is your feature request related to a problem? Please describe.
This is in relation to one of the new features added in v0.4.0:

• The ability to use --archives option to deploy the worker binaries and assemblies that UDFs depend on (Expose a way to override assembly search paths used by AssemblyLoader. #187)

The --archives option only works for YARM setups. Databricks, for example, does not use YARM so the --archives parameter has no effect.

Describe the solution you'd like
For the application .zip file to be transferred to all workers, extracted and the contents to be in the assembly search path.

Describe alternatives you've considered
A partial workaround is adding the applications .zip file name to the --files parameter meets the first requirement of transferring it to all workers. However, it is not automatically extracted or added to the assembly search path.

Databricks Spark-Submit example:

["--files","DotNetSparkTest.zip","--class","org.apache.spark.deploy.dotnet.DotnetRunner","/dbfs/spark-dotnet/microsoft-spark-2.4.x-0.4.0.jar","/dbfs/apps/DotNetSparkTest.zip","DotNetSparkTest"]