Leverage the Dependency Plugin in the li-hadoop-plugin subproject

[convexquad]

@nntnag17 Reminded me that the Dependency Plugin in the hadoop-plugin subproject can emit warnings when you include certain artifacts in your Hadoop Runtime configuration, and that this is just a config change.

We should use this to warn / block users from including Hadoop artifacts in their zips. I think we should block com.linkedin.hadoop and org.apache.hadoop artifacts. We should probably just warn if they include com.linkedin.pig and com.linkedin.hive artifacts (I'm not sure if we should block or warn if they include org.apache.pig and org.apache.hive artifacts, since they should use the com.linkedin versions of these).

I'm also not sure if we should block or warn org.apache.spark artifacts - probably we want to block them.

Since this enhancement could block users if we get it wrong, we should be careful with it. We should include Jack Dintruff in the discussion and also discuss with Hadoop Dev team leads.

[jdintruff]

Hey @convexquad @nntnag17 sorry that this never made its way to my inbox. I think this is a fantastic idea and could even be written based on some of our internal ByteRay blacklisted artifacts which are known to conflict with other Hadoop artifacts. If this hasn't already been fixed, let me know and I can take a look.

[convexquad]

@jdintruff By default the Hadoop Plugin warns you against including open-source dependencies from org.apache.<hadoop, hive, pig, oozie>, which you can see here: https://github.com/linkedin/linkedin-gradle-plugin-for-apache-hadoop/blob/master/li-hadoop-plugin/src/main/resources/dependencyPattern.json

This feature works quite robustly, and I'm sure we could find more ways to leverage it - if we have known artifacts that should be blacklisted, we should definitely add them.