Skip to content

Refactor Hadoop Validator into its own project #92

@convexquad

Description

@convexquad

We've got the great new Hadoop / Pig validator enhancements in the 0.10.x series thanks to @kurtgodel95 and @nntnag17. I would like to make some minor refactorings as we bring this code in preparation for wide use at LinkedIn (and in the open-source community).

I would to pull https://github.com/linkedin/linkedin-gradle-plugin-for-apache-hadoop/tree/master/hadoop-plugin/src/main/groovy/com/linkedin/gradle/hadoopValidator into its own top-level subproject called "hadoop-validator". This will more easily enable this code to be tested, and possibly re-used as an API dependency in other projects. Additionally, we'll be able to isolate its dependencies in its own build.gradle file, which will help us increase the reliability of the Hadoop Plugin as the LinkedIn Gradle plugins classpath is already getting filled with all kinds of stuff.

We might additionally need to pull some other things in the hadoop-plugin subproject out (I think the hadoopValidator code uses the HDFS utility classes in the Hadoop Plugin). If so, we can additionally pull out another subproject called "hadoop-base" on which all other subprojects can depend.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions