Skip to content

Implement Hadoop DSL Canonical Compiler and Azkaban Reverse Job Compiler #101

@convexquad

Description

@convexquad

The idea is to implement a Hadoop DSL compiler that actually writes out formatted Hadoop DSL. This would be necessary for a Azkaban job files -> Hadoop DSL reverse compiler.

In people's Hadoop DSL files they will have all kinds of Groovy that is actually evaluated at Gradle build time. They end up with a Hadoop DSL structure in memory. The canonical compiler would spit out the Hadoop DSL structure as Hadoop DSL itself, but obviously with no Groovy logic in it.

Once the canonical compiler is in place, the task of implementing an Azkaban reverse compiler would simply consist of a processor for the Azkaban .job and .properties files that builds up the corresponding Hadoop DSL structure in memory and then just writes it out using the canonical compiler.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions