[GOBBLIN-2241] Attempt partial commit only in case commit policy is partial commit

[Blazer-007]

This pull request introduces an enhancement to how job commit policies are determined and used in the workflow implementation. The main change is the addition of support for retrieving the JobCommitPolicy directly from a Typesafe Config object, which improves flexibility and integration with configuration-driven workflows. Additionally, the workflow now conditionally attempts a partial commit based on the job's commit policy.

Configuration and Commit Policy Integration:

• Added a new method getCommitPolicy(Config config) to JobCommitPolicy, allowing retrieval of commit policy from a Typesafe Config object, in addition to the existing Properties and State-based methods. [1] [2]
• Updated ProcessWorkUnitsWorkflowImpl to use the new JobCommitPolicy.getCommitPolicy(Config) method, enabling commit policy evaluation directly from workflow configuration. [1] [2]

Workflow Commit Logic:

• Modified the workflow to conditionally attempt a partial commit only if the current commit policy allows it, improving error handling and adherence to configuration. [1] [2]
  Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-2241

Description

• Here are some details about my PR, including screenshots (if applicable):
  This PR introduces change to attempt partial commit only if commit policy is specified in job properties.
  This is being added to avoid cases when PostPublishStep runs even in case we have job commit policy as full but partial commit is invoked somehow through SafeDatasetCommit

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:
  NA

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"