[Misc] Batch: Improve batch output persistence resilience during execution and finalization

[zhangjyr]

Pull Request Description

• tolerate output persistence failures for a bounded number of time buckets via AIBRIX_BATCH_OUTPUT_PERSISTENCE_FAILURE_THRESHOLD
• keep batch jobs progressing while transient storage persistence failures are still below the configured threshold
• reroute unreadable finalized output parts into synthesized error records so finalization still produces a consistent error file
• strengthen multipart aggregation and finalize-path coverage with targeted tests

Important: Before submitting, please complete the description above and review the checklist below.

---

Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

• [Bug]: Corrections to existing functionality
• [CI]: Changes to build process or CI pipeline
• [Docs]: Updates or additions to documentation
• [API]: Modifications to aibrix's API or interface
• [CLI]: Changes or additions to the Command Line Interface
• [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

• PR title includes appropriate prefix(es)
• Changes are clearly explained in the PR description
• New and existing tests pass successfully
• Code adheres to project style and best practices
• Documentation updated to reflect changes (if applicable)
• Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

[gemini-code-assist]

Code Review

This pull request introduces robust handling for output persistence failures and finalization rerouting in the batch job driver. Key changes include mechanisms to defer persistence failures using time-bucketed thresholds, reroute unreadable completed output parts into synthesized error files during finalization, and support local fallback part providers in multipart uploads. The reviewer feedback is highly constructive and identifies critical issues: a bug in the serial worker where deferred requests are incorrectly treated as successful completions, a potential resource leak where orphaned staged parts are not aborted during finalization, and a code simplification opportunity using the walrus operator.