Skip to content

Add progress indicator for batch processing #110

@profrod-ai

Description

@profrod-ai

When processing multiple OpenAPI specifications in batch mode, users have no indication of progress. This is especially frustrating for large batches that may take 10+ minutes to complete.

Current Behavior

pdm run batch --all
# ... long silence ...
# Eventually shows completion

Desired Behavior

pdm run batch --all
Processing OpenAPI specifications: 100%|████████████| 25/25 [02:15<00:00,  5.42s/file]
Currently processing: discord.openapi.json
✅ Processed 23 successfully, ❌ 2 failed

Requirements

  • Show overall progress (current/total files)
  • Display current file being processed
  • Show elapsed time and ETA
  • Display final summary with success/failure counts
  • Use a well-established progress bar library

Steps to Complete

  1. Fork the repository
  2. Clone locally: git clone https://github.com/YOUR_USERNAME/arazzo-engine.git
  3. Navigate to: cd arazzo-engine/generator
  4. Set up environment: pdm install
  5. Create branch: git checkout -b feat/batch-progress-bar
  6. Add tqdm dependency to pyproject.toml
  7. Study the batch processing code in arazzo_generator/batch/batch_generator.py
  8. Integrate progress bar in the process_spec_list() method
  9. Test with a few files: pdm run batch --spec-list test_files.txt
  10. Ensure progress bar works correctly
  11. Update documentation if needed
  12. Commit and open PR

Files to Edit

  • pyproject.toml (add tqdm dependency)
  • arazzo_generator/batch/batch_generator.py
  • README.md (update CLI documentation if needed)

Implementation Hints

from tqdm import tqdm

# In process_spec_list method:
for spec_path in tqdm(spec_list, desc="Processing OpenAPI specs"):
    # Update description with current file
    tqdm.set_description(f"Processing: {os.path.basename(spec_path)}")
    # ... existing processing code ...

Acceptance Criteria

  • Progress bar shows during batch processing
  • Current file name is displayed
  • Progress percentage is accurate
  • ETA is shown when possible
  • Final summary shows success/failure counts
  • Progress bar works with both --all and --spec-list modes
  • No performance degradation
  • Progress bar is disabled when output is redirected

Additional Context

  • Use tqdm library (widely used and well-maintained)
  • Consider adding a --quiet flag to disable progress bar
  • Make sure it works well with existing logging output

Estimated Time

⏱️ 2-3 hours

Need Help?

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions