Skip to content

feat: flat-layout CVAT campaign tools and resilient shard writing - #204

Closed
cau-git wants to merge 2 commits into
mainfrom
cau/cleanup-and-fixes
Closed

feat: flat-layout CVAT campaign tools and resilient shard writing#204
cau-git wants to merge 2 commits into
mainfrom
cau/cleanup-and-fixes

Conversation

@cau-git

@cau-git cau-git commented Mar 16, 2026

Copy link
Copy Markdown
Member

Summary

Flat-layout CVAT campaign tools

Three new scripts for flat delivery directories (complement to the existing submission-style tools):

Script Purpose
convert_flat_cvat_deliveries_to_docling.py Batch-converts flat CVAT XML deliveries → DoclingDocument JSON + HTML; supports parallelism, dry-run, and structured run reports
flat_cvat_deliveries_to_hf.py Aggregates flat-batch exports into a HuggingFace-ready parquet dataset
cvat_task_batch_uploader_flat.py Uploads <task>/images/ + <task>/annotations.xml layouts as tasks to a CVAT project

save_shard_to_disk rework

  • Returns SaveShardResult (written/skipped counts, next shard id) instead of None
  • Automatically splits batches into multiple shards when estimated size exceeds 1.5 GB
  • Per-shard ArrowCapacityError recovery with skipped_records.jsonl audit log

Extensions

  • cvat_create_annotation_tasks_from_folders.py: added --start-index, --end-index, --chunk-number-offset for resumable chunked runs

cau-git added 2 commits March 16, 2026 10:48
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
@github-actions

github-actions Bot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @cau-git, all your commits are properly signed off. 🎉

@cau-git
cau-git force-pushed the cau/cleanup-and-fixes branch from 2c8e5f3 to 8f7d3c8 Compare March 16, 2026 10:13
@mergify

mergify Bot commented Mar 16, 2026

Copy link
Copy Markdown

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@cau-git
cau-git requested a review from maxmnemonic March 16, 2026 10:14
@cau-git cau-git changed the title chore: deps cleanup feat: flat-layout CVAT campaign tools and resilient shard writing Mar 31, 2026
@cau-git cau-git closed this Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant