All pull requests

Pull requests

• fix(core): stop passing unsupported limit to datasets.load_dataset in the HuggingFace strategy

  #1096·moneebullah25 opened Oct 10, 2026·3 of 4 tasks·

  ·
  1

• fix(dataset): skip datasets allotted no samples under max_sample_num

  #1095·moneebullah25 opened Oct 9, 2026·4 of 5 tasks·

  ·
  1

• fix(utils): invalidate lenient JSONL cache when source files change

  #1094·moneebullah25 opened Oct 9, 2026·3 of 5 tasks·

  ·
  1

• fix(embedding): prevent model instance repr from contaminating text inputs

  #1090·cmgzn opened Oct 9, 2026Collaborator·

  ·

• feat(ray): restore adaptive batches after input pressure eases

  #1088·fengrui-z opened Oct 9, 2026Collaborator·

  ·
  1

• fix(mapper): honour drop_no_head in remove_header_mapper

  #1084·MohammadHijjawi97 opened Oct 1, 2026Contributor·

  ·
  1

• feat(ray-partitioned): opt-in streaming recovery for row-preserving mappers

  #1075·cmgzn opened Sep 22, 2026Collaborator·

  ·

• feat(ray): add opt-in GPU probe resource sampling

  #1072·fengrui-z opened Sep 16, 2026Collaborator·

  ·

• ci(docs): cache Sphinx builds in Actions instead of Pages history

  #1071·cmgzn opened Sep 16, 2026Collaborator·

  ·

• fix(ci): support metadata 2.5 in PyPI publishing

  #1069·cmgzn opened Sep 9, 2026Collaborator·

  ·

• feat: support OrcaRouter as a named OpenAI-compatible provider

  #1058·nissrin2020ali-ux opened Aug 28, 2026·

  ·

• [WIP] [Feature] Integrate Multi-Node Elastic Data Sharding into Data-Juicer Core

  #1042·Qirui-jiao opened Aug 13, 2026Collaborator·

  ·

• feat: auto-shard eligible multinode jobs

  #1041·fengrui-z opened Aug 13, 2026Collaborator·

  ·

• [Draft] feat: add ElasticJuicer OOM-safe adaptive Ray batching MVP

  #1009·fengrui-z opened Jul 14, 2026Collaborator·

  ·
  4

• use custom gzip compression to tolerate FUSE-mounted OSS/S3

  #980·ideal opened May 15, 2026Contributor·

  ·
  4

• Add video_human_3d_pose_mapper.

  dj:openhancement
  #976·Qirui-jiao opened May 10, 2026Collaborator·

  ·
  17

• Add normal map op, optimal flow op, and universal segmentation op for videos.

  dj:multimodaldj:openhancement
  #970·Qirui-jiao opened Apr 27, 2026Collaborator·

  ·
  19

• [WIP] feat(agent): training-ready data recipes, learnable-value mappers, cross-model similarity

  agentdj:opdj:post-tuning
  #969·yxdyc opened Apr 20, 2026Collaborator·

  ·
  3

• Add face keypoints/animal pose ops & Extend ops for frame-sequence input

  dj:openhancement
  #966·Qirui-jiao opened Apr 14, 2026Collaborator·

  ·
  9

• better parallelism in partitioned ray executor

  #945·cyruszhang opened Mar 17, 2026Collaborator·

  ·
  6

• [WIP] arXiv/PDF to Markdown mappers + dj-op one-shot runner

  dj:op
  #917·yxdyc opened Feb 14, 2026Collaborator·

  ·
  7

• [WIP] Multi-branch executor

  dj:coreenhancement
  #916·yxdyc opened Feb 13, 2026Collaborator·

  ·
  1

  6

• [WIP] feat: Add combined_logical_filter operator with AND/OR support

  dj:op
  #914·yxdyc opened Feb 13, 2026Collaborator·

  ·
  6

• Feat: Support paimon, iceberg, hudi, delta lake, hdfs data source.

  #911·Dludora opened Feb 11, 2026Collaborator·

  ·
  29

  

• [WIP] Feat: Add video_calibration_mapper and video_split_by_frame_mapper

  #902·1van2ha0 opened Feb 1, 2026·3 tasks·

  ·
  7