Skip to content

early stop worker got Exception Error #1310

Open
@kamirdin

Description

🐛 Describe the bug

dp = ...
dp = dp.sharding_filter()
rs = MultiProcessingReadingService(num_workers=4)
dataloader = DataLoader2(dp, reading_service=rs)

for _ in dataloader:
    pass

dataloader.shutdown()
Exception: Can not expect any response without submitted request
This exception is thrown by __iter__ of _IterateQueueDataPipes(datapipes=[QueueWrapper, QueueWrapper, QueueWrapper, QueueWrapper])

related to 901b483

Versions

[pip3] numpy==1.26.4
[pip3] pytorch-wpe==0.0.1
[pip3] rotary-embedding-torch==0.6.4
[pip3] torch==2.1.1
[pip3] torch-complex==0.4.4
[pip3] torchaudio==2.1.1
[pip3] torchdata==0.7.1

Activity

linked a pull request that will close this issue on Aug 5, 2024
andrewkho

andrewkho commented on Aug 6, 2024

@andrewkho
Contributor

Hi @kamirdin, please see #1196 as we won't be supporting dataloader2 nor datapipes in torchdata moving forward

qmpzzpmq

qmpzzpmq commented on Nov 7, 2024

@qmpzzpmq

@kamirdin have you fix it? I get same issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Participants

    @andrewkho@qmpzzpmq@kamirdin

    Issue actions

      early stop worker got Exception Error · Issue #1310 · pytorch/data