Skip to content

"can only join an iterable" #257

Open
@esnible

Description

Describe the bug
When Replicate isn't "warmed up", there are strange exceptions.

For example, examples/callback/README.md tells me to python repair_main.py, but this produces a confusing stack trace.

Perhaps the code that invokes Replicate or LiteLLM needs to retry on certain failures?

It would be easier to fix this if I could reproduce this. Once the failure happens then I can't produce the failure again.

To Reproduce
python repair_main.py when Replicate isn't warmed up.

Expected behavior
Either correct behavior or an error message that tells me to run the program again would be preferrable.

Additional context
The confusing stack trace was

Traceback (most recent call last):
  File "/Users/snible/src/prompt-declaration-language/examples/callback/repair_main.py", line 30, in <module>
    pdl_output: PDLResult = pdl.pdl.exec_file(
                            ~~~~~~~~~~~~~~~~~^
        "./repair_prompt.pdl", scope=cast(pdl.pdl.ScopeType, pdl_input)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl.py", line 151, in exec_file
    result = exec_program(program, config, scope, loc, output)
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl.py", line 73, in exec_program
    result, _, scope, trace = process_prog(state, scope, prog, loc)
                              ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 241, in process_prog
    for result, document, final_scope, trace in schedule([doc_generator]):
                                                ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_scheduler.py", line 107, in schedule
    msg = gen.send(v)
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 299, in step_block
    result, background, scope, trace = yield from step_advanced_block(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        state, scope, block, loc
        ^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 350, in step_advanced_block
    raise exc from exc
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 332, in step_advanced_block
    result, background, scope, trace = yield from step_block_body(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        state, scope, block, loc
        ^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 449, in step_block_body
    result, background, scope, trace = yield from step_blocks_of(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/Users/snible/.pyenv/versions/pdl/lib/python3.13/site-packages/pdl/pdl_interpreter.py", line 833, in step_blocks_of
    raise PDLRuntimeError(
    ...<3 lines>...
    ) from exc
pdl.pdl_interpreter.PDLRuntimeError: Error during 'replicate/ibm-granite/granite-3.0-8b-instruct' model call: litellm.APIError: ReplicateException - can only join an iterable

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions