Skip to content

Nix Evaluation Ingestion Hangs and Prematurely Terminates Due to Logic Errors #866

@DarshanCode2005

Description

@DarshanCode2005

Description: The drain_lines utility in the Nix evaluation listner is responsible for batching output lines from nix-eval-jobs for real-time ingestion. It currently suffers from two major logic flaws:

  1. Timeout accumulation: Instead of resetting the timeout to a fixed duration from the current time, it adds to the previous deadline. This causes the timeout to accumulate , potentially causing delays of hours before a batch is yielded if the evaluation process silents.
  2. Incomplete EOF: The function treats any trailing whitespace-only line as a stream EOF signal. Nix evaluations frequently output blank lines, if these occur at the end of a timeout window, the ingestion stops early, leading to incomplite data.

Buggy Code: currently inside src/shared/listeners/nix_evaluation.py

# 1. Timeout Drift Bug
new_deadline = old_deadline + timeout  # BUG: Accumulates instead of resetting
cm.reschedule(new_deadline)

# ...

# 2. Premature EOF Bug
while lines and (
    lines[-1] == b"" or lines[-1].decode("utf8").strip() == ""
):
    eof = True # BUG: Any blank/whitespace line is treated as end-of-stream
    # Drop the last line.
    lines = lines[:-1]

Patch Could be:

# Replace deadline calculation
new_deadline = asyncio.get_running_loop().time() + timeout
# Update trailing line handling
while lines and (lines[-1].decode("utf8").strip() == ""):
     if lines[-1] == b"":
         eof = True
     lines = lines[:-1]

This fix ensures the timer always waits exactly timeout seconds from the last line read, and only declares the stream finished if the reader returns an actual empty byte string (b"").

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions