Skip to content

Add bounded concurrent URL transport - #822

Merged
chubes4 merged 2 commits into
mainfrom
feat/820-concurrent-http-transport
Aug 3, 2026
Merged

Add bounded concurrent URL transport#822
chubes4 merged 2 commits into
mainfrom
feat/820-concurrent-http-transport

Conversation

@chubes4

@chubes4 chubes4 commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Fixes #820

Summary

  • Adds Static_Site_Importer_URL_Fetcher::fetch_many() with explicit global and per-origin in-flight limits, caller-keyed deterministic results, deadlines, and cancellation.
  • Uses nonblocking IP-pinned sockets while retaining hostname SNI and Host handling; every redirect target is revalidated before scheduling.
  • Adds deterministic transport-driver coverage for limits, completion reordering, redirect private-IP rejection, deadline cancellation, and serial-equivalent responses.

Evidence

  • php tests/smoke-url-concurrent-fetcher.php passed.
  • Focused URL runtime, collector, and batch smoke tests passed.
  • homeboy review test --changed-since origin/main --placement local --summary passed: 12/12, run 898852d6-abd6-4686-acf4-b2bd53010e41.
  • Scoped Homeboy lint/PHPStan passed for the new handle and transport test files; fetcher scoped PHPStan passed before final formatting.
  • PHP syntax and git diff --check passed before commit.

Compatibility and Security

The existing single-fetch API remains unchanged. Concurrent work validates public IPs before each connection, pins the socket to a validated address, preserves hostname TLS SNI/Host, applies existing redirect, header, body, content-type, and timeout policies, and closes active sockets when the shared deadline is exhausted.

AI Assistance

OpenAI gpt-5.6-sol via OpenCode implemented the bounded concurrent validated-IP transport and deterministic tests. Chris Huber reviewed and remains responsible for every line.

chubes4 added 2 commits August 3, 2026 10:41
OpenAI gpt-5.6-sol via OpenCode implemented the bounded concurrent validated-IP transport and deterministic tests. Chris Huber reviewed and remains responsible for every line.
OpenAI gpt-5.6-sol via OpenCode restored focused WordPress formatting and verified the concurrent URL transport. Chris Huber reviewed and remains responsible for every line.
@chubes4
chubes4 merged commit 1d9f07d into main Aug 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add bounded concurrent validated-IP HTTP transport

1 participant