Skip to content

Add Parallel Static Asset Loading #64

@jeremyandrews

Description

@jeremyandrews

Problem

The validate_and_load_static_assets function in goose-eggs currently loads all static assets sequentially. This doesn't match how browsers actually work - they open multiple parallel connections (or use HTTP/2 streams) to download assets concurrently.
On pages with many assets (photo galleries, team pages with avatars, etc.), the sequential approach produces transaction times that are longer than what users actually experience. This makes the load test less representative of real-world behavior.

Proposed Solution

Add parallel fetching to validate_and_load_static_assets with sensible defaults based on HTTP version, plus an option to configure the behavior.

Implementation

Introduce a ParallelLoadingStrategy enum:

rustpub enum ParallelLoadingStrategy {
    /// Current behavior: load assets sequentially
    Sequential,
    /// Auto-detect based on HTTP version:
    /// - HTTP/1.1: 6 concurrent requests
    /// - HTTP/2+: 100 concurrent requests
    Auto,
    /// Fixed number of parallel requests
    Fixed(usize),
}

// Updated function signature:
pub async fn validate_and_load_static_assets(
    transaction: &mut GooseTransaction,
    goose_request: GooseRequest,
    strategy: Option<ParallelLoadingStrategy>, // Defaults to Auto
) -> GooseTaskSetResult {
    // ...
}

Default Behavior

  • HTTP/1.1: 6 parallel requests per domain (typical browser limit)
  • HTTP/2 and newer: 100 parallel requests (reflecting multiplexed streams)

Example Implementation

// After parsing asset URLs from parent page...

let resolved_strategy = strategy.unwrap_or(ParallelLoadingStrategy::Auto);
let concurrency_limit = match resolved_strategy {
    ParallelLoadingStrategy::Sequential => 1,
    ParallelLoadingStrategy::Auto => {
        match parent_response.version() {
            reqwest::Version::HTTP_11 | reqwest::Version::HTTP_10 => 6,
            _ => 100, // HTTP/2, HTTP/3
        }
    }
    ParallelLoadingStrategy::Fixed(limit) => limit,
};

// Execute requests with concurrency limit
use futures::stream::StreamExt;

let results = futures::stream::iter(asset_futures)
    .buffer_unordered(concurrency_limit)
    .collect::<Vec<_>>()
    .await;

Alternatives Considered

  • Keep sequential loading: Simple but doesn't represent real browser behavior
  • Unlimited parallel requests: Could overwhelm the client or create unrealistic load patterns

Additional Context

This change would make Goose's load patterns more closely match actual browser behavior when loading pages with multiple static assets.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions