Merge upstream/main into gos-282-no-content

Keep both tests added after the cache-reload test: no-content and hard reload.

Author: jaytaph

README.md

@@ -12,27 +12,31 @@ the [contribution guide](CONTRIBUTING.md).
 Gosub is a modular, embeddable browser engine. The primary entry point is `GosubEngine` in the
 [`gosub_engine`](crates/gosub_engine) crate. You provide a render backend and a compositor; the
 engine owns a multi-zone/tab model, an async networking stack, cookie and storage isolation per
-zone, and an event bus. Your user-agent (UA) drives everything via `TabCommand` and reacts to
+zone, sandboxed child processes for parsing and rendering, and an event bus. Your user-agent (UA) drives everything via `TabCommand` and reacts to
 `EngineEvent`.
 
 **Core components:**
 
 | Crate | Role |
 |---|---|
-| `gosub_engine` | `GosubEngine` — the unified entry point |
+| `gosub_engine` | `GosubEngine` - the unified entry point |
 | `gosub_interface` | Shared traits wiring the components together (the config system) |
 | `gosub_html5` | HTML5 tokenizer / parser |
 | `gosub_css3` | CSS3 tokenizer / parser |
-| [`gosub-sonar`](https://github.com/gosub-io/gosub-sonar) | Networking stack (async, streaming, priority-scheduled) — external crate |
+| [`gosub-sonar`](https://github.com/gosub-io/gosub-sonar) | Networking stack (async, streaming, priority-scheduled) - external crate |
 | `gosub_lattice` | CSS table layout |
-| `gosub_render_pipeline` | Render pipeline — layout (Taffy), stages, tiling, compositor |
+| `gosub_render_pipeline` | Render pipeline - layout (Taffy), stages, tiling, compositor |
 | `gosub_renderer_cairo` | Cairo render backend (CPU) |
 | `gosub_renderer_skia` | Skia render backend (CPU / GPU) |
 | `gosub_renderer_vello` | Vello / wgpu render backend (GPU) |
-| `gosub_fontmanager` | Font system — text shaping and measurement |
-| `gosub_jsapi` | Browser Web API implementations (console, fetch, DOM, …) |
+| `gosub_renderer_dynamic` | Picks one of the render backends at runtime |
+| `gosub_svg` | SVG rendering |
+| `gosub_fontmanager` | Font system - text shaping and measurement |
+| `gosub_jsapi` | Browser Web API implementations (console, fetch, DOM, ...) |
 | `gosub_v8` | V8 JavaScript engine bindings |
 | `gosub_config` | Configuration store |
+| `gosub_sandbox` | OS-level sandboxing: syscall filters, namespaces, child-process spawning |
+| `gosub_ipc` | Transport between the engine and its child processes |
 
 For the full crate listing see [`docs/crates.md`](docs/crates.md).
 
@@ -41,76 +45,92 @@ For the full crate listing see [`docs/crates.md`](docs/crates.md).
 
 The engine is under active development. What works today:
 
-- **Multi-zone / multi-tab model** — zones isolate cookies and storage; tabs are controlled via `TabCommand`
-- **Async networking** — streaming HTTP fetcher with priority queues, inflight coalescing, redirect handling, and per-zone cookie isolation
-- **Event-driven UA interface** — `TabCommand` flows in; `EngineEvent` (navigation, downloads, redraw, crashes) flows out on a control bus, with per-resource detail on a separate opt-in stream
-- **HTML5 and CSS3 parsing** — spec-compliant parsers for both, including shadow DOM (parsing, flat-tree traversal, CSS scoping)
-- **Resource pipeline** — parser-driven sub-resource discovery; stylesheets, images, web fonts and data URLs all fetched through one fetcher, with hierarchical cancellation
-- **Layout and paint** — block, inline, flex and grid via Taffy, floats, absolute and fixed positioning, and CSS tables via `gosub_lattice`
-- **Form controls** — text editing, and colour / date / time pickers opened by the embedder through `EngineEvent::PickerRequested`
-- **Pluggable render backends** — Null (headless), Cairo (GTK4), Skia (CPU / GPU), Vello (wgpu)
-- **Conformance measurement** — a gated web-platform-tests run in CI, plus a reftest runner; see [docs/wpt.md](docs/wpt.md) for where the numbers stand
-
-**[`docs/status.md`](docs/status.md) is the detailed version of this list** — per component,
+- **Multi-zone / multi-tab model** - zones isolate cookies and storage; tabs are controlled via `TabCommand`
+- **Async networking** - streaming HTTP fetcher with priority queues, inflight coalescing, redirect handling, and per-zone cookie isolation; `gemini`, `gopher` and `finger` next to `http(s)`, `data` and `file`
+- **Process isolation** - on Linux, networking, cookies, image decoding, file-backed `localStorage` and rendering (with Parley or cosmic-text fonts) run in sandboxed child processes, on by default once the embedder calls `child_process::dispatch_with::<C>()` first in `main`; see [docs/process-isolation.md](docs/process-isolation.md)
+- **Web security policies** - CORS, mixed content, referrer policy, ORB, an SSRF policy over address spaces, and per-document security state
+- **Encrypted storage** - every SQLite store (cookies, localStorage, places, config) opens with an optional SQLCipher key
+- **Event-driven UA interface** - `TabCommand` flows in; `EngineEvent` (navigation, downloads, redraw, crashes) flows out on a control bus, with per-resource detail on a separate opt-in stream
+- **HTML5 and CSS3 parsing** - spec-compliant parsers for both, including shadow DOM (parsing, flat-tree traversal, CSS scoping)
+- **Resource pipeline** - parser-driven sub-resource discovery; stylesheets, images, web fonts and data URLs all fetched through one fetcher, with hierarchical cancellation
+- **Layout and paint** - block, inline, flex and grid via Taffy, floats, absolute and fixed positioning, and CSS tables via `gosub_lattice`
+- **Mobile** - `<meta name="viewport">` layout width and pinch zoom through a visual viewport; the engine runs on Android, in-process
+- **Form controls** - text editing, and colour / date / time pickers opened by the embedder through `EngineEvent::PickerRequested`
+- **Pluggable render backends** - Null (headless), Cairo (GTK4), Skia (CPU / GPU), Vello (wgpu)
+- **Conformance measurement** - a gated web-platform-tests run in CI, plus a reftest runner; see [docs/wpt.md](docs/wpt.md) for where the numbers stand
+
+**[`docs/status.md`](docs/status.md) is the detailed version of this list** - per component,
 what works and what does not, with the numbers.
 
 Not there yet: scripting is not wired into the engine. The V8 and web-API crates exist and
-build, but no page runs JavaScript — see [docs/javascript.md](docs/javascript.md). WPT drives
+build, but no page runs JavaScript - see [docs/javascript.md](docs/javascript.md). WPT drives
 the DOM through a separate test-only QuickJS binding (`gosub_domjs`).
 
 
+## Gosub Beacon
+
+[Gosub Beacon](https://github.com/gosub-io/gosub-beacon) is the browser built on this engine,
+for Linux, macOS, Windows and Android. It builds against this repository's `beacon` branch, not
+`main`. Builds are on its [releases page](https://github.com/gosub-io/gosub-beacon/releases).
+
+
 ## Documentation
 
 **[`docs/README.md`](docs/README.md) indexes every page under `/docs`.** The pages below are the
 ones worth reading first.
 
 **Getting started**
 
-- [Tutorial](docs/tutorial.md) — start the engine, open a tab, navigate, handle events
-- [Configuration](docs/configuration.md) — choosing a render backend and font system
-- [Running the examples](docs/examples.md) — headless, GUI (winit / GTK4 / egui), and component tools
-- [Headless usage](docs/headless.md) — rendering real pages without a window
-- [WebAssembly](docs/webassembly.md) — compile and run the engine in the browser
-- [Development](docs/development.md) — tests and benchmarks
+- [Tutorial](docs/tutorial.md) - start the engine, open a tab, navigate, handle events
+- [Configuration](docs/configuration.md) - choosing a render backend and font system
+- [Running the examples](docs/examples.md) - headless, GUI (winit / GTK4 / egui), and component tools
+- [Headless usage](docs/headless.md) - rendering real pages without a window
+- [WebAssembly](docs/webassembly.md) - compile and run the engine in the browser
+- [Development](docs/development.md) - tests and benchmarks
 
 **Reference**
 
-- [What the engine can do today](docs/status.md) — per-component capability and gaps
-- [Crates](docs/crates.md) — the workspace crate layout, and where to find anything
-- [Component tools](docs/binaries.md) — the standalone `cargo run --bin …` tools
+- [Embedder API](docs/embedder-api.md) - every `TabCommand` and `EngineEvent`, and what to do
+  with each
+- [What the engine can do today](docs/status.md) - per-component capability and gaps
+- [Crates](docs/crates.md) - the workspace crate layout, and where to find anything
+- [Component tools](docs/binaries.md) - the standalone `cargo run --bin ...` tools
 
 **Architecture**
 
-- [The two worlds](docs/two-worlds.md) — the two parallel document/style models and the seam
+- [The two worlds](docs/two-worlds.md) - the two parallel document/style models and the seam
   that joins them. Read this before diving into either.
-- [Interface trait families](docs/interface.md) — `ModuleConfiguration` and the `Has*` traits
+- [Interface trait families](docs/interface.md) - `ModuleConfiguration` and the `Has*` traits
   that wire the workspace together
-- [Module configuration](docs/moduleconfig.md) — the same picture from an embedder's side:
+- [Module configuration](docs/moduleconfig.md) - the same picture from an embedder's side:
   what `DefaultRenderConfig` wires for you and how to go fully custom
-- [Zones and tabs](docs/zones-and-tabs.md) — zones as isolated profiles, tabs as independent
+- [Zones and tabs](docs/zones-and-tabs.md) - zones as isolated profiles, tabs as independent
   worker tasks, and the command/event flow between them
-- [Resource pipelines](docs/resource-pipeline.md) — how fetched bytes become typed assets
-- [CSS internals](docs/css.md) — `gosub_css3` from text to computed value
-- [HTML5 parsing](docs/html5.md) — tokenizer, tree builder, arena DOM
+- [Process isolation](docs/process-isolation.md) - the broker, its sandboxed children, and
+  the embedder contract that turns them on; [security assessment](docs/security-assessment.md)
+  reads the same design as an attacker would
+- [Resource pipelines](docs/resource-pipeline.md) - how fetched bytes become typed assets
+- [CSS internals](docs/css.md) - `gosub_css3` from text to computed value
+- [HTML5 parsing](docs/html5.md) - tokenizer, tree builder, arena DOM
 - [Render pipeline](docs/render-pipeline/README.md)
-- [Lattice table layout](docs/lattice.md) · [Fonts](docs/fonts.md) · [JavaScript stack](docs/javascript.md)
-- [Networking — architecture](docs/network/net-architecture.md) and [design notes](docs/network/net-design.md)
+- [Lattice table layout](docs/lattice.md) - [Fonts](docs/fonts.md) - [JavaScript stack](docs/javascript.md)
+- [Networking - architecture](docs/network/net-architecture.md) and [design notes](docs/network/net-design.md)
 - [Cookies](docs/cookies.md)
 - [Storage (local / session)](docs/datastores.md)
-- [Pump](docs/network/pump.md) — moving HTTP stream data to targets
+- [Pump](docs/network/pump.md) - moving HTTP stream data to targets
 
 **Conformance**
 
-- [**WPT quickstart**](docs/wpt-quickstart.md) — clone to a fixed test in about ten minutes.
+- [**WPT quickstart**](docs/wpt-quickstart.md) - clone to a fixed test in about ten minutes.
   The fastest way to make a first contribution.
-- [Web-platform-tests](docs/wpt.md) — the harness in full: what is bound, the expectations
+- [Web-platform-tests](docs/wpt.md) - the harness in full: what is bound, the expectations
   format, the reftest runner, and what CI does
 
 
 ## Contributing
 
 We welcome contributions. Because the engine is still taking shape, a lot of work is exploratory
-— building proofs-of-concept, reading specs, and making architectural decisions — rather than
+- building proofs-of-concept, reading specs, and making architectural decisions - rather than
 pure coding.
 
 Join us on [Zulip](https://chat.developer.gosub.io) or [Discord](https://chat.gosub.io) before

crates/gosub_engine/src/engine/engine.rs

@@ -1373,6 +1373,37 @@ mod tests {
         engine.shutdown().await.expect("shutdown");
     }
 
+    /// A detached tab gives its `max_tabs` slot back at once, before its worker has
+    /// stopped, so a zone that keeps opening and closing tabs never runs out.
+    #[tokio::test]
+    async fn detached_tab_frees_its_slot() {
+        let mut engine = engine_with_max_zones(1);
+        let _join = tokio::spawn(engine.start().expect("start"));
+        let mut zone = engine.zone_builder().services(services()).create().expect("zone");
+        let max = ZoneConfig::default().max_tabs;
+
+        let mut tabs = Vec::new();
+        for _ in 0..max {
+            tabs.push(zone.tab_builder().create().await.expect("tab"));
+        }
+        assert!(matches!(
+            zone.tab_builder().create().await,
+            Err(EngineError::TabLimitExceeded)
+        ));
+
+        for _ in 0..max * 2 {
+            let closing = zone.detach_tab(tabs.remove(0).tab_id).expect("known tab");
+            assert!(zone.detach_tab(crate::tab::TabId::new()).is_none(), "unknown id");
+            tabs.push(zone.tab_builder().create().await.expect("the slot is free"));
+            tokio::time::timeout(Duration::from_secs(10), closing)
+                .await
+                .expect("the worker stops");
+        }
+
+        engine.close_zone(zone).await;
+        engine.shutdown().await.expect("shutdown");
+    }
+
     /// A page served as an attachment is offered as a download; `RenderDownload` overrides
     /// that and loads the already-spooled body as the page. No second request is made.
     #[tokio::test(flavor = "current_thread")]
@@ -3971,6 +4002,93 @@ mod tests {
         assert!(tab.can_go_back(), "the cursor went back to /page's entry");
     }
 
+    /// A reload ignoring the cache asks for a fresh copy all the way to the server
+    /// (`Cache-Control` and `Pragma: no-cache`, no validators); a plain reload does not.
+    #[tokio::test]
+    async fn reload_ignoring_cache_sends_no_cache() {
+        use crate::events::{NavigationEvent, TabCommand};
+        use cow_utils::CowUtils;
+        use tokio::io::{AsyncReadExt, AsyncWriteExt};
+
+        let (heads_tx, mut heads) = mpsc::unbounded_channel::<String>();
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let port = listener.local_addr().unwrap().port();
+        tokio::spawn(async move {
+            while let Ok((mut stream, _)) = listener.accept().await {
+                let heads_tx = heads_tx.clone();
+                tokio::spawn(async move {
+                    // The whole head: the test reads its headers, not only the request line.
+                    let mut buf = Vec::new();
+                    let mut chunk = [0u8; 4096];
+                    while !buf.windows(4).any(|w| w == b"\r\n\r\n") {
+                        match stream.read(&mut chunk).await {
+                            Ok(0) | Err(_) => break,
+                            Ok(n) => buf.extend_from_slice(&chunk[..n]),
+                        }
+                    }
+                    let head = String::from_utf8_lossy(&buf).cow_to_ascii_lowercase().into_owned();
+                    if head.starts_with("get /page ") {
+                        let _ = heads_tx.send(head);
+                    }
+                    let page = "<html><body>page</body></html>";
+                    let response = format!(
+                        "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nCache-Control: max-age=3600\r\nETag: \"v1\"\r\n\
+                         Content-Length: {}\r\nConnection: close\r\n\r\n{page}",
+                        page.len()
+                    );
+                    let _ = stream.write_all(response.as_bytes()).await;
+                });
+            }
+        });
+
+        let mut engine = engine_with_max_zones(1);
+        let mut events = engine.subscribe_events();
+        let _join = tokio::spawn(engine.start().expect("start"));
+        let mut zone = engine.zone_builder().services(services()).create().expect("zone");
+        let tab = zone.tab_builder().create().await.expect("tab");
+
+        async fn finished(events: &mut broadcast::Receiver<EngineEvent>) {
+            tokio::time::timeout(Duration::from_secs(10), async {
+                loop {
+                    match events.recv().await {
+                        Ok(EngineEvent::Navigation {
+                            event: NavigationEvent::Finished { .. },
+                            ..
+                        }) => return,
+                        Ok(_) => continue,
+                        Err(e) => panic!("event stream closed: {e}"),
+                    }
+                }
+            })
+            .await
+            .expect("timed out waiting for the load to finish");
+        }
+        let no_cache = |head: &str| head.contains("cache-control: no-cache") && head.contains("pragma: no-cache");
+
+        tab.navigate(format!("http://127.0.0.1:{port}/page"))
+            .await
+            .expect("navigate");
+        // The server records a request before it answers, so by `Finished` it is queued.
+        finished(&mut events).await;
+        assert!(!no_cache(&heads.try_recv().unwrap()), "a first load is a plain one");
+
+        tab.send(TabCommand::Reload { ignore_cache: true })
+            .await
+            .expect("reload");
+        finished(&mut events).await;
+        let head = heads.try_recv().expect("the hard reload was answered from the cache");
+        assert!(no_cache(&head), "hard reload without no-cache:\n{head}");
+        assert!(!head.contains("if-none-match"), "hard reload sent a validator:\n{head}");
+
+        tab.send(TabCommand::Reload { ignore_cache: false })
+            .await
+            .expect("reload");
+        finished(&mut events).await;
+        if let Ok(head) = heads.try_recv() {
+            assert!(!no_cache(&head), "a plain reload asked for no-cache:\n{head}");
+        }
+    }
+
     /// Session history end to end: two navigations push two entries, GoBack moves the cursor
     /// (announced immediately via HistoryChanged) and refetches the first page, GoForward
     /// returns to the second. Verifies the tree from the embedder's point of view only.