Remove unused Tintype DAP path argument and update docs

Summary:
Remove the redundant `pytb_file` positional argument and its fail-fast exit code from `tintype-dap-server`; DAP clients already provide the authoritative snapshot path through `launch.pytbPath`. Update the CLI tests and internal launcher documentation accordingly.

Also correct the binary-format parsing example and expand the platform, VS Code, CLI, demo, and serialized collection documentation.

Reviewed By: aperez

Differential Revision: D116650471

fbshipit-source-id: 0698e6ff05597efabbc8fcd0729717fb58b15de5

Author: pdepetro

PYTHON_API.md

@@ -478,7 +478,10 @@ Represents a deserialized complex Python object that doesn't map to a builtin ty
 
 ### `SerializedListObject(list)`
 
-A `list` subclass representing a deserialized list (or list subclass) with a custom repr and additional attributes accessible via `__dict__`.
+A `list` subclass representing a deserialized list, set, or tuple subclass with
+a custom repr and additional attributes accessible via `__dict__`. Collection
+contents are exposed through the list interface even when the captured object
+was a set or tuple subclass.
 
 ### `SerializedDictObject(dict)`
 
@@ -538,17 +541,21 @@ with sampling(interval, path=path):
 
 | Command | Description |
 |---------|-------------|
+| `tintype-dap-server [--listen [HOST:]PORT]` | Serve snapshots through the Debug Adapter Protocol. The default transport is stdio; `--listen` enables a loopback TCP listener. Snapshot paths are supplied by the DAP client's `launch.pytbPath` request field. |
 | `python -m tintype.utils.tintype_dump /path/to/file.pytb` | Dump a snapshot file as structured text or JSON. |
 | `python -m tintype.utils.tintype_viewer /path/to/file.pytb` | Interactive curses-based snapshot file viewer. |
 
 ## Demo Scripts
 
+After installing Tintype from the repository checkout, run these commands from
+the checkout root. The demo modules are not included in installed wheels.
+
 | Script | Description |
 |--------|-------------|
-| `python -m tintype.demo.tintype_demo` | Captures snapshots with complex object graphs. |
-| `python -m tintype.demo.demo_all_threads` | Captures all threads (CPU workers, native sleepers, lock contention). |
-| `python -m tintype.demo.demo_sampling` | Periodic sampling with ALL_THREADS and SINGLE_THREAD modes. |
-| `python -m tintype.demo.stress_all_threads` | Stress test: concurrent snapshots, ephemeral threads, sampling interactions. |
+| `python demo/tintype_demo.py` | Captures snapshots with complex object graphs. |
+| `python demo/demo_all_threads.py` | Captures all threads (CPU workers, native sleepers, lock contention). |
+| `python demo/demo_sampling.py` | Periodic sampling with ALL_THREADS and SINGLE_THREAD modes. |
+| `python demo/stress_all_threads.py` | Stress test: concurrent snapshots, ephemeral threads, sampling interactions. |
 
 ## Related Documentation
 

README.md

@@ -21,8 +21,20 @@ Built as a C++/Python hybrid using pybind11, tintype is designed for minimal ove
 pip install tintype
 ```
 
-**Requirements:** Python ≥ 3.12, a C++20 compiler, and zstd. On macOS, 10.15 or
-newer is required
+**Requirements:** Python ≥ 3.12, a C++20 compiler, and the zstd development
+headers. Tintype currently supports Linux and macOS; Windows is not supported.
+Source builds require macOS 10.15 or newer, while published macOS wheels target
+macOS 11 or newer.
+
+Install the zstd development package before building from source:
+
+```bash
+# Ubuntu/Debian
+sudo apt-get install libzstd-dev
+
+# macOS
+brew install zstd
+```
 
 ## Quick Start
 

dap/cli.py

@@ -25,7 +25,6 @@
 
 import argparse
 import enum
-import os
 import sys
 
 from tintype.dap.server import run_session_on_stdio, serve
@@ -46,9 +45,6 @@ class LauncherExitCode(enum.IntEnum):
     # Generic failure: import error, no snapshots available,
     # unexpected downstream error, etc.
     ERROR = 1
-    # Input validation failure: the ``.pytb`` file is missing, not a
-    # regular file, or SnapshotReader rejected it as malformed.
-    INVALID_PYTB = 2
     # SIGINT / Ctrl-C received before the session cleanly
     # disconnected — 128 + SIGINT(2) per POSIX convention.
     KEYBOARD_INTERRUPT = 130
@@ -87,17 +83,6 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace:
             "Protocol. Defaults to stdio; pass --listen to opt into TCP."
         ),
     )
-    parser.add_argument(
-        "pytb_file",
-        nargs="?",
-        default=None,
-        help=(
-            "Optional path to the .pytb snapshot file. When supplied, the CLI "
-            "fast-fails with INVALID_PYTB if the path is missing or not a "
-            "regular file. When omitted, all path validation is deferred to "
-            "the DAP ``launch`` request's ``pytbPath`` argument."
-        ),
-    )
     parser.add_argument(
         "--listen",
         type=_parse_listen,
@@ -119,33 +104,19 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace:
 def main(argv: list[str] | None = None) -> LauncherExitCode:
     """Run the DAP server over stdio (default) or TCP (``--listen``).
 
-    When ``pytb_file`` is supplied as a positional argument, fast-fails
-    on a missing path with :attr:`LauncherExitCode.INVALID_PYTB` before
-    the transport starts. When omitted, all path validation is
-    deferred to the DAP ``launch`` handler — which surfaces failures
-    as DAP errors the client can render cleanly via
-    ``arguments.pytbPath`` — so the path can live entirely in the
-    launch request body.
+    Snapshot path validation is handled by the DAP ``launch`` handler, which
+    receives the path through the request's ``pytbPath`` argument and can
+    surface failures as DAP errors the client can render cleanly.
 
     Returns:
         :attr:`LauncherExitCode.OK` on clean shutdown.
         :attr:`LauncherExitCode.ERROR` when the session loop returns a
         non-zero status.
-        :attr:`LauncherExitCode.INVALID_PYTB` when a ``pytb_file`` was
-        supplied but does not exist or is not a regular file.
         :attr:`LauncherExitCode.KEYBOARD_INTERRUPT` on SIGINT during the
         serve loop.
     """
     args = _parse_args(argv)
 
-    if args.pytb_file is not None and not os.path.isfile(args.pytb_file):
-        print(
-            f"tintype_dap_server: pytb path does not exist or is not a "
-            f"file: {args.pytb_file}",
-            file=sys.stderr,
-        )
-        return LauncherExitCode.INVALID_PYTB
-
     try:
         if args.listen is not None:
             host, port = args.listen

dap/tests/test_cli.py

@@ -6,15 +6,12 @@
 
 """Tests for ``tintype.dap.cli`` (the DAP-server CLI).
 
-Covers argparse wiring (stdio default + ``--listen`` opt-in), file
-validation, and exit-code mapping. The end-to-end wire-format tests
-live in ``test_integration.py``.
+Covers argparse wiring (stdio default + ``--listen`` opt-in) and exit-code
+mapping. The end-to-end wire-format tests live in ``test_integration.py``.
 """
 
 from __future__ import annotations
 
-import os
-import tempfile
 import unittest
 import unittest.mock
 
@@ -26,78 +23,47 @@ class ListenArgParsingTest(unittest.TestCase):
     """``--listen`` defaults to off (stdio); a value opts into TCP."""
 
     def test_stdio_is_default(self) -> None:
-        args = module._parse_args(["/tmp/snap.pytb"])
-        self.assertEqual(args.pytb_file, "/tmp/snap.pytb")
+        args = module._parse_args([])
         self.assertIsNone(args.listen)
 
     def test_parses_port_only_as_loopback(self) -> None:
-        args = module._parse_args(["/tmp/snap.pytb", "--listen", "1234"])
+        args = module._parse_args(["--listen", "1234"])
         self.assertEqual(args.listen, ("127.0.0.1", 1234))
 
     def test_parses_host_and_port(self) -> None:
-        args = module._parse_args(["/tmp/snap.pytb", "--listen", "127.0.0.1:45678"])
+        args = module._parse_args(["--listen", "127.0.0.1:45678"])
         self.assertEqual(args.listen, ("127.0.0.1", 45678))
 
     def test_parses_ephemeral_port(self) -> None:
-        args = module._parse_args(["/tmp/snap.pytb", "--listen", "0"])
+        args = module._parse_args(["--listen", "0"])
         self.assertEqual(args.listen, ("127.0.0.1", 0))
 
     def test_rejects_malformed_listen(self) -> None:
         with self.assertRaises(SystemExit):
-            module._parse_args(["/tmp/snap.pytb", "--listen", "not-a-port"])
-
-    def test_pytb_file_optional(self) -> None:
-        """``pytb_file`` is optional — when omitted, validation defers to
-        the DAP ``launch`` handler. Argparse must not raise."""
-        args = module._parse_args([])
-        self.assertIsNone(args.pytb_file)
-        self.assertIsNone(args.listen)
-
-
-class MainFileValidationTest(unittest.TestCase):
-    """The CLI fast-fails on obvious path mistakes before touching transports."""
-
-    def test_returns_invalid_pytb_for_missing_pytb_file(self) -> None:
-        self.assertEqual(
-            module.main(["/no/such/file.pytb"]),
-            LauncherExitCode.INVALID_PYTB,
-        )
-
-    def test_returns_invalid_pytb_for_directory_as_pytb(self) -> None:
-        with tempfile.TemporaryDirectory() as tmpdir:
-            self.assertEqual(module.main([tmpdir]), LauncherExitCode.INVALID_PYTB)
+            module._parse_args(["--listen", "not-a-port"])
 
 
 class MainDispatchTest(unittest.TestCase):
     """The CLI dispatches to stdio by default and TCP under ``--listen``."""
 
-    def _make_pytb(self) -> str:
-        f = tempfile.NamedTemporaryFile(suffix=".pytb", delete=False)
-        f.write(b"not a real tintype file, but exists")
-        f.close()
-        self.addCleanup(lambda: os.unlink(f.name))
-        return f.name
-
     def test_stdio_default_calls_run_session_on_stdio(self) -> None:
-        path = self._make_pytb()
         with (
             unittest.mock.patch(
                 "tintype.dap.cli.run_session_on_stdio", return_value=0
             ) as stdio_mock,
             unittest.mock.patch("tintype.dap.cli.serve") as serve_mock,
         ):
-            rc = module.main([path])
+            rc = module.main([])
         self.assertEqual(rc, LauncherExitCode.OK)
         stdio_mock.assert_called_once_with()
         serve_mock.assert_not_called()
 
     def test_listen_routes_to_serve(self) -> None:
-        path = self._make_pytb()
         with (
             unittest.mock.patch("tintype.dap.cli.serve", return_value=0) as serve_mock,
             unittest.mock.patch("tintype.dap.cli.run_session_on_stdio") as stdio_mock,
         ):
-            rc = module.main([path, "--listen", "127.0.0.1:4242"])
+            rc = module.main(["--listen", "127.0.0.1:4242"])
         self.assertEqual(rc, LauncherExitCode.OK)
         serve_mock.assert_called_once_with(host="127.0.0.1", port=4242)
         stdio_mock.assert_not_called()
@@ -106,47 +72,18 @@ def test_nonzero_return_maps_to_error(self) -> None:
         """A non-zero session-loop return maps to
         :attr:`LauncherExitCode.ERROR` so unexpected shutdown rcs
         surface as a generic error rather than leaking raw values."""
-        path = self._make_pytb()
         with unittest.mock.patch(
             "tintype.dap.cli.run_session_on_stdio", return_value=7
         ):
-            self.assertEqual(module.main([path]), LauncherExitCode.ERROR)
+            self.assertEqual(module.main([]), LauncherExitCode.ERROR)
 
     def test_keyboard_interrupt_maps_to_exit_code(self) -> None:
         """SIGINT during the serve loop surfaces as the conventional
         128 + SIGINT(2) exit code via the enum."""
-        path = self._make_pytb()
         with unittest.mock.patch(
             "tintype.dap.cli.run_session_on_stdio", side_effect=KeyboardInterrupt
         ):
-            self.assertEqual(module.main([path]), LauncherExitCode.KEYBOARD_INTERRUPT)
-
-    def test_no_pytb_dispatches_to_stdio(self) -> None:
-        """``pytb_file`` may be omitted entirely; the CLI's existence
-        check is skipped and the transport launches as usual. Path
-        validation is left to ``handle_launch``."""
-        with (
-            unittest.mock.patch(
-                "tintype.dap.cli.run_session_on_stdio", return_value=0
-            ) as stdio_mock,
-            unittest.mock.patch("tintype.dap.cli.serve") as serve_mock,
-        ):
-            rc = module.main([])
-        self.assertEqual(rc, LauncherExitCode.OK)
-        stdio_mock.assert_called_once_with()
-        serve_mock.assert_not_called()
-
-    def test_no_pytb_with_listen_dispatches_to_serve(self) -> None:
-        """Same as ``test_no_pytb_dispatches_to_stdio`` but for the
-        TCP transport branch — ``--listen`` works without a positional."""
-        with (
-            unittest.mock.patch("tintype.dap.cli.serve", return_value=0) as serve_mock,
-            unittest.mock.patch("tintype.dap.cli.run_session_on_stdio") as stdio_mock,
-        ):
-            rc = module.main(["--listen", "0"])
-        self.assertEqual(rc, LauncherExitCode.OK)
-        serve_mock.assert_called_once_with(host="127.0.0.1", port=0)
-        stdio_mock.assert_not_called()
+            self.assertEqual(module.main([]), LauncherExitCode.KEYBOARD_INTERRUPT)
 
 
 class LauncherExitCodeTest(unittest.TestCase):
@@ -157,14 +94,13 @@ class LauncherExitCodeTest(unittest.TestCase):
     def test_values_match_posix_conventions(self) -> None:
         self.assertEqual(LauncherExitCode.OK.value, 0)
         self.assertEqual(LauncherExitCode.ERROR.value, 1)
-        self.assertEqual(LauncherExitCode.INVALID_PYTB.value, 2)
         self.assertEqual(LauncherExitCode.KEYBOARD_INTERRUPT.value, 130)
 
     def test_is_int_enum_so_values_pass_through_sys_exit(self) -> None:
         # IntEnum members compare equal to their underlying int value,
         # which ``sys.exit()`` needs for the entrypoint's
         # ``sys.exit(main())`` call to produce the expected OS exit code.
-        self.assertEqual(LauncherExitCode.INVALID_PYTB, 2)
+        self.assertEqual(LauncherExitCode.ERROR, 1)
         self.assertEqual(int(LauncherExitCode.OK), 0)
 
 

demo/tintype_chain_demo.py

@@ -49,3 +49,7 @@ def main() -> None:
         },
     )
     process_request()
+
+
+if __name__ == "__main__":
+    main()

demo/tintype_demo.py

@@ -201,3 +201,7 @@ def main() -> None:
         },
     )
     _print_stats()
+
+
+if __name__ == "__main__":
+    main()

demo/tintype_truncated_demo.py

@@ -165,3 +165,7 @@ def main() -> None:
         f"View with: python -m tintype.utils.tintype_viewer {OUTPUT_PATH}",
         file=sys.stderr,
     )
+
+
+if __name__ == "__main__":
+    main()

snapshot_lib/FILE_FORMAT.md

@@ -10,7 +10,10 @@ The Tintype file stores snapshots of Python traceback information including:
 - Source file contents for referenced files
 - User-provided metadata
 
-The file is compressed using zstd. All positions (Pos) are absolute byte offsets from the start of the **uncompressed** data. All offsets (Offset) are relative to a section start (e.g., object heap offsets are relative to the object heap start).
+Output files are zstd-compressed by default, but callers can request an
+uncompressed file. All positions (Pos) are absolute byte offsets from the start
+of the **uncompressed** data. All offsets (Offset) are relative to a section
+start (e.g., object heap offsets are relative to the object heap start).
 
 All multi-byte integers are stored in little-endian byte order. All on-disk structs use `#pragma pack(push, 1)` to eliminate compiler-inserted alignment padding. This makes the binary format portable across platforms (e.g., files written on Linux can be read on macOS).
 
@@ -383,7 +386,9 @@ To read the statistics:
 ## Reading the File
 
 ### Decompression
-The entire file is zstd compressed. Decompress the full file before parsing.
+Read the first four bytes as a little-endian `uint32`. If they equal the magic
+number, the file is already uncompressed and can be parsed directly. Otherwise,
+treat the file as a zstd frame and decompress it fully before parsing.
 
 ### Output Compaction
 During writing, the working file has a zero-filled gap between the snapshot records section and the object heap (used to allow the snapshot records section to grow without relocating the heap). When the output file is produced, this gap is stripped: the snapshot records are immediately followed by the object heap data, and the `FileHeader` position fields (`objectHeapPos`, `fileTablePos`, `envPos`, `manifestPos`, `metadataPos`, `statsPos`) are adjusted to reflect the compacted layout. The `lastSnapshotPos` and all positions within snapshot records (`prevSnapshotPos`, `objectMapPos`) are not adjusted because they fall before the gap. Readers see a contiguous file with no gap.
@@ -415,10 +420,13 @@ import zstandard
 
 def read_snapshot_file(path):
     with open(path, 'rb') as f:
-        compressed = f.read()
+        file_data = f.read()
 
-    dctx = zstandard.ZstdDecompressor()
-    data = dctx.decompress(compressed)
+    if file_data[:4] == struct.pack('<I', 0x50595442):
+        data = file_data
+    else:
+        dctx = zstandard.ZstdDecompressor()
+        data = dctx.decompress(file_data)
 
     # Read FileHeader (100 bytes, packed)
     magic, version = struct.unpack_from('<II', data, 0)
@@ -431,19 +439,19 @@ def read_snapshot_file(path):
      manifest_pos, manifest_size,
      metadata_pos, metadata_size,
      stats_pos, stats_count) = struct.unpack_from(
-        '<QIQIQQQQQQQI', data, 8)
+        '<QIQQIQQQQQQQI', data, 8)
 
     # Read metadata (raw JSON string, size from header)
     metadata_json = data[metadata_pos:metadata_pos + metadata_size]
 
-    # Read most recent snapshot (32-byte header)
+    # Read most recent snapshot (33-byte header)
     pos = last_snapshot_pos
-  timestamp, prev_pos, stacktrace_count, obj_map_pos, obj_map_count, flags = \
+    timestamp, prev_pos, stacktrace_count, obj_map_pos, obj_map_count, flags = \
         struct.unpack_from('<QQIQIB', data, pos)
     pos += 33  # sizeof(SnapshotRecordHeader)
     is_truncated = (flags & 0x01) != 0
 
-    # Read stacktraces (37-byte header + thread name + frames)
+    # Read stacktraces (41-byte header + thread name + frames)
     for s in range(stacktrace_count):
         id, frame_count, exception_python_id, cause_id, context_id, st_flags, \
             thread_name_length = struct.unpack_from('<QIQQQ BI', data, pos)