Skip to content

SAMPLES.md

Latest commit

 

History

History
287 lines (211 loc) · 17.6 KB

SAMPLES.md

File metadata and controls

287 lines (211 loc) · 17.6 KB

Samples

The samples/ directory contains ten example programs (plus three demo scripts and a Python server), from a one-shot hello-world to a full data-source-driven pipeline with frame post-processing and live streaming over pipes / WebSocket. Each is built as an independent CMake target so you can open any one in isolation.

samples/
  basic_read.cpp              vtx_sample_read                 reader smoke test
  basic_write.cpp             vtx_sample_write                writer smoke test
  basic_diff.cpp              vtx_sample_diff                 hash-based diff
  ready_api.cpp               vtx_sample_ready_api            chunk-0 ready signalling
  post_process_write.cpp      vtx_sample_post_process_write   writer-side post-processor
  generate_replay.cpp         vtx_sample_generate             arena simulator -> 3 data sources
  advance_write.cpp           vtx_sample_advance_write        3 data sources -> 3 .vtx replays (with post-processor)

  pipe_producer.cpp           vtx_sample_pipe_producer        emits length-prefixed JSON frames on stdout
  pipe_consumer.cpp           vtx_sample_pipe_consumer        ingests pipe frames -> .vtx (stdin / client / server modes)
  websocket_consumer.cpp      vtx_sample_websocket_consumer   ingests WebSocket frames -> .vtx (ws:// + wss://)

  pipe_demo.bat               anonymous-pipe demo: producer | consumer (single launcher)
  named_pipe_vtx.bat          named-pipe demo: VTX side       (run first; creates the pipe and waits)
  named_pipe_producer.bat     named-pipe demo: producer side  (retry-connects, streams until Enter)
  websocket_server.py         minimal Python WebSocket server pushing synthetic JSON frames

  arena_mappings.h            JSON data model + JsonMapping<T> + StructMapping<T> + StructFrameBinding<ArenaFrame>
  arena_generated.h           autogenerated schema constants + typed Views + Mutators + ForEachX helpers
  schemas/
    arena_data.proto          Protobuf game schema (namespace arena_pb)
    arena_data.fbs            FlatBuffers game schema (namespace arena_fb)
  content/
    writer/arena/
      arena_schema.json       VTX property schema (consumed by the writer)
      arena_replay_data.*     data-source files produced by vtx_sample_generate
    reader/arena/
      arena_from_*_ds.vtx     .vtx replays produced by vtx_sample_advance_write

All samples set VS_DEBUGGER_WORKING_DIRECTORY to samples/, so relative paths like content/reader/arena/arena_from_fbs_ds.vtx just work when run from the IDE or from the samples/ directory.

Quick start

Build everything in one shot and run the arena pipeline end-to-end:

cmake --build build --config Release --target vtx_sample_generate vtx_sample_advance_write

cd samples
..\build\bin\Release\vtx_sample_generate.exe
..\build\bin\Release\vtx_sample_advance_write.exe
..\build\bin\Release\vtx_sample_read.exe

The samples

1. basic_read -- reader smoke test

Opens a .vtx file via VTX::OpenReplayFile(), prints header and footer metadata, and enumerates the buckets in frame 0.
Links against vtx_reader (pulls vtx_common)
Default input content/reader/arena/arena_from_fbs_ds.vtx
Override vtx_sample_read <path-to.vtx>

Key APIs exercised: VTX::OpenReplayFile, IVtxReaderFacade::GetHeader / GetFooter / GetFrameSync, Frame::GetBuckets.

2. basic_write -- writer smoke test

Writes 100 synthetic frames (no external data source, no schema-driven mapping). Useful as the simplest possible template for the writer API.
Links against vtx_writer
argv[1] schema JSON path (default: content/writer/arena/arena_schema.json)
argv[2] output .vtx path (default: sample_output.vtx)

Key APIs exercised: VTX::WriterFacadeConfig, VTX::CreateFlatBuffersWriterFacade, IVtxWriterFacade::RecordFrame / Flush / Stop, raw PropertyContainer construction.

3. basic_diff -- cheap per-frame delta

Reads two consecutive frames and flags entities whose xxHash64 content_hash changed. Shows the first step of a diff workflow; a full structural diff would use vtx_differ's DiffRawFrames() (see SDK_API.md).
Links against vtx_reader, vtx_differ
Default input content/reader/arena/arena_from_fbs_ds.vtx

4. generate_replay -- arena simulator + data-source producer

Simulates a 60-second 5v5 arena match (3600 frames at 60 FPS), then serializes the result into three data-source files in different wire formats. No .vtx output is produced here -- that is advance_write's job.
Links against vtx_writer
Needs codegen Yes -- arena schemas (see below)
Produces content/writer/arena/arena_replay_data.json
content/writer/arena/arena_replay_data.proto.bin
content/writer/arena/arena_replay_data.fbs.bin

The simulation models player patrols, projectile fire, kill/respawn cycles, and match-phase transitions (warmup / playing / roundend). It's deterministic (seed 42) so every run produces byte-identical data.

5. advance_write -- data-source consumer + writer driver

Reads the three data-source files produced by generate_replay, maps each record into a VTX::Frame using SDK-native integration primitives, runs a writer-side post-processor (ArenaConsistencyProcessor using the codegen-generated typed API), and writes one .vtx replay per source. The same processor instance is registered per pipeline -- demonstrating that frame post-processing is orthogonal to the source format: the same logic applies whether the frame came from JSON, Protobuf, or FlatBuffers.
Links against vtx_writer, vtx_common
Needs codegen Yes -- arena schemas + arena_generated.h (now also includes *Mutator classes and ForEachX helpers)
Reads all three content/writer/arena/arena_replay_data.* files
Produces content/reader/arena/arena_from_json_ds.vtx
content/reader/arena/arena_from_proto_ds.vtx
content/reader/arena/arena_from_fbs_ds.vtx

Three different mapping strategies, one per format -- all mirror the pattern used by real integrations (tools/integrations/rl/, tools/integrations/lt/, tools/integrations/sf/):

Format Mapping primitive Dispatched by Field resolution
JSON VTX::JsonMapping<T> specialisations (in arena_mappings.h) VTX::UniversalDeserializer<>::Load<ArenaReplayJson> compile-time tuple of MakeField(name, &T::member)
Protobuf VTX::ProtoBinding<T>::Transfer() (in advance_write.cpp) VTX::GenericProtobufLoader::LoadFrame runtime SchemaRegistry lookup by name
FlatBuffers VTX::FlatBufferBinding<T>::Transfer() (in advance_write.cpp) VTX::GenericFlatBufferLoader::LoadFrame PropertyAddressCache -- O(1) after first lookup

Each data source is wrapped in a VTX::IFrameDataSource implementation -- three tiny classes (ArenaJsonDataSource, ArenaProtoDataSource, ArenaFbsDataSource) with the contract:

class IFrameDataSource {
    virtual bool   Initialize() = 0;
    virtual bool   GetNextFrame(Frame& out_frame, GameTime::GameTimeRegister& out_time) = 0;
    virtual size_t GetExpectedTotalFrames() const = 0;
};

That's the same interface a real ingestion pipeline would implement -- see tools/integrations/rl/rl15/rl15_data_source.h for an example over a streaming protobuf file.

6. post_process_write -- post-processor end-to-end

The minimum end-to-end demo of the writer-side frame post-processor pipeline. Builds synthetic frames, registers a PlayerHealthProcessor (clamp Health to [0, 100], derive IsAlive, count cross-frame stats), records 30 frames, then re-opens the .vtx with the reader to prove the persisted bytes contain the post-processed values.
Links against vtx_writer, vtx_reader
Needs codegen Yes -- arena_generated.h (uses PlayerMutator + ForEachPlayer + ForEachPlayerView)
argv[1] schema JSON path (default: content/writer/arena/arena_schema.json)
argv[2] output .vtx path (default: post_processed.vtx)
argv[3] frame count (default: 30)

Key APIs exercised: VTX::IFramePostProcessor (Init / Process / Clear / PrintInfo), writer->SetPostProcessor, FrameMutationView::GetBucket, VTX::ArenaSchema::ForEachPlayer/ForEachPlayerView (codegen-generated), PlayerMutator::SetHealth / SetIsAlive (codegen-generated).

The processor has zero hardcoded strings, zero PropertyKey<T> members, and no manual entity_type_id gating -- everything comes from the codegen-generated wrappers in arena_generated.h. Full feature reference in POST_PROCESSING.md.

7. websocket_consumer -- live ingestion over WebSocket

Connects as a WebSocket client to ws://host:port/path (or wss://... for TLS), reads one JSON frame per incoming message, and records the stream into a .vtx. Demonstrates WebSocketFrameDataSource<Adapter> and the declarative JsonMapping + UniversalDeserializer pattern -- the JSON->struct step is fully declarative, only the struct->VTX::Frame mapping is spelled out.
Links against vtx_writer
argv[1] URL (default: ws://127.0.0.1:8765/)
argv[2] output .vtx path (default: websocket_output.vtx)
argv[3] schema JSON (default: content/writer/arena/arena_schema.json)
Companion samples/websocket_server.py -- Python WebSocket server pushing synthetic JSON frames (requires pip install websockets)

Key APIs: VTX::WebSocketFrameDataSource, VTX::JsonMapping<WsFrame> + JsonMapping<WsEntity> specialisations, VTX::UniversalDeserializer<>::Load<WsFrame>(JsonAdapter). Two-terminal demo:

:: Terminal 1
python samples\websocket_server.py
:: Terminal 2
vtx_sample_websocket_consumer ws://127.0.0.1:8765/ out.vtx samples\content\writer\arena\arena_schema.json

8. pipe_producer + pipe_consumer -- pipe-based streaming

A pair of CLI tools exercising PipeFrameDataSource. The producer emits length-prefixed JSON frames ([uint32 LE size][JSON], terminated by a zero-size sentinel) to stdout; the consumer reads from stdin, connects to a pipe a producer already created, or creates one as the server.

Producer modes (argv[1]):

  • vtx_sample_pipe_producer N -- bounded: send N frames + sentinel, exit.
  • vtx_sample_pipe_producer 0 -- continuous: stream at ~60 fps until you press Enter in the terminal. A detached stdin watcher thread sets an atomic stop flag; the loop then emits the sentinel and exits cleanly, so the consumer finalises a valid .vtx.

Consumer modes (argv[1]):

  • - -- read from stdin (for shell pipelines producer | consumer).
  • <path> -- connect to a pipe / FIFO a producer already created.
  • serve:<path> -- VTX creates the named pipe / FIFO and waits for a producer to connect. This is the mode for an independent external producer (e.g. a game injector).

Three demo .bat scripts wrap the common scenarios:

Script Demonstrates
pipe_demo.bat anonymous pipe, single launcher: producer N | consumer - out.vtx schema.json
named_pipe_vtx.bat named-pipe demo, VTX side -- runs the consumer with serve:\\.\pipe\vtx, waits for a producer
named_pipe_producer.bat named-pipe demo, producer side -- retry-connects, streams continuously until Enter

The named-pipe demo is the genuine "two fully independent processes" scenario: the two bats run in two separate terminals; their producer / consumer find each other over \\.\pipe\vtx. Mirrors the real use case -- VTX as the long-running recorder, an external producer (game injector, capture daemon) opens the pipe as a client when its session starts.

External producers without the SDK

The [uint32 LE size][payload] framing is self-contained -- a producer in any language can feed VTX without linking the SDK. Minimal C / Win32 example (game-injector shaped):

HANDLE pipe;
for (;;) {
    pipe = CreateFileA("\\\\.\\pipe\\vtx", GENERIC_WRITE,
                       0, NULL, OPEN_EXISTING, 0, NULL);
    if (pipe != INVALID_HANDLE_VALUE) break;
    Sleep(200);   // VTX not up yet, retry
}
// per frame:
uint32_t len = (uint32_t)strlen(json);
WriteFile(pipe, &len, 4, &written, NULL);
WriteFile(pipe, json, len, &written, NULL);
// end of session:
uint32_t sentinel = 0;
WriteFile(pipe, &sentinel, 4, &written, NULL);
CloseHandle(pipe);

The payload format is whatever the consumer-side Adapter parses (JSON in the sample; swap the adapter to ingest Protobuf, FlatBuffers, or a custom binary layout).

Arena schema codegen

The advanced samples depend on generated C++ code for the two arena wire schemas:

Source Tool Namespace Output
samples/schemas/arena_data.proto ${VTX_PROTOC_EXE} (thirdparty/protobuf/bin/protoc.exe on Windows bundled; system protoc on Linux/macOS) arena_pb:: ${build}/samples/arena_generated/arena_data.pb.{h,cc}
samples/schemas/arena_data.fbs $<TARGET_FILE:flatc> (FlatBuffers built from FetchContent source) arena_fb:: ${build}/samples/arena_generated/arena_data_generated.h

Both codegen steps are declared in samples/CMakeLists.txt as add_custom_command rules, so re-editing either schema triggers a rebuild automatically.

arena_generated.h (hand-checked into the repo, under samples/) is a separate autogenerated file. From the VTX property schema JSON it emits, per struct:

  • Constants: ArenaSchema::Player::UniqueID, ArenaSchema::Player::StructName, etc.
  • EntityType enum: ArenaSchema::EntityType::Player = 0, Projectile = 1, ...
  • Read-only views: PlayerView, ProjectileView, MatchStateView -- wrap EntityView, cache PropertyKey<T> in static locals.
  • Mutators: PlayerMutator, ProjectileMutator, MatchStateMutator -- wrap EntityMutator, expose Get* (same as Views) plus Set* for scalars and GetMutable* for arrays / nested structs.
  • Iteration helpers: ForEachPlayer(bucket, accessor, fn) / ForEachPlayerView(bucket, accessor, fn) -- filter a BucketMutator (or const Bucket&) by entity_type_id and invoke fn with the strongly-typed mutator/view.

Produced by:

python scripts/vtx_codegen.py samples/content/writer/arena/arena_schema.json samples/arena_generated.h ArenaSchema

Regenerate it whenever arena_schema.json changes. The same script handles any schema -- the same pattern applies to League, CS, Rocket League, or any other integration that ships its own JSON. The SDK doesn't know about the game; the codegen output knows.

What each sample teaches

You want to... Read
Open a .vtx and print its contents basic_read.cpp
Build PropertyContainers by hand and write them basic_write.cpp
Compare two frames at all basic_diff.cpp
Wait for the reader's first chunk to land (poll / block / callback) ready_api.cpp
Simulate game data and persist it as a raw data source generate_replay.cpp
Implement IFrameDataSource over JSON using JsonMapping<T> advance_write.cpp -- ArenaJsonDataSource + arena_mappings.h
Implement IFrameDataSource over Protobuf using ProtoBinding<T> advance_write.cpp -- ArenaProtoDataSource + ProtoBinding<arena_pb::*> specialisations
Implement IFrameDataSource over FlatBuffers using FlatBufferBinding<T> advance_write.cpp -- ArenaFbsDataSource + FlatBufferBinding<arena_fb::*> specialisations
Hook a writer-side frame post-processor with full lifecycle post_process_write.cpp -- PlayerHealthProcessor
Run the same post-processor across multiple data sources advance_write.cpp -- ArenaConsistencyProcessor registered per pipeline
Use codegen-generated typed mutators (PlayerMutator, ForEachPlayer) post_process_write.cpp + advance_write.cpp
See the same patterns in a production integration tools/integrations/rl/rl15/
Stream frames live from a CLI process into VTX (anonymous pipe) pipe_producer.cpp + pipe_consumer.cpp + pipe_demo.bat
Have an external producer and VTX run as two independent processes over a named pipe named_pipe_vtx.bat + named_pipe_producer.bat + pipe_consumer.cpp (server mode)
Stream frames continuously and stop on a user command pipe_producer.cpp (continuous mode -- producer 0, Enter to stop)
Ingest frames live from a WebSocket server websocket_consumer.cpp + websocket_server.py
Feed VTX from a non-SDK producer (any language, any process) pipe_consumer.cpp in serve: mode -- producer just speaks [uint32 size][payload] framing

Running the samples

Running order matters for the arena pipeline

vtx_sample_generate                       # simulate -> content/writer/arena/*.{json,proto.bin,fbs.bin}
vtx_sample_advance_write                  # read those -> content/reader/arena/arena_from_*_ds.vtx
vtx_sample_read [path]                    # inspect any of them
vtx_sample_diff [path]                    # diff frames 0 and 1

vtx_sample_write is independent -- it produces its own sample_output.vtx with synthetic data.

Troubleshooting

"Could not open: content/reader/arena/arena_from_fbs_ds.vtx" Run vtx_sample_generate and then vtx_sample_advance_write first, from the samples/ directory.

Schema-registry warnings about field counts Regenerate arena_generated.h after editing content/writer/arena/arena_schema.json.

zstd at runtime There is no runtime zstd dependency. The library is linked statically from the FetchContent build, so no DLL / .so sits next to the sample executables.