Skip to content

Add stefgen docs #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tigrannajaryan merged 1 commit into from
Jul 4, 2025
Merged

Add stefgen docs #103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ <h2>Write STEF Records</h2>
<h2>Learn More</h2>
<ul>
<li><a href="https://github.com/splunk/stef/blob/main/stef-spec/specification.md">STEF Specification</a> (detailed format and protocol)</li>
<li><a href="./stefgen.html">Stefgen Code Generator</a> (generate code from STEF schemas)</li>
<li><a href="./benchmarks.html">Benchmarks</a> (performance results)</li>
<li><a href="https://github.com/splunk/stef">GitHub Repository</a></li>
</ul>
Expand Down
143 changes: 143 additions & 0 deletions docs/stefgen.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Stefgen - STEF Code Generator</title>
<link href="https://fonts.googleapis.com/css?family=Roboto:400,700&display=swap" rel="stylesheet">
<link rel="stylesheet" href="./style.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/themes/prism.css">
<link rel="stylesheet" href="./prism-stef.css">
</head>
<body>
<header>
<h1>Stefgen</h1>
<nav>
<a href="./index.html">Home</a>
<a href="https://github.com/splunk/stef">GitHub</a>
<a href="https://github.com/splunk/stef/blob/main/stef-spec/specification.md">Specification</a>
</nav>
</header>
<main>
<p><strong>Stefgen</strong> is a command-line code generator that takes STEF schema files as input and generates type-safe serialization and deserialization code for multiple programming languages. It parses STEF schema definitions and produces optimized code that can efficiently read and write STEF-encoded data.</p>

<h2>Installation</h2>
<p>Build stefgen from source:</p>
<pre><code class="language-bash">cd stefgen
make build</code></pre>
<p>This will create the <code>stefgen</code> binary in the <code>bin/</code> directory.</p>

<h2>Usage</h2>
<pre><code class="language-bash">stefgen [flags] &lt;path-to-schema-file&gt;</code></pre>

<h2>Command Line Arguments</h2>
<table>
<thead>
<tr>
<th>Flag</th>
<th>Type</th>
<th>Required</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>-lang</code></td>
<td>string</td>
<td>Yes</td>
<td>Target language for code generation. Supported values: <code>go</code>, <code>java</code></td>
</tr>
<tr>
<td><code>-outdir</code></td>
<td>string</td>
<td>Yes</td>
<td>Output directory where generated source files will be written</td>
</tr>
<tr>
<td><code>-testoutdir</code></td>
<td>string</td>
<td>No</td>
<td>Output directory for test files. If unspecified, defaults to <code>outdir</code>. Only used with <code>-lang=java</code></td>
</tr>
<tr>
<td><code>&lt;schema-file&gt;</code></td>
<td>path</td>
<td>Yes</td>
<td>Path to the STEF schema file (.stef) to process</td>
</tr>
</tbody>
</table>

<h2>Examples</h2>

<h3>Generate Go Code</h3>
<pre><code class="language-bash"># Generate Go code from a STEF schema
stefgen -lang=go -outdir=./generated schema.stef</code></pre>

<h3>Generate Java Code</h3>
<pre><code class="language-bash"># Generate Java code with separate test directory
stefgen -lang=java -outdir=./src/main/java -testoutdir=./src/test/java schema.stef</code></pre>

<h3>Generate Java Code (Simple)</h3>
<pre><code class="language-bash"># Generate Java code with tests in same directory
stefgen -lang=java -outdir=./java-gen schema.stef</code></pre>

<h2>Supported Languages</h2>
<ul>
<li><strong>Go</strong> - Generates Go structs with serialization methods</li>
<li><strong>Java</strong> - Generates Java classes with serialization support</li>
</ul>

<h2>Generated Code Structure</h2>

<p>stefgen creates:</p>
<ul>
<li>Struct, oneof and multimap definitions matching your STEF schema</li>
<li>Type-safe accessors for all fields</li>
<li>Reader for deserialization</li>
<li>Writer for serialization</li>
<li>Read/write unit tests with randomized inputs, matching the schema</li>
</ul>

<h2>Example Workflow</h2>

<h3>1. Define Your Schema</h3>
<p>Create a STEF schema file (e.g., <code>user.stef</code>):</p>
<pre><code class="language-stef">package userdata

struct User root {
Id uint64
Name string
}</code></pre>

<h3>2. Generate Code</h3>
<pre><code class="language-bash">stefgen -lang=go -outdir=./generated user.stef</code></pre>

<h3>3. Use Generated Code</h3>
<p>The generated Go code can then be imported and used in your application:</p>
<pre><code class="language-go">import "github.com/splunk/stef/go/pkg"
import "your-module/generated/userdata"

// Prepare a memory buffer to write the STEF stream to.
buf := &amp;pkg.MemChunkWriter{}

// Create a Writer for the JSON-like schema
w := userdata.NewWriter(buf, pkg.WriterOptions{})

// Build a record in memory.
writer.Record.Value.SetId(1234)
writer.Record.Value.SetName("Foo Bar")

// Write the record to the stream.
writer.Write()

// Flush the stream to the buffer.
writer.Flush()</code></pre>

<p class="footer-note">For more information about STEF schema syntax and features, see the <a href="https://github.com/splunk/stef/blob/main/stef-spec/specification.md">STEF Specification</a>.</p>
</main>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/prism.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/components/prism-go.min.js"></script>
<script src="./prism-stef.js"></script>
</body>
</html>
2 changes: 1 addition & 1 deletion makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ all: docs-validate
cd benchmarks && make all

.PHONY: build-ci
build-ci:
build-ci: docs-install-deps docs-validate
cd stefgen && make all
cd go/pkg && make all
cd go/grpc && make
Expand Down
Loading