Skip to content

Commit cc020d3

Browse files
committed
Update Serialization guide and other pages for v0.11
1 parent a10cbce commit cc020d3

5 files changed

Lines changed: 36 additions & 104 deletions

File tree

ksy_style_guide.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
= KSY Style Guide
22
Kaitai Project
3-
v0.10
3+
v0.11
44
:toc: left
55
:numbered:
66

lang_php.adoc

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
= Kaitai Struct: PHP notes
22

3-
The Kaitai Struct https://github.com/kaitai-io/kaitai_struct_php_runtime[runtime for PHP] requires PHP >= 7.0 and can be used in two different ways:
3+
The Kaitai Struct https://github.com/kaitai-io/kaitai_struct_php_runtime[runtime library for PHP] requires PHP >= 7.0 and can be used in two different ways:
44

55
1) As part of Kaitai Struct (recommended approach)
66

@@ -17,22 +17,22 @@ cd kaitai-test
1717
#######################################################################################
1818
# Download Kaitai Struct Compiler, link can be found at https://kaitai.io/#download page
1919

20-
curl -OL https://bintray.com/artifact/download/kaitai-io/universal/0.10/kaitai-struct-compiler-0.10.zip
20+
curl -fLO https://github.com/kaitai-io/kaitai_struct_compiler/releases/download/0.11/kaitai-struct-compiler-0.11.zip
2121

22-
unzip kaitai-struct-compiler-0.10.zip
22+
unzip kaitai-struct-compiler-0.11.zip
2323

24-
cd kaitai-struct-compiler-0.10
24+
cd kaitai-struct-compiler-0.11
2525

2626
###############
2727
# Check version
2828

2929
bin/kaitai-struct-compiler --version
30-
# Shows kaitai-struct-compiler 0.10
30+
# Shows kaitai-struct-compiler 0.11
3131

3232
################################################
3333
# Download and compile ELF file format - elf.ksy
3434

35-
curl -OL https://raw.githubusercontent.com/kaitai-io/kaitai_struct_formats/master/executable/elf.ksy
35+
curl -fLO https://raw.githubusercontent.com/kaitai-io/kaitai_struct_formats/master/executable/elf.ksy
3636

3737
bin/kaitai-struct-compiler --target php --outdir my-elf-parser --php-namespace 'My\Parser' elf.ksy
3838

lang_python.adoc

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
= Kaitai Struct: Python notes
22
Kaitai Project
3-
v0.10
3+
v0.11
44
:toc: left
55

66
[[overview]]
@@ -10,10 +10,13 @@ v0.10
1010
=== Supported Python versions
1111

1212
Kaitai Struct supports Python 3.4 and later as well as Python 2.7.
13-
It is strongly recommended to use Python 3.8 or newer,
14-
because Python 3.7 and 2.7 have reached end-of-life
13+
It is strongly recommended to use Python 3.9 or newer,
14+
because Python 3.8 and 2.7 have reached end-of-life
1515
and no longer receive official support from the Python developers.
1616

17+
Please note that Kaitai Struct 0.11 is the last version to support Python 2.7.
18+
Versions 0.12 and above will only work on Python 3.
19+
1720
[[ksc]]
1821
=== Kaitai Struct compiler invocation
1922

serialization.adoc

Lines changed: 20 additions & 91 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@ Kaitai Project
33
:toc: left
44
:tabs-sync-option:
55

6-
NOTE: Serialization for Java and Python is made thanks to financial support https://nlnet.nl/project/Kaitai-Serialization[from the NLnet Foundation].
6+
NOTE: Serialization for Java and Python was made thanks to financial support https://nlnet.nl/project/Kaitai-Serialization/[from the NLnet Foundation].
7+
8+
IMPORTANT: Feature available since v0.11.
79

810
For a long time, you could only use Kaitai Struct for parsing, not serialization (writing data to file). However, due to high user interest in this feature, we've added serialization support to Kaitai Struct.
911

@@ -19,40 +21,6 @@ While parsing allows you extract data from existing files or byte streams based
1921

2022
== Getting started
2123

22-
Once you have the .ksy specification of the format you want to serialize, you need a version of `kaitai-struct-compiler` that supports serialization. The latest 0.10 compiler doesn't have it yet; you need to build the compiler from source at the moment.
23-
24-
=== Building the compiler from source
25-
26-
Don't worry, it should be straightforward:
27-
28-
1. Install `sbt` by following the steps at https://www.scala-sbt.org/1.x/docs/Setup.html.
29-
30-
2. Clone the https://github.com/kaitai-io/kaitai_struct_compiler repository, checkout the https://github.com/kaitai-io/kaitai_struct_compiler/tree/serialization[*serialization*] branch.
31-
+
32-
[source,shell]
33-
----
34-
git clone -b serialization https://github.com/kaitai-io/kaitai_struct_compiler.git
35-
cd kaitai_struct_compiler
36-
----
37-
38-
3. Build the compiler using `sbt` that you installed earlier:
39-
+
40-
[source,shell]
41-
----
42-
sbt --error compilerJVM/stage
43-
----
44-
+
45-
If no error is printed, there should be a compiler build in `jvm/target/universal/stage/bin/kaitai-struct-compiler`. If you run `jvm/target/universal/stage/bin/kaitai-struct-compiler --help`, you should see the `--read-write` option in the usage text:
46-
+
47-
[source,highlight=5]
48-
----
49-
Usage: kaitai-struct-compiler [options] <file>...
50-
51-
<file>... source files (.ksy)
52-
-t, --target <language> target languages (graphviz, csharp, rust, all, perl, java, go, cpp_stl, php, lua, python, nim, html, ruby, construct, javascript)
53-
-w, --read-write generate read-write support in classes (implies `--no-auto-read --zero-copy-substream false`, Java and Python only, default: read-only)
54-
----
55-
5624
=== Compiling a .ksy specification in read-write mode
5725

5826
[tabs]
@@ -64,7 +32,7 @@ You can compile a .ksy spec to Java classes with serialization support like this
6432
6533
[source,shell]
6634
----
67-
jvm/target/universal/stage/bin/kaitai-struct-compiler --read-write --no-auto-read -t java <ksy-file>
35+
kaitai-struct-compiler --read-write --no-auto-read -t java <ksy-file>
6836
----
6937
--
7038
@@ -75,7 +43,7 @@ You can compile a .ksy spec to Python classes with serialization support like th
7543
7644
[source,shell]
7745
----
78-
jvm/target/universal/stage/bin/kaitai-struct-compiler --read-write --no-auto-read -t python <ksy-file>
46+
kaitai-struct-compiler --read-write --no-auto-read -t python <ksy-file>
7947
----
8048
--
8149
======
@@ -155,82 +123,41 @@ with KaitaiStream(open("path/to/some.gif", 'rb')) as _io:
155123
----
156124
======
157125

158-
=== Installing the runtime library with serialization support
126+
=== Installing the runtime library
159127

160-
As with the compiler, the latest released 0.10 KS runtime libraries don't have serialization capabilities yet.
128+
You must add the Kaitai Struct runtime library for the selected target language to your project, as all generated modules depend on it.
161129

162130
[tabs]
163131
======
164132
Java::
165133
+
166134
--
167-
In Java, you need to checkout the https://github.com/kaitai-io/kaitai_struct_java_runtime repo:
135+
The Java runtime library is published https://central.sonatype.com/artifact/io.kaitai/kaitai-struct-runtime[in the Maven Central Repository]. The source code is available https://github.com/kaitai-io/kaitai_struct_java_runtime[on GitHub].
168136
169-
[source,shell]
170-
----
171-
git clone https://github.com/kaitai-io/kaitai_struct_java_runtime.git
172-
cd kaitai_struct_java_runtime
173-
----
174-
175-
The runtime library is a dependency of all Java code generated by `kaitai-struct-compiler`, so you have to build it and make it available to your generated Java "format library" at compile time. If you use https://maven.apache.org/[Maven], run this command in the `kaitai_struct_java_runtime` directory to build it and install it to your local Maven repository:
176-
177-
[source,shell]
178-
----
179-
mvn install
180-
----
181-
182-
[NOTE]
183-
=====
184-
If the `gpg` command isn't available on your system, `mvn install` will fail because of `maven-gpg-plugin` used to sign artifacts when publishing. In that case, comment this plugin in `pom.xml` like this:
185-
186-
[source,xml,highlight="2,9"]
187-
----
188-
</plugin>
189-
<!-- <plugin>
190-
<groupId>org.apache.maven.plugins</groupId>
191-
<artifactId>maven-gpg-plugin</artifactId>
192-
<version>1.5</version>
193-
<executions>
194-
...
195-
</executions>
196-
</plugin> -->
197-
</plugins>
198-
</build>
199-
----
200-
=====
137+
After you install it, you can import the I/O stream classes defined in the runtime library like this (generated modules do it too):
201138
202-
Now you can include the serialization-capable Java runtime library in your project like this:
203-
204-
[source,xml]
139+
[source,java]
205140
----
206-
<dependency>
207-
<groupId>io.kaitai</groupId>
208-
<artifactId>kaitai-struct-runtime</artifactId>
209-
<version>0.11-SNAPSHOT</version>
210-
</dependency>
141+
import io.kaitai.struct.ByteBufferKaitaiStream;
142+
import io.kaitai.struct.KaitaiStream;
211143
----
212144
213-
But note that the `0.11-SNAPSHOT` version only exists in your local Maven repository (`~/.m2`) after you ran `mvn install` in the Java runtime library folder.
145+
For brevity, these imports will be omitted from the code snippets later in this guide, but they're often needed.
214146
--
215147
216148
Python::
217149
+
218150
--
219-
In Python, you need to install the runtime library from the https://github.com/kaitai-io/kaitai_struct_python_runtime repo. You can do it with https://pypi.org/project/pip/[pip] (package installer for Python); you also need https://git-scm.com/[Git] while running the command because the installation involves cloning the source code from GitHub:
151+
The Python runtime library is published https://pypi.org/project/kaitaistruct/[on PyPI]. The source code is available https://github.com/kaitai-io/kaitai_struct_python_runtime[on GitHub].
220152
221-
[source,shell]
222-
----
223-
python -m pip install -U --pre git+https://github.com/kaitai-io/kaitai_struct_python_runtime.git
224-
----
225-
226-
After that, you can import the serialization-capable `KaitaiStream` class defined in the runtime library like this (generated modules do it too):
153+
After you install it, you can import the `KaitaiStream` class defined in the runtime library like this (generated modules do it too):
227154
228155
[source,python]
229156
----
230157
from kaitaistruct import KaitaiStream
231158
----
232159
233-
For brevity, this import will be omitted from code snippets later in this guide, but it's often needed.
160+
For brevity, this import will be omitted from the code snippets later in this guide, but it's often needed.
234161
--
235162
======
236163

@@ -725,7 +652,7 @@ The Java's `+_invalidate{Inst}+` / Python's `+_invalidate_{inst}+` method invali
725652

726653
=== Parse instances
727654

728-
They have setters and their own `+_check{Inst}+` (`+_check_{inst}+`) method which you should call. Additionally, you can also use a special boolean `set{Inst}_ToWrite` setter (in Python you'd assign a boolean to a property `+{inst}__to_write+`), allowing you to disable writing of a specific instance (as `r.set{Inst}_ToWrite(false)` in Java, or `+r.{inst}__to_write = False+` in Python) in a particular KS object. This may be useful for C-style `union` members (several overlapping fields with different types, but only one applies in any object), lookaheads or other positional instances you don't want to write.
655+
They have setters. Additionally, you can also use a special boolean `set{Inst}_ToWrite` setter (in Python you'd assign a boolean to a property `+{inst}__to_write+`), allowing you to disable writing of a specific instance (as `r.set{Inst}_ToWrite(false)` in Java, or `+r.{inst}__to_write = False+` in Python) in a particular KS object. This may be useful for C-style `union` members (several overlapping fields with different types, but only one applies in any object), lookaheads or other positional instances you don't want to write.
729656

730657
=== Parameters
731658

@@ -745,7 +672,9 @@ Current serialization support relies on fixed-length streams, meaning that once
745672

746673
=== Enums
747674

748-
Enum values not present in the enum definition are not supported in Java or Python right now. An attempt to write them causes `NullPointerException` in Java, `AttributeError` in Python.
675+
Currently, generated enum classes in Java can only represent known values defined in the enum. When converting an integer to an enum, integers that don't correspond to any defined enum member are mapped to `null`, which cannot be serialized (a `NullPointerException` will be thrown).
676+
677+
In Python, this limitation doesn't exist: all enum values (both defined and undefined) can be serialized.
749678

750679
=== Bit-sized integers
751680

user_guide.adoc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
= Kaitai Struct User Guide
22
Kaitai Project
3-
v0.10
3+
v0.11
44
:toc: left
55
:numbered:
66

@@ -43,9 +43,9 @@ A list of Web IDE features is available on the https://github.com/kaitai-io/kait
4343
Note that there are two different versions of the Web IDE:
4444

4545
1. https://ide.kaitai.io/ — *stable* version, has the stable Kaitai Struct
46-
compiler (currently 0.10, released 2022-07-08)
46+
compiler (currently 0.11, released 2025-09-07)
4747
2. https://ide.kaitai.io/devel/ — unstable development version, has the *latest*
48-
compiler (the most recent 0.11-SNAPSHOT)
48+
compiler (the most recent 0.12-SNAPSHOT)
4949

5050
If you want to use the latest features, use the https://ide.kaitai.io/devel/[*devel* Web IDE].
5151

0 commit comments

Comments
 (0)