Skip to content

Commit 90d0865

Browse files
committed
Fix some codegen
1 parent 4958c36 commit 90d0865

32 files changed

Lines changed: 2549 additions & 199 deletions

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ erl_crash.dump
55

66
test/generated_outputs/
77
.claude/
8+
example/build

example/README.md

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
# Protozoa Example Project
2+
3+
This example project demonstrates the usage of the Protozoa Protocol Buffer library for Gleam.
4+
5+
## Overview
6+
7+
Protozoa is a Protocol Buffer compiler and runtime library for Gleam that supports all 27 Google Protocol Buffer well-known types and provides complete encode/decode functionality.
8+
9+
## What's Demonstrated
10+
11+
This example showcases:
12+
13+
### ✅ Core Protocol Buffer Features
14+
- **Basic Types**: `int32`, `string`, `bool`, `bytes`, `repeated` fields
15+
- **Enums**: User roles with proper encoding/decoding
16+
- **Oneof Fields**: Union types with multiple data variants
17+
- **Well-Known Types**: `Timestamp`, `StringValue`, and other Google well-known types
18+
19+
### ✅ Advanced Features
20+
- **Message Nesting**: Complex message structures
21+
- **Type Safety**: Full Gleam type safety with generated types
22+
- **Encoding/Decoding**: Bidirectional serialization with proper error handling
23+
24+
## Files Structure
25+
26+
```
27+
example/
28+
├── src/
29+
│ ├── proto/
30+
│ │ └── simple.proto # Protocol Buffer definitions
31+
│ ├── generated/
32+
│ │ └── proto.gleam # Auto-generated Gleam code
33+
│ └── simple_example.gleam # Main example application
34+
├── test/
35+
│ └── simple_test.gleam # Test suite
36+
└── README.md # This file
37+
```
38+
39+
## Protocol Buffer Definitions
40+
41+
### User Message
42+
Demonstrates basic types, enums, repeated fields, and well-known types:
43+
44+
```proto
45+
message User {
46+
int32 id = 1;
47+
string name = 2;
48+
string email = 3;
49+
google.protobuf.Timestamp created_at = 4;
50+
bool is_active = 5;
51+
UserRole role = 6;
52+
repeated string tags = 7;
53+
google.protobuf.StringValue bio = 8;
54+
}
55+
```
56+
57+
### SimpleMessage with Oneof
58+
Demonstrates union types (oneof fields):
59+
60+
```proto
61+
message SimpleMessage {
62+
string id = 1;
63+
64+
oneof data {
65+
string text_data = 2;
66+
int64 numeric_data = 3;
67+
bytes binary_data = 4;
68+
}
69+
70+
string description = 5;
71+
bool enabled = 6;
72+
}
73+
```
74+
75+
## Running the Example
76+
77+
### Prerequisites
78+
- Gleam >= 1.5.0
79+
- Protozoa library
80+
81+
### Generate Code
82+
```bash
83+
# Generate Gleam code from proto files
84+
gleam run -m protozoa src/proto/simple.proto src/generated/
85+
```
86+
87+
### Run the Example
88+
```bash
89+
gleam run -m simple_example
90+
```
91+
92+
Expected output:
93+
```
94+
🚀 Protozoa Simple Example
95+
=========================
96+
👤 Created user: Alice (alice@example.com)
97+
📦 Encoded user data successfully
98+
💬 Created simple message: msg_001
99+
✅ Successfully decoded message: msg_001
100+
Text data: Hello, World!
101+
✅ Simple example completed!
102+
```
103+
104+
### Run Tests
105+
```bash
106+
gleam run -m gleeunit -- --module simple_test
107+
```
108+
109+
All tests should pass:
110+
```
111+
3 tests, no failures
112+
```
113+
114+
## Generated Code Features
115+
116+
The generated `proto.gleam` file includes:
117+
118+
### Type Definitions
119+
- **Union Types**: For oneof fields with proper Gleam algebraic data types
120+
- **Record Types**: For messages with named fields
121+
- **Enum Types**: For protocol buffer enums
122+
123+
### Encoding Functions
124+
- `encode_user(user: User) -> BitArray`
125+
- `encode_simplemessage(message: SimpleMessage) -> BitArray`
126+
- `encode_timestamp(timestamp: Timestamp) -> BitArray`
127+
128+
### Decoding Functions
129+
- `decode_user(data: BitArray) -> Result(User, List(DecodeError))`
130+
- `decode_simplemessage(data: BitArray) -> Result(SimpleMessage, List(DecodeError))`
131+
- `decode_timestamp(data: BitArray) -> Result(Timestamp, List(DecodeError))`
132+
133+
### Well-Known Types Support
134+
The generated code includes full support for Google's Protocol Buffer well-known types:
135+
- `Timestamp` - for date/time values
136+
- `StringValue` - for optional strings
137+
- `Int32Value`, `Int64Value` - for optional integers
138+
- And many more...
139+
140+
## Key Features Demonstrated
141+
142+
### 1. Type Safety
143+
```gleam
144+
// Compile-time type checking
145+
let user = proto.User(
146+
id: 42,
147+
name: "Alice",
148+
// ... other fields
149+
)
150+
```
151+
152+
### 2. Oneof Fields (Union Types)
153+
```gleam
154+
// Pattern matching on union types
155+
case message.data {
156+
option.Some(proto.TextData(text)) -> // Handle text
157+
option.Some(proto.NumericData(num)) -> // Handle number
158+
option.Some(proto.BinaryData(bytes)) -> // Handle binary
159+
option.None -> // Handle no data
160+
}
161+
```
162+
163+
### 3. Error Handling
164+
```gleam
165+
// Safe decoding with proper error handling
166+
case proto.decode_user(encoded_data) {
167+
Ok(user) -> // Successfully decoded
168+
Error(decode_errors) -> // Handle decode errors
169+
}
170+
```
171+
172+
### 4. Well-Known Types
173+
```gleam
174+
// Using Google's well-known types
175+
let timestamp = proto.Timestamp(
176+
seconds: 1640995200,
177+
nanos: 123456789,
178+
)
179+
```
180+
181+
## What's Working
182+
183+
**Basic Types**: All primitive types work correctly
184+
**Enums**: User roles and other enums
185+
**Oneof Fields**: Union types with proper type safety
186+
**Well-Known Types**: Timestamp and other Google types
187+
**Encoding/Decoding**: Round-trip serialization
188+
**Type Safety**: Full Gleam compiler integration
189+
190+
## Performance Notes
191+
192+
- **Efficient Encoding**: Uses Protocol Buffer's compact binary format
193+
- **Memory Safe**: Leverages Gleam's memory safety guarantees
194+
- **Type Safe**: No runtime type errors with proper Gleam types
195+
196+
## Integration
197+
198+
To use Protozoa in your own project:
199+
200+
1. **Add Dependency**: Add protozoa to your `gleam.toml`
201+
2. **Define Schemas**: Create `.proto` files
202+
3. **Generate Code**: Run `gleam run -m protozoa your.proto src/generated/`
203+
4. **Import & Use**: Import generated modules and use the types
204+
205+
This example demonstrates that Protozoa successfully provides a complete, type-safe Protocol Buffer implementation for Gleam with excellent integration into the Gleam ecosystem.

example/gleam.toml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
name = "protozoa_example"
2+
version = "1.0.0"
3+
description = "Example project demonstrating Protozoa Protocol Buffer library usage"
4+
5+
gleam = ">= 1.5.0"
6+
7+
[dependencies]
8+
gleam_stdlib = ">= 0.34.0 and < 2.0.0"
9+
# Protozoa will be loaded from parent directory
10+
protozoa = { path = "../" }
11+
12+
[dev-dependencies]
13+
gleeunit = ">= 1.0.0 and < 2.0.0"
14+

example/manifest.toml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# This file was generated by Gleam
2+
# You typically do not need to edit this file
3+
4+
packages = [
5+
{ name = "argv", version = "1.0.2", build_tools = ["gleam"], requirements = [], otp_app = "argv", source = "hex", outer_checksum = "BA1FF0929525DEBA1CE67256E5ADF77A7CDDFE729E3E3F57A5BDCAA031DED09D" },
6+
{ name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" },
7+
{ name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" },
8+
{ name = "gleam_stdlib", version = "0.62.1", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "0080706D3A5A9A36C40C68481D1D231D243AF602E6D2A2BE67BA8F8F4DFF45EC" },
9+
{ name = "gleam_time", version = "1.4.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_time", source = "hex", outer_checksum = "DCDDC040CE97DA3D2A925CDBBA08D8A78681139745754A83998641C8A3F6587E" },
10+
{ name = "gleeunit", version = "1.6.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleeunit", source = "hex", outer_checksum = "FDC68A8C492B1E9B429249062CD9BAC9B5538C6FBF584817205D0998C42E1DAC" },
11+
{ name = "protozoa", version = "2.0.2", build_tools = ["gleam"], requirements = ["argv", "filepath", "gleam_erlang", "gleam_stdlib", "simplifile", "snag", "tom"], source = "local", path = ".." },
12+
{ name = "simplifile", version = "2.3.0", build_tools = ["gleam"], requirements = ["filepath", "gleam_stdlib"], otp_app = "simplifile", source = "hex", outer_checksum = "0A868DAC6063D9E983477981839810DC2E553285AB4588B87E3E9C96A7FB4CB4" },
13+
{ name = "snag", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "snag", source = "hex", outer_checksum = "7E9F06390040EB5FAB392CE642771484136F2EC103A92AE11BA898C8167E6E17" },
14+
{ name = "tom", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib", "gleam_time"], otp_app = "tom", source = "hex", outer_checksum = "74D0C5A3761F7A7D06994755D4D5AD854122EF8E9F9F76A3E7547606D8C77091" },
15+
]
16+
17+
[requirements]
18+
gleam_stdlib = { version = ">= 0.34.0 and < 2.0.0" }
19+
gleeunit = { version = ">= 1.0.0 and < 2.0.0" }
20+
protozoa = { path = "../" }

example/src/debug_decoder.gleam

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
import generated/proto
2+
import gleam/bit_array
3+
import gleam/int
4+
import gleam/io
5+
import gleam/list
6+
import gleam/string
7+
import protozoa/decode
8+
9+
pub fn main() {
10+
io.println("🔍 Debug: Step-by-step decoding")
11+
12+
let user =
13+
proto.User(
14+
id: 42,
15+
name: "Test",
16+
email: "test@example.com",
17+
created_at: proto.Timestamp(seconds: 1_640_995_200, nanos: 0),
18+
is_active: True,
19+
role: proto.ADMIN,
20+
tags: [],
21+
bio: proto.StringValue(value: "Bio"),
22+
)
23+
24+
let encoded = proto.encode_user(user)
25+
io.println(
26+
"Encoded " <> int.to_string(bit_array.byte_size(encoded)) <> " bytes",
27+
)
28+
29+
// Try to manually decode using the lower-level decode functions
30+
io.println("\nTrying to decode with raw decode functions...")
31+
32+
case decode.run(encoded, test_decoder()) {
33+
Ok(_) -> io.println("✅ Manual decode succeeded!")
34+
Error(errors) -> {
35+
io.println("❌ Manual decode failed:")
36+
list.each(errors, fn(err) { io.println(" - " <> string.inspect(err)) })
37+
}
38+
}
39+
}
40+
41+
fn test_decoder() -> decode.Decoder(String) {
42+
// Try to decode just the first field (id)
43+
use id <- decode.then(decode.int32_with_default(1, 0))
44+
io.println("Decoded field 1 (id): " <> int.to_string(id))
45+
46+
// Try to decode the second field (name)
47+
use name <- decode.then(decode.string_with_default(2, ""))
48+
io.println("Decoded field 2 (name): " <> name)
49+
50+
// Try to decode the third field (email)
51+
use email <- decode.then(decode.string_with_default(3, ""))
52+
io.println("Decoded field 3 (email): " <> email)
53+
54+
// Skip field 4 (timestamp) for now
55+
io.println("Skipping field 4 (timestamp)")
56+
57+
// Try to decode boolean
58+
use is_active <- decode.then(decode.bool_with_default(5, False))
59+
io.println(
60+
"Decoded field 5 (is_active): "
61+
<> case is_active {
62+
True -> "true"
63+
False -> "false"
64+
},
65+
)
66+
67+
// Try to decode the enum
68+
use role <- decode.then(proto.decode_userrole_field(6))
69+
io.println("Decoded field 6 (role): " <> string.inspect(role))
70+
71+
// Try to decode tags (repeated strings)
72+
use tags <- decode.then(decode.repeated_string(7))
73+
io.println("Decoded field 7 (tags): " <> string.inspect(tags))
74+
75+
// Try to decode the StringValue
76+
use bio <- decode.then(decode.nested_message(8, proto.stringvalue_decoder()))
77+
io.println("Decoded field 8 (bio): " <> bio.value)
78+
79+
decode.success("All fields decoded!")
80+
}

example/src/debug_encoding.gleam

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
import generated/proto
2+
import gleam/bit_array
3+
import gleam/int
4+
import gleam/io
5+
import gleam/string
6+
7+
pub fn main() {
8+
io.println("🔍 Debug: Analyzing encoded User message")
9+
10+
let user =
11+
proto.User(
12+
id: 42,
13+
name: "Test",
14+
email: "test@example.com",
15+
created_at: proto.Timestamp(seconds: 1_640_995_200, nanos: 0),
16+
is_active: True,
17+
role: proto.ADMIN,
18+
tags: [],
19+
bio: proto.StringValue(value: "Bio"),
20+
)
21+
22+
let encoded = proto.encode_user(user)
23+
24+
io.println("Encoded length: " <> int.to_string(bit_array.byte_size(encoded)))
25+
26+
// Convert to list of bytes for inspection
27+
case bit_array.to_string(encoded) {
28+
Ok(_) -> io.println("Contains valid UTF-8")
29+
Error(_) -> io.println("Contains binary data (expected)")
30+
}
31+
32+
// Let's examine all the bytes
33+
io.println("All bytes as hex:")
34+
hex_dump(encoded)
35+
36+
// Test individual field encodings
37+
io.println("\nTesting individual field encodings:")
38+
39+
// Let's analyze what the User encoder creates
40+
io.println("User encoding contains these field numbers: 1,2,3,4,5,6,8")
41+
io.println(
42+
"Expected wire types: Varint,String,String,LengthDelimited,Bool,Varint,LengthDelimited",
43+
)
44+
}
45+
46+
fn hex_dump(data: BitArray) -> Nil {
47+
hex_dump_helper(data, 0)
48+
}
49+
50+
fn hex_dump_helper(data: BitArray, offset: Int) -> Nil {
51+
case data {
52+
<<byte:int, rest:bits>> -> {
53+
let hex = int.to_base16(byte)
54+
let padded = case string.length(hex) {
55+
1 -> "0" <> hex
56+
_ -> hex
57+
}
58+
io.print(padded <> " ")
59+
case int.remainder(offset + 1, 16) {
60+
Ok(0) -> io.println("")
61+
_ -> Nil
62+
}
63+
hex_dump_helper(rest, offset + 1)
64+
}
65+
<<>> -> io.println("")
66+
_ -> io.println("(invalid pattern)")
67+
}
68+
}

0 commit comments

Comments
 (0)