Skip to content

Commit 7caf79c

Browse files
DavidJBiancoclaude
andcommitted
Document Windows-only Phase 1 limitation and plan multi-OS support for Phase 2
Added Phase 1.11: Document Windows-Only Limitation - Validation warning for non-Windows systems - README documentation of Phase 1 constraint - Architecture documentation updates Enhanced Phase 2.2: Additional Log Formats & Multi-OS Support - Added Linux formats: syslog, auditd, bash_history - Existing formats: snort, web - Total 5+ formats for MVP Added Phase 2.10: OS-Aware Activity Generation - OS detection utility (utils/os.py) - OS-specific activity generation branching - Activity templates for Windows/Linux command mapping - OS-aware emitter initialization - Mixed environment support - Comprehensive testing plan This addresses the root cause: system.os field is currently metadata-only in Phase 1 but will be fully functional by MVP through Phase 2.10. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 029ba66 commit 7caf79c

1 file changed

Lines changed: 73 additions & 6 deletions

File tree

TODO.md

Lines changed: 73 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,24 @@
136136
- [ ] Manual testing: Generate logs and verify format compliance
137137
- [ ] Update README with Phase 1 status and basic usage
138138

139-
**Phase 1 Milestone:** Can generate small, consistent datasets across 2 log formats with schema validation.
139+
### 1.11 Document Windows-Only Limitation
140+
141+
- [ ] Add validation warning when non-Windows OS detected in scenario
142+
- [ ] Modify `validation/schema.py` to check system.os field
143+
- [ ] Warn (not error) if OS is not Windows-based
144+
- [ ] Include suggestion to use Windows or wait for Phase 2 multi-OS support
145+
- [ ] Update README.md to document Phase 1 Windows-only support
146+
- [ ] Add "Current Limitations" section
147+
- [ ] Clearly state that Phase 1 generates Windows Event Logs for all systems
148+
- [ ] Note that system.os field is stored but not used in Phase 1
149+
- [ ] Add comment in retail-store-ftp-attack.yaml explaining behavior
150+
- [ ] Note that MUSIC-SRV-01 (Ubuntu) will generate Windows logs in Phase 1
151+
- [ ] Keep as test case for Phase 2 multi-OS implementation
152+
- [ ] Document in architecture docs that OS field exists but isn't used in Phase 1
153+
- [ ] Update PRD.md or create architecture.md if needed
154+
- [ ] Explain that OS detection will be implemented in Phase 2.10
155+
156+
**Phase 1 Milestone:** Can generate small, consistent datasets across 2 log formats with schema validation. **Known limitation:** Windows-only log generation (documented and validated).
140157

141158
---
142159

@@ -153,16 +170,31 @@
153170
- [ ] Test: Parallel emitter execution with state consistency
154171
- [ ] Test: No data races or deadlocks
155172

156-
### 2.2 Additional Log Formats (3 more)
173+
### 2.2 Additional Log Formats & Multi-OS Support
174+
175+
**Goal:** Add Linux log formats and additional formats for 5+ total MVP formats
157176

158177
- [ ] `formats/definitions/syslog.yaml` - Linux syslog format (RFC 5424)
178+
- [ ] Support process execution logs (daemon.log, syslog)
179+
- [ ] Support authentication logs (auth.log)
180+
- [ ] Include facility/severity mapping
181+
- [ ] `formats/definitions/auditd.yaml` - Linux audit logs (similar to Windows Security logs)
182+
- [ ] Support EXECVE records (process execution)
183+
- [ ] Support USER_AUTH records (authentication)
184+
- [ ] Support SYSCALL records (system calls)
185+
- [ ] `formats/definitions/bash_history.yaml` - Bash command history format
186+
- [ ] Support timestamped command history
187+
- [ ] Per-user history files
159188
- [ ] `formats/definitions/snort.yaml` - Snort/Suricata alert format (fast alert)
160189
- [ ] `formats/definitions/web.yaml` - W3C web log format
161190
- [ ] `generation/emitters/syslog.py` - Syslog emitter
191+
- [ ] `generation/emitters/auditd.py` - Auditd emitter
192+
- [ ] `generation/emitters/bash_history.py` - Bash history emitter
162193
- [ ] `generation/emitters/snort.py` - Snort emitter
163194
- [ ] `generation/emitters/web.py` - Web log emitter
164-
- [ ] Test: All 5 formats generate valid output
165-
- [ ] Test: Cross-format consistency across all 5 formats
195+
- [ ] Test: All formats generate valid output
196+
- [ ] Test: Cross-format consistency across all formats
197+
- [ ] Test: Linux formats parse correctly with standard Linux log viewers
166198

167199
### 2.3 Progress Reporting
168200

@@ -236,13 +268,48 @@
236268

237269
### 2.9 Phase 2 Testing & Scenarios
238270

239-
- [ ] Integration test: 8-hour scenario with all 5 formats
271+
- [ ] Integration test: 8-hour scenario with all 5+ formats
240272
- [ ] Create test fixture: `fixtures/scenarios/medium-dataset.yaml` (100 users, 8 hours)
241273
- [ ] Performance benchmarks (time, memory) for medium datasets
242274
- [ ] Unit test coverage: maintain 95%+ overall
243275
- [ ] Update README with Phase 2 capabilities
244276

245-
**Phase 2 Milestone:** Can generate medium-scale datasets (100K+ events) across all 5 MVP formats in parallel with good performance.
277+
### 2.10 OS-Aware Activity Generation
278+
279+
**Goal:** Enable generation of OS-specific logs based on system.os field (addresses Phase 1 limitation)
280+
281+
- [ ] Create OS detection utility in `utils/os.py`
282+
- [ ] `OSType` enum (Windows, Linux, MacOS, Unknown)
283+
- [ ] `detect_os_type(system: System) -> OSType` function
284+
- [ ] Pattern matching for OS strings (flexible matching: "Windows 10", "Ubuntu", "macOS", etc.)
285+
- [ ] Handle variations: "Windows Server 2019", "Ubuntu 22.04 LTS", "Red Hat", etc.
286+
- [ ] Refactor `generation/activity.py` for OS branching
287+
- [ ] `generate_process()` - detect OS and route to OS-specific process generator
288+
- [ ] `_generate_windows_process()` - existing Windows Event 4688 logic (refactored)
289+
- [ ] `_generate_linux_process()` - emit syslog/auditd entries
290+
- [ ] Update process templates to include OS-specific paths and commands
291+
- [ ] Maintain existing ActivityGenerator API (no breaking changes)
292+
- [ ] Create `generation/activity_templates.py` - OS-specific command mappings
293+
- [ ] Map storyline activities to OS-specific implementations
294+
- [ ] Windows: paths (C:\Windows\...), event IDs (4624, 4688, etc.)
295+
- [ ] Linux: paths (/usr/bin/..., /bin/...), syslog facilities, auditd record types
296+
- [ ] Common commands: whoami, ps/tasklist, netstat, etc.
297+
- [ ] Support for LOLBins on both platforms
298+
- [ ] Update `generation/engine.py` for OS-aware emitter initialization
299+
- [ ] Check system.os when creating emitters per system
300+
- [ ] Windows systems → Windows Event emitters
301+
- [ ] Linux systems → syslog/auditd/bash_history emitters
302+
- [ ] Mixed environments → coordinate across both
303+
- [ ] Zeek emitter remains OS-agnostic (network visibility)
304+
- [ ] Test: Windows system generates Windows Event logs
305+
- [ ] Test: Linux system generates syslog/auditd logs
306+
- [ ] Test: Mixed environment (Windows + Linux) generates appropriate logs per system
307+
- [ ] Test: Storyline activities correctly mapped to OS-specific commands
308+
- [ ] Test: Cross-log consistency in mixed environments (Zeek sees connections between Windows and Linux systems)
309+
- [ ] Update retail-store-ftp-attack.yaml to properly generate Linux logs for MUSIC-SRV-01
310+
- [ ] Remove Phase 1 validation warning for non-Windows systems
311+
312+
**Phase 2 Milestone:** Can generate medium-scale datasets (100K+ events) across all 5+ MVP formats in parallel with good performance. **Multi-OS support:** Windows and Linux systems generate appropriate OS-specific logs.
246313

247314
---
248315

0 commit comments

Comments
 (0)