|
136 | 136 | - [ ] Manual testing: Generate logs and verify format compliance |
137 | 137 | - [ ] Update README with Phase 1 status and basic usage |
138 | 138 |
|
139 | | -**Phase 1 Milestone:** Can generate small, consistent datasets across 2 log formats with schema validation. |
| 139 | +### 1.11 Document Windows-Only Limitation |
| 140 | + |
| 141 | +- [ ] Add validation warning when non-Windows OS detected in scenario |
| 142 | + - [ ] Modify `validation/schema.py` to check system.os field |
| 143 | + - [ ] Warn (not error) if OS is not Windows-based |
| 144 | + - [ ] Include suggestion to use Windows or wait for Phase 2 multi-OS support |
| 145 | +- [ ] Update README.md to document Phase 1 Windows-only support |
| 146 | + - [ ] Add "Current Limitations" section |
| 147 | + - [ ] Clearly state that Phase 1 generates Windows Event Logs for all systems |
| 148 | + - [ ] Note that system.os field is stored but not used in Phase 1 |
| 149 | +- [ ] Add comment in retail-store-ftp-attack.yaml explaining behavior |
| 150 | + - [ ] Note that MUSIC-SRV-01 (Ubuntu) will generate Windows logs in Phase 1 |
| 151 | + - [ ] Keep as test case for Phase 2 multi-OS implementation |
| 152 | +- [ ] Document in architecture docs that OS field exists but isn't used in Phase 1 |
| 153 | + - [ ] Update PRD.md or create architecture.md if needed |
| 154 | + - [ ] Explain that OS detection will be implemented in Phase 2.10 |
| 155 | + |
| 156 | +**Phase 1 Milestone:** Can generate small, consistent datasets across 2 log formats with schema validation. **Known limitation:** Windows-only log generation (documented and validated). |
140 | 157 |
|
141 | 158 | --- |
142 | 159 |
|
|
153 | 170 | - [ ] Test: Parallel emitter execution with state consistency |
154 | 171 | - [ ] Test: No data races or deadlocks |
155 | 172 |
|
156 | | -### 2.2 Additional Log Formats (3 more) |
| 173 | +### 2.2 Additional Log Formats & Multi-OS Support |
| 174 | + |
| 175 | +**Goal:** Add Linux log formats and additional formats for 5+ total MVP formats |
157 | 176 |
|
158 | 177 | - [ ] `formats/definitions/syslog.yaml` - Linux syslog format (RFC 5424) |
| 178 | + - [ ] Support process execution logs (daemon.log, syslog) |
| 179 | + - [ ] Support authentication logs (auth.log) |
| 180 | + - [ ] Include facility/severity mapping |
| 181 | +- [ ] `formats/definitions/auditd.yaml` - Linux audit logs (similar to Windows Security logs) |
| 182 | + - [ ] Support EXECVE records (process execution) |
| 183 | + - [ ] Support USER_AUTH records (authentication) |
| 184 | + - [ ] Support SYSCALL records (system calls) |
| 185 | +- [ ] `formats/definitions/bash_history.yaml` - Bash command history format |
| 186 | + - [ ] Support timestamped command history |
| 187 | + - [ ] Per-user history files |
159 | 188 | - [ ] `formats/definitions/snort.yaml` - Snort/Suricata alert format (fast alert) |
160 | 189 | - [ ] `formats/definitions/web.yaml` - W3C web log format |
161 | 190 | - [ ] `generation/emitters/syslog.py` - Syslog emitter |
| 191 | +- [ ] `generation/emitters/auditd.py` - Auditd emitter |
| 192 | +- [ ] `generation/emitters/bash_history.py` - Bash history emitter |
162 | 193 | - [ ] `generation/emitters/snort.py` - Snort emitter |
163 | 194 | - [ ] `generation/emitters/web.py` - Web log emitter |
164 | | -- [ ] Test: All 5 formats generate valid output |
165 | | -- [ ] Test: Cross-format consistency across all 5 formats |
| 195 | +- [ ] Test: All formats generate valid output |
| 196 | +- [ ] Test: Cross-format consistency across all formats |
| 197 | +- [ ] Test: Linux formats parse correctly with standard Linux log viewers |
166 | 198 |
|
167 | 199 | ### 2.3 Progress Reporting |
168 | 200 |
|
|
236 | 268 |
|
237 | 269 | ### 2.9 Phase 2 Testing & Scenarios |
238 | 270 |
|
239 | | -- [ ] Integration test: 8-hour scenario with all 5 formats |
| 271 | +- [ ] Integration test: 8-hour scenario with all 5+ formats |
240 | 272 | - [ ] Create test fixture: `fixtures/scenarios/medium-dataset.yaml` (100 users, 8 hours) |
241 | 273 | - [ ] Performance benchmarks (time, memory) for medium datasets |
242 | 274 | - [ ] Unit test coverage: maintain 95%+ overall |
243 | 275 | - [ ] Update README with Phase 2 capabilities |
244 | 276 |
|
245 | | -**Phase 2 Milestone:** Can generate medium-scale datasets (100K+ events) across all 5 MVP formats in parallel with good performance. |
| 277 | +### 2.10 OS-Aware Activity Generation |
| 278 | + |
| 279 | +**Goal:** Enable generation of OS-specific logs based on system.os field (addresses Phase 1 limitation) |
| 280 | + |
| 281 | +- [ ] Create OS detection utility in `utils/os.py` |
| 282 | + - [ ] `OSType` enum (Windows, Linux, MacOS, Unknown) |
| 283 | + - [ ] `detect_os_type(system: System) -> OSType` function |
| 284 | + - [ ] Pattern matching for OS strings (flexible matching: "Windows 10", "Ubuntu", "macOS", etc.) |
| 285 | + - [ ] Handle variations: "Windows Server 2019", "Ubuntu 22.04 LTS", "Red Hat", etc. |
| 286 | +- [ ] Refactor `generation/activity.py` for OS branching |
| 287 | + - [ ] `generate_process()` - detect OS and route to OS-specific process generator |
| 288 | + - [ ] `_generate_windows_process()` - existing Windows Event 4688 logic (refactored) |
| 289 | + - [ ] `_generate_linux_process()` - emit syslog/auditd entries |
| 290 | + - [ ] Update process templates to include OS-specific paths and commands |
| 291 | + - [ ] Maintain existing ActivityGenerator API (no breaking changes) |
| 292 | +- [ ] Create `generation/activity_templates.py` - OS-specific command mappings |
| 293 | + - [ ] Map storyline activities to OS-specific implementations |
| 294 | + - [ ] Windows: paths (C:\Windows\...), event IDs (4624, 4688, etc.) |
| 295 | + - [ ] Linux: paths (/usr/bin/..., /bin/...), syslog facilities, auditd record types |
| 296 | + - [ ] Common commands: whoami, ps/tasklist, netstat, etc. |
| 297 | + - [ ] Support for LOLBins on both platforms |
| 298 | +- [ ] Update `generation/engine.py` for OS-aware emitter initialization |
| 299 | + - [ ] Check system.os when creating emitters per system |
| 300 | + - [ ] Windows systems → Windows Event emitters |
| 301 | + - [ ] Linux systems → syslog/auditd/bash_history emitters |
| 302 | + - [ ] Mixed environments → coordinate across both |
| 303 | + - [ ] Zeek emitter remains OS-agnostic (network visibility) |
| 304 | +- [ ] Test: Windows system generates Windows Event logs |
| 305 | +- [ ] Test: Linux system generates syslog/auditd logs |
| 306 | +- [ ] Test: Mixed environment (Windows + Linux) generates appropriate logs per system |
| 307 | +- [ ] Test: Storyline activities correctly mapped to OS-specific commands |
| 308 | +- [ ] Test: Cross-log consistency in mixed environments (Zeek sees connections between Windows and Linux systems) |
| 309 | +- [ ] Update retail-store-ftp-attack.yaml to properly generate Linux logs for MUSIC-SRV-01 |
| 310 | +- [ ] Remove Phase 1 validation warning for non-Windows systems |
| 311 | + |
| 312 | +**Phase 2 Milestone:** Can generate medium-scale datasets (100K+ events) across all 5+ MVP formats in parallel with good performance. **Multi-OS support:** Windows and Linux systems generate appropriate OS-specific logs. |
246 | 313 |
|
247 | 314 | --- |
248 | 315 |
|
|
0 commit comments