Skip to content

Commit f1a0274

Browse files
committed
Comprehensive EdgeFlow implementation and lint fixes
1 parent a13a196 commit f1a0274

1,253 files changed

Lines changed: 48104 additions & 6511 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,37 @@ tests/
4848
*.joblib
4949

5050
# Allow only small test/example model files
51+
!models/simple_cnn.keras
52+
53+
# Ignore generated files
54+
generated/
55+
56+
# Ignore report files
57+
report.md
58+
59+
# Ignore local model files
60+
models/*.h5
61+
models/*.tflite
62+
models/*.onnx
63+
models/*.pb
64+
models/__pycache__/
65+
66+
# Ignore IDE and editor files
67+
.vscode/
68+
.idea/
69+
*.swp
70+
*.swo
71+
*~
72+
73+
# Ignore local environment files
74+
.env
75+
.env.local
76+
77+
# Ignore Python cache
78+
__pycache__/
79+
*.py[cod]
80+
*$py.class
81+
**/__pycache__/
5182
!test.tflite
5283
!test_optimized.tflite
5384
!example_*.tflite
@@ -136,3 +167,4 @@ wheels/
136167
.env.development.local
137168
.env.test.local
138169
.env.production.local
170+
node_modules/

README.md

Lines changed: 47 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,18 @@
1-
EdgeFlow Compiler with Semantic Analysis
2-
==========================================
1+
# EdgeFlow Compiler with Semantic Analysis
32

43
EdgeFlow is a domain-specific language (DSL) and compiler for optimizing TensorFlow Lite models for edge deployment. Users write simple `.ef` configuration files that describe optimization strategies (e.g., INT8 quantization), target devices, and goals like latency or size. The CLI parses these configs and runs the optimization pipeline.
54

65
**NEW: Comprehensive Semantic Analysis System** - The compiler now includes a sophisticated semantic analyzer that validates DSL models for shape compatibility, parameter ranges, resource constraints, and device compatibility before code generation.
76

87
Project Status: CLI with semantic analysis, tests, and CI. Parser and optimizer integration points ready.
98

10-
Overview
11-
--------
9+
## Overview
1210

1311
- Language: EdgeFlow `.ef` configuration files
1412
- Targets: TFLite models on edge devices (e.g., Raspberry Pi)
1513
- Pipeline: parse config → optimize model → (future) benchmark → report
1614

17-
Example `.ef`
18-
-------------
15+
## Example `.ef`
1916

2017
```bash
2118
model_path = "path/to/model.tflite"
@@ -25,8 +22,7 @@ target_device = "raspberry_pi"
2522
optimize_for = latency
2623
```
2724

28-
Installation
29-
------------
25+
## Installation
3026

3127
- Python 3.11 (CI target)
3228
- Install runtime dependencies:
@@ -41,54 +37,50 @@ For development (linting, tests, coverage, hooks):
4137
pip install -r requirements-dev.txt
4238
```
4339

44-
Usage
45-
-----
40+
## Usage
4641

4742
Basic:
4843

4944
```bash
50-
python edgeflowc.py path/to/config.ef
45+
python -m edgeflow.compiler.edgeflowc path/to/config.ef
5146
```
5247

5348
Verbose:
5449

5550
```bash
56-
python edgeflowc.py path/to/config.ef --verbose
51+
python -m edgeflow.compiler.edgeflowc path/to/config.ef --verbose
5752
```
5853

5954
Help and Version:
6055

6156
```bash
62-
python edgeflowc.py --help
63-
python edgeflowc.py --version
57+
python -m edgeflow.compiler.edgeflowc --help
58+
python -m edgeflow.compiler.edgeflowc --version
6459
```
6560

66-
Expected Behavior
67-
-----------------
61+
## Expected Behavior
6862

6963
- Missing file:
7064

7165
```bash
72-
python edgeflowc.py non_existent.ef
66+
python -m edgeflow.compiler.edgeflowc non_existent.ef
7367
# Error: File 'non_existent.ef' not found
7468
```
7569

7670
- Wrong extension:
7771

7872
```bash
79-
python edgeflowc.py invalid.txt
73+
python -m edgeflow.compiler.edgeflowc invalid.txt
8074
# Error: Invalid file extension. Expected '.ef' file
8175
```
8276

83-
CLI Options
84-
-----------
77+
## CLI Options
8578

8679
- `config_path`: Positional `.ef` file path (required)
8780
- `-v, --verbose`: Enable verbose debug output
8881
- `--version`: Print CLI version and exit
8982

90-
Language Toolchain (ANTLR)
91-
-------------------------
83+
## Language Toolchain (ANTLR)
9284

9385
Prereqs:
9486

@@ -105,13 +97,12 @@ Generate Python parser/lexer into the `parser/` package:
10597

10698
After generation, `parser/` contains `EdgeFlowLexer.py`, `EdgeFlowParser.py`, `EdgeFlowVisitor.py`, etc. The CLI automatically uses them when present; otherwise it falls back to a simple line-based parser.
10799

108-
Running the Compiler
109-
--------------------
100+
## Running the Compiler
110101

111102
Parse a `.ef` config and run the (placeholder) optimization pipeline:
112103

113104
```bash
114-
python edgeflowc.py path/to/config.ef
105+
python -m edgeflow.compiler.edgeflowc path/to/config.ef
115106
```
116107

117108
## Semantic Analysis System
@@ -166,43 +157,39 @@ else:
166157

167158
## Project Structure
168159

169-
Architecture
170-
------------
171-
172160
```bash
173161
edgeFlow/
174-
├── edgeflowc.py # CLI entry point
175-
├── semantic_analyzer/ # 🆕 Semantic analysis system
176-
│ ├── __init__.py # Main exports
177-
│ ├── analyzer.py # Core semantic analyzer
178-
│ ├── error_types.py # Error definitions and collection
179-
│ ├── ir_nodes.py # IR graph and node structures
180-
│ ├── constraints.py # Parameter ranges and device constraints
181-
│ └── compiler_integration.py # Integration with compiler pipeline
182-
├── parser/ # ANTLR-generated modules + wrapper
183-
├── optimizer.py # Model optimization logic
184-
├── benchmarker.py # Performance measurement tools
185-
├── reporter.py # Report generation
186-
├── examples/ # 🆕 Semantic analysis examples
187-
│ └── semantic_analysis_examples.py
188-
├── tests/ # Unit tests
189-
│ ├── test_cli.py
190-
│ └── test_semantic_analyzer.py # 🆕 Semantic analyzer tests
191-
├── .github/workflows/ci.yml # CI: lint, type, test, coverage badge
162+
├── src/
163+
│ └── edgeflow/
164+
│ ├── compiler/ # Core compiler logic
165+
│ ├── deployment/ # Deployment orchestration
166+
│ ├── optimization/ # Optimization strategies
167+
│ ├── analysis/ # Static and semantic analysis
168+
│ ├── reporting/ # Error reporting and explainability
169+
│ ├── benchmarking/ # Performance benchmarking
170+
│ ├── config/ # Configuration handling
171+
│ ├── ir/ # Intermediate Representation
172+
│ ├── pipeline/ # Pipeline orchestration
173+
│ ├── utils/ # Utility functions
174+
│ ├── parser/ # Parser logic
175+
│ ├── backend/ # Backend API
176+
│ └── frontend/ # Frontend application
177+
├── scripts/ # Helper scripts
178+
├── tests/ # Unit and integration tests
179+
├── docs/ # Documentation
180+
├── .github/workflows/ # CI/CD workflows
192181
├── requirements.txt # Runtime dependencies
193182
├── requirements-dev.txt # Dev/test dependencies
194183
├── README.md # This file
195-
└── .pre-commit-config.yaml # Pre-commit hooks
184+
└── Makefile # Build automation
196185
```
197186

198-
Integration Points
199-
------------------
187+
## Integration Points
200188

201189
- Parser (`parser.parse_ef(path)`): `edgeflowc.load_config` tries to import and call this. If not found yet, it falls back to returning a minimal config with raw text.
202190
- Optimizer (`optimizer.optimize(config)`): `edgeflowc.optimize_model` tries to import and call this. If not found yet, it logs a message and continues.
203191

204-
Development
205-
-----------
192+
## Development
206193

207194
Set up pre-commit hooks:
208195

@@ -225,8 +212,7 @@ Run tests with coverage:
225212
pytest -q --cov=edgeflowc --cov-report=term-missing
226213
```
227214

228-
CI/CD
229-
-----
215+
## CI/CD
230216

231217
GitHub Actions runs on pushes and PRs for Python 3.11:
232218

@@ -235,14 +221,13 @@ GitHub Actions runs on pushes and PRs for Python 3.11:
235221
- Tests with coverage (fail below 90%)
236222
- Coverage badge artifact generated via `genbadge`
237223

238-
Web Interface
239-
-------------
224+
## Web Interface
240225

241226
Backend (FastAPI):
242227

243-
- App entry: `backend/app.py`
228+
- App entry: `src/edgeflow/backend/app.py`
244229
- Endpoints with strict CLI parity:
245-
- `POST /api/compile` (maps to `python edgeflowc.py config.ef`)
230+
- `POST /api/compile` (maps to `python -m edgeflow.compiler.edgeflowc config.ef`)
246231
- `POST /api/compile/verbose` (maps to `--verbose`)
247232
- `POST /api/optimize` (optimization phase)
248233
- `POST /api/benchmark` (benchmarking)
@@ -264,8 +249,7 @@ docker-compose up --build
264249
# Frontend: http://localhost:3000
265250
```
266251

267-
Production (CD + Reverse Proxy)
268-
-------------------------------
252+
## Production (CD + Reverse Proxy)
269253

270254
- Continuous Deployment builds/pushes GHCR images, then deploys over SSH with Docker Compose on the server.
271255
- Public site: <https://edgeflow.pointblank.club/>
@@ -274,29 +258,24 @@ Production (CD + Reverse Proxy)
274258
- Frontend: `13000` (container 3000)
275259
- Recommended: bind services to `127.0.0.1` and expose via Nginx with TLS (Certbot). Frontend proxies `/api/*` to backend inside the Docker network; backend need not be directly exposed.
276260

277-
Contributing
278-
------------
261+
## Contributing
279262

280263
- Open a PR with a focused set of changes
281264
- Ensure `black`, `isort`, `flake8`, and `mypy` pass
282265
- Add/Update tests to maintain ≥90% coverage
283266
- Clearly document changes in docstrings and README where relevant
284267

285-
Security Notes
286-
--------------
268+
## Security Notes
287269

288270
- The CLI validates that the input path is a regular file with a `.ef` extension.
289271
- Paths are normalized and resolved; the CLI does not follow any network or remote sources.
290272
- Future work: sandbox model handling and ensure safe file operations during optimization.
291273

292-
License
293-
-------
274+
## License
294275

295276
TBD (add the appropriate license file for your project).
296277

297-
298-
Compatibility Check (CLI)
299-
-------------------------
278+
## Compatibility Check (CLI)
300279

301280
- `--check-only`: run device compatibility check and exit
302281
- `--device-spec-file <path>`: load custom device specs (CSV/JSON)

backend/tests/conftest.py

Lines changed: 0 additions & 15 deletions
This file was deleted.

0 commit comments

Comments
 (0)