Skip to content

Commit 1044293

Browse files
azvoleffazvoleff
committed
Update readme
1 parent a59f8b0 commit 1044293

File tree

1 file changed

+164
-7
lines changed

1 file changed

+164
-7
lines changed

‎README.md‎

Lines changed: 164 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,142 @@
11
# trends.earth Environment
22

3+
[![Tests](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/tests.yml/badge.svg)](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/tests.yml)
4+
[![Code Quality](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/code-quality.yml/badge.svg)](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/code-quality.yml)
5+
[![Security Scan](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/security.yml/badge.svg)](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/security.yml)
6+
37
This project belongs to the trends.earth project by Conservation International.
48

59
This repository implements the Core Platform of the trends.earth Environment for running Google Earth Engine scripts.
610

7-
## Status
11+
## Project Structure
812

9-
[![Tests](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/tests.yml/badge.svg)](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/tests.yml)
10-
[![Code Quality](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/code-quality.yml/badge.svg)](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/code-quality.yml)
11-
[![Security Scan](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/security.yml/badge.svg)](https://github.com/ConservationInternational/trends.earth-Environment/actions/workflows/security.yml)
13+
```
14+
trends.earth-Environment/
15+
├── gefcore/ # Core Python package
16+
│ ├── __init__.py # Package initialization and main entry point
17+
│ ├── api.py # API client for trends.earth-API
18+
│ ├── loggers.py # Custom logging handlers
19+
│ └── runner.py # Script execution engine
20+
├── scripts/ # Utility scripts
21+
│ └── dependency_manager.py # Security and dependency management
22+
├── tests/ # Test suite
23+
├── .github/workflows/ # CI/CD workflows
24+
├── Dockerfile # Container build configuration
25+
├── main.py # Application entry point
26+
├── entrypoint.sh # Docker container entrypoint
27+
├── requirements.txt # Production dependencies
28+
├── requirements-dev.txt # Development dependencies
29+
├── pyproject.toml # Project configuration and tools
30+
└── run_tests.sh # Test execution script
31+
```
32+
33+
## Related Projects
34+
35+
- [trends.earth API](https://github.com/ConservationInternational/trends.earth-API) - Backend API service
36+
- [trends.earth CLI](https://github.com/ConservationInternational/trends.earth-CLI) - Command-line interface
37+
38+
## Development Setup
39+
40+
### Prerequisites
41+
42+
- Python 3.10+
43+
- Docker (for containerized development)
44+
- Google Earth Engine service account (for production use)
45+
46+
### Local Development
47+
48+
1. **Clone the repository:**
49+
```bash
50+
git clone https://github.com/ConservationInternational/trends.earth-Environment.git
51+
cd trends.earth-Environment
52+
```
53+
54+
2. **Create and activate virtual environment:**
55+
```bash
56+
python -m venv .venv
57+
source .venv/bin/activate # On Windows: .venv\Scripts\activate
58+
```
59+
60+
3. **Install dependencies:**
61+
```bash
62+
pip install -r requirements.txt
63+
pip install -r requirements-dev.txt
64+
```
65+
66+
4. **Run tests:**
67+
```bash
68+
./run_tests.sh
69+
# Or directly with pytest
70+
pytest tests/ -v
71+
```
72+
73+
### Docker Development
74+
75+
1. **Build the container:**
76+
```bash
77+
docker build -t trends-earth-env .
78+
```
79+
80+
2. **Run the container:**
81+
```bash
82+
docker run --rm \
83+
-e ENV=development \
84+
-e TESTING=false \
85+
trends-earth-env
86+
```
87+
88+
## Environment Variables
89+
90+
Required environment variables for production use:
1291

13-
Check out the other parts of the trends.earth project:
92+
- `ENV` - Environment mode (`production`, `staging`, `development`, `test`)
93+
- `TESTING` - Set to `true` for test environments
94+
- `ROLLBAR_SCRIPT_TOKEN` - Rollbar error reporting token
95+
- `GOOGLE_PROJECT_ID` - Google Cloud Project ID
96+
- `GEE_ENDPOINT` - Google Earth Engine API endpoint
97+
- `API_URL` - trends.earth API base URL
98+
- `API_USER` - API authentication username
99+
- `API_PASSWORD` - API authentication password
100+
- `EXECUTION_ID` - Unique execution identifier
101+
- `PARAMS_S3_BUCKET` - S3 bucket for parameters
102+
- `PARAMS_S3_PREFIX` - S3 key prefix for parameters
14103

15-
- The API [trends.earth API](https://github.com/ConservationInternational/trends.earth-API)
104+
## Testing
105+
106+
The project includes comprehensive testing with pytest:
107+
108+
```bash
109+
# Run all tests
110+
./run_tests.sh
111+
112+
# Run specific test files
113+
pytest tests/test_api.py -v
114+
115+
# Run with coverage
116+
pytest tests/ --cov=gefcore --cov-report=html
117+
118+
# Run only fast tests (exclude slow integration tests)
119+
pytest tests/ -m "not slow"
120+
```
121+
122+
## Code Quality
123+
124+
Code quality is maintained using:
125+
126+
- **Ruff** - Fast Python linter and formatter
127+
- **mypy** - Static type checking
128+
- **pytest** - Testing framework with 92%+ coverage requirement
129+
130+
Run quality checks:
131+
132+
```bash
133+
# Linting and formatting
134+
ruff check gefcore/ tests/
135+
ruff format gefcore/ tests/
136+
137+
# Type checking
138+
mypy gefcore/ --ignore-missing-imports
139+
```
16140

17141
## Security
18142

@@ -46,6 +170,39 @@ safety scan -r requirements.txt
46170
# Scan code for security issues
47171
bandit -r gefcore/
48172

49-
# Container security scan (requires Docker)
173+
## Architecture
174+
175+
### Core Modules
176+
177+
- **`gefcore.api`** - HTTP client for communicating with trends.earth-API, handles authentication, retries, and error handling
178+
- **`gefcore.runner`** - Main script execution engine that initializes Google Earth Engine and executes user scripts
179+
- **`gefcore.loggers`** - Custom logging handlers that send logs to the API and handle different environment configurations
180+
- **`scripts.dependency_manager`** - Security and dependency management utilities
181+
182+
### Execution Flow
183+
184+
1. Container starts via `main.py` → `gefcore.__init__.py`
185+
2. Environment validation and logger setup
186+
3. Google Earth Engine authentication and initialization
187+
4. Script parameter retrieval from S3
188+
5. User script execution with monitoring and logging
189+
6. Results and status reporting back to API
190+
191+
## Container security scan (requires Docker)
50192
docker run --rm -v $(pwd):/workspace aquasec/trivy fs /workspace
51193
```
194+
195+
## Contributing
196+
197+
1. Fork the repository
198+
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
199+
3. Make your changes and add tests
200+
4. Run the test suite (`./run_tests.sh`)
201+
5. Ensure code quality checks pass
202+
6. Commit your changes (`git commit -m 'Add amazing feature'`)
203+
7. Push to the branch (`git push origin feature/amazing-feature`)
204+
8. Open a Pull Request
205+
206+
## License
207+
208+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

0 commit comments

Comments
 (0)