Skip to content

Commit b2b8a0d

Browse files
Javid912Javid912
committed
Add .gitignor, LEARNING_GUIDE.md and more
1 parent 62312d3 commit b2b8a0d

15 files changed

+1118
-125
lines changed

‎.github/workflows/ci.yml

+92
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
name: GitHub Events Analytics CI
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v3
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v4
18+
with:
19+
python-version: '3.9'
20+
21+
- name: Install dependencies
22+
run: |
23+
python -m pip install --upgrade pip
24+
pip install pytest pytest-cov
25+
pip install -r data-collector/requirements.txt
26+
pip install -r spark-jobs/requirements.txt
27+
28+
- name: Run tests
29+
run: |
30+
pytest --cov=data-collector --cov=spark-jobs
31+
32+
- name: Upload coverage report
33+
uses: codecov/codecov-action@v3
34+
35+
lint:
36+
runs-on: ubuntu-latest
37+
38+
steps:
39+
- uses: actions/checkout@v3
40+
41+
- name: Set up Python
42+
uses: actions/setup-python@v4
43+
with:
44+
python-version: '3.9'
45+
46+
- name: Install dependencies
47+
run: |
48+
python -m pip install --upgrade pip
49+
pip install flake8 black isort
50+
51+
- name: Lint with flake8
52+
run: |
53+
flake8 data-collector spark-jobs --count --select=E9,F63,F7,F82 --show-source --statistics
54+
55+
- name: Check formatting with black
56+
run: |
57+
black --check data-collector spark-jobs
58+
59+
- name: Check imports with isort
60+
run: |
61+
isort --check-only --profile black data-collector spark-jobs
62+
63+
docker:
64+
runs-on: ubuntu-latest
65+
needs: [test, lint]
66+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
67+
68+
steps:
69+
- uses: actions/checkout@v3
70+
71+
- name: Set up Docker Buildx
72+
uses: docker/setup-buildx-action@v2
73+
74+
- name: Login to DockerHub
75+
uses: docker/login-action@v2
76+
with:
77+
username: ${{ secrets.DOCKERHUB_USERNAME }}
78+
password: ${{ secrets.DOCKERHUB_TOKEN }}
79+
80+
- name: Build and push data collector
81+
uses: docker/build-push-action@v4
82+
with:
83+
context: ./data-collector
84+
push: true
85+
tags: ${{ secrets.DOCKERHUB_USERNAME }}/github-events-collector:latest
86+
87+
- name: Build and push spark jobs
88+
uses: docker/build-push-action@v4
89+
with:
90+
context: ./spark-jobs
91+
push: true
92+
tags: ${{ secrets.DOCKERHUB_USERNAME }}/github-events-processor:latest

‎.gitignore

+67
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Python
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.so
6+
.Python
7+
env/
8+
build/
9+
develop-eggs/
10+
dist/
11+
downloads/
12+
eggs/
13+
.eggs/
14+
lib/
15+
lib64/
16+
parts/
17+
sdist/
18+
var/
19+
*.egg-info/
20+
.installed.cfg
21+
*.egg
22+
venv/
23+
.venv/
24+
.env
25+
26+
# Docker
27+
.docker/
28+
29+
# Logs
30+
*.log
31+
logs/
32+
33+
# Spark
34+
spark-warehouse/
35+
metastore_db/
36+
derby.log
37+
checkpoint/
38+
/tmp/checkpoints/
39+
40+
# Kafka
41+
kafka-logs/
42+
43+
# Database
44+
*.db
45+
*.sqlite3
46+
47+
# Grafana
48+
grafana-data/
49+
50+
# IDE
51+
.idea/
52+
.vscode/
53+
*.swp
54+
*.swo
55+
56+
# OS specific
57+
.DS_Store
58+
.DS_Store?
59+
._*
60+
.Spotlight-V100
61+
.Trashes
62+
ehthumbs.db
63+
Thumbs.db
64+
65+
# Project specific
66+
postgres-data/
67+
spark-checkpoints/

‎CONTRIBUTING.md

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Contributing to GitHub Events Analytics
2+
3+
Thank you for considering contributing to GitHub Events Analytics! This document provides guidelines and instructions for contributing to the project.
4+
5+
## Table of Contents
6+
7+
1. [Code of Conduct](#code-of-conduct)
8+
2. [Getting Started](#getting-started)
9+
3. [How to Contribute](#how-to-contribute)
10+
4. [Development Workflow](#development-workflow)
11+
5. [Pull Request Process](#pull-request-process)
12+
6. [Coding Standards](#coding-standards)
13+
7. [Testing](#testing)
14+
8. [Documentation](#documentation)
15+
16+
## Code of Conduct
17+
18+
This project and everyone participating in it is governed by our Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to the project maintainers.
19+
20+
## Getting Started
21+
22+
1. Fork the repository on GitHub
23+
2. Clone your fork locally
24+
3. Set up the development environment following the instructions in the README.md
25+
4. Create a new branch for your feature or bug fix
26+
27+
## How to Contribute
28+
29+
You can contribute to the project in several ways:
30+
31+
- **Reporting Bugs**: Create an issue describing the bug, including steps to reproduce, expected behavior, and actual behavior
32+
- **Suggesting Enhancements**: Create an issue describing your enhancement suggestion with a clear title and detailed description
33+
- **Code Contributions**: Submit a pull request with your changes
34+
- **Documentation**: Improve or add documentation
35+
- **Reviewing Pull Requests**: Review and comment on open pull requests
36+
37+
## Development Workflow
38+
39+
1. Create a new branch from `main` for your changes
40+
2. Make your changes in small, logical commits
41+
3. Add or update tests as necessary
42+
4. Ensure all tests pass
43+
5. Update documentation as needed
44+
6. Submit a pull request
45+
46+
## Pull Request Process
47+
48+
1. Ensure your code follows the project's coding standards
49+
2. Update the README.md or other documentation if necessary
50+
3. Include tests for your changes
51+
4. Ensure the test suite passes
52+
5. Submit the pull request to the `main` branch
53+
6. The pull request will be reviewed by maintainers
54+
7. Address any feedback or requested changes
55+
8. Once approved, your pull request will be merged
56+
57+
## Coding Standards
58+
59+
- Follow PEP 8 style guide for Python code
60+
- Use meaningful variable and function names
61+
- Write docstrings for all functions, classes, and modules
62+
- Keep functions small and focused on a single task
63+
- Use type hints where appropriate
64+
- Format code with Black and sort imports with isort
65+
66+
## Testing
67+
68+
- Write unit tests for all new functionality
69+
- Ensure existing tests pass with your changes
70+
- Run the test suite before submitting a pull request:
71+
```
72+
./run_tests.sh
73+
```
74+
- Aim for high test coverage
75+
76+
## Documentation
77+
78+
- Update documentation for any changed functionality
79+
- Document new features or components
80+
- Keep the README.md up to date
81+
- Use clear and concise language
82+
- Include examples where appropriate
83+
84+
Thank you for contributing to GitHub Events Analytics!

0 commit comments

Comments
 (0)