Skip to content

Commit 52ed2fe

Browse files
temanisyarattemanisyarat
authored
Merge pull request #41 from williamu04/readme
Comprehensive readme
2 parents 65ffab9 + fe68877 commit 52ed2fe

1 file changed

Lines changed: 189 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 189 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,189 @@
1-
yeah
1+
# Teman Isyarat — Project Manager
2+
3+
[![GitHub](https://img.shields.io/badge/project-temanisyarat-blue)](https://github.com/williamu04/temanisyarat)
4+
[![Obsidian](https://img.shields.io/badge/editor-Obsidian-7C3AED)](https://obsidian.md)
5+
[![Status](https://img.shields.io/badge/status-active-brightgreen)]()
6+
7+
> **Teman Isyarat** (Indonesian for "Sign Friend") is an AI-powered Indonesian Sign Language (BISINDO) recognition system — developed by a student team at **Universitas Sebelas Maret (UNS)** under the Hibah Jarprak program, in partnership with **GERKATIN Solo**.
8+
9+
This repository is the **central project management and documentation hub** for the Teman Isyarat project. It uses an **Obsidian vault** structure to organize architecture decisions, technical specifications, task tracking, and team logs. The actual source code (landmark extraction, model training, mobile app, website, backend) lives in separate repositories.
10+
11+
---
12+
13+
## Table of Contents
14+
15+
- [About the Project](#about-the-project)
16+
- [Repository Structure](#repository-structure)
17+
- [Technology Stack](#technology-stack)
18+
- [Architecture Decisions](#architecture-decisions)
19+
- [Getting Started](#getting-started)
20+
- [Contributing](#contributing)
21+
- [Team](#team)
22+
- [Acknowledgments](#acknowledgments)
23+
- [License](#license)
24+
25+
---
26+
27+
## About the Project
28+
29+
BISINDO (Bahasa Isyarat Indonesia) is the native sign language used by the Deaf community in Indonesia. Despite its widespread use, technological support for BISINDO recognition remains limited, creating communication barriers between Deaf and hearing individuals.
30+
31+
Teman Isyarat tackles this by building a **real-time gesture recognition system** capable of classifying **20 BISINDO vocabulary words** (Central Java dialect) using:
32+
33+
- **MediaPipe** for holistic landmark extraction (hands + pose)
34+
- **GRU neural networks** for temporal gesture classification
35+
- **Flutter + Kotlin** for an on-device Android application
36+
- **Video augmentation** (FFmpeg) to expand a limited dataset
37+
38+
The project timeline spans **February — June 2026**, covering dataset collection with GERKATIN Solo, model development, mobile app implementation, user field testing, and community socialization.
39+
40+
### Vocabulary Set (20 Gestures)
41+
42+
| Category | Words | **Total** |
43+
| ------------ | ------------------------------ | --------- |
44+
| **Pronouns** | Aku, Kamu, Dia | 3 |
45+
| **Common** | Salam, Terimakasih, Maaf, Nama | 4 |
46+
| **Time** | Hari ini, Besok | 2 |
47+
| **Color** | Merah, Kuning | 2 |
48+
| **Family** | Ayah, Ibu | 2 |
49+
| **Count** | Satu, Dua, Tiga | 3 |
50+
| **Other** | Teman, Buku | 2 |
51+
| **Fruit** | Apel, Pisang | 2 |
52+
53+
54+
---
55+
56+
## Repository Structure
57+
58+
```
59+
.
60+
├── docs/
61+
│ ├── adr/ # Architecture Decision Records (why)
62+
│ └── spec/ # Technical Specifications (how)
63+
├── tasks/
64+
│ ├── backlog.md # Administrative backlog & grant tasks
65+
│ ├── logbook.md # Academic logbook (course credit)
66+
│ └── sprint-current.md # Current sprint tracking
67+
├── logs/ # Individual team member progress logs
68+
├── assets/ # Design mockups, screenshots, diagrams
69+
├── .github/workflows/ # CI/CD (auto-merge, asset renaming)
70+
├── CONTRIBUTING.md # Contribution guidelines (INA)
71+
└── README.md # ← You are here
72+
```
73+
74+
### Key Sections
75+
76+
| Path | Description |
77+
|---|---|
78+
| `docs/adr/` | 6 ADRs covering video strategy, GRU selection, gesture count, GERKATIN partnership, grant reporting, and extraction analysis |
79+
| `docs/spec/` | 5 specs detailing MediaPipe extraction, mobile stack, FFmpeg augmentation, recording standards, and Flutter+MediaPipe integration |
80+
| `tasks/` | Project backlog, academic logbook, and sprint planning |
81+
| `logs/` | Per-member work logs (update with every PR) |
82+
| `assets/` | UI mockups, documentation images, branding assets |
83+
84+
---
85+
86+
## Technology Stack
87+
88+
The broader Teman Isyarat software ecosystem spans multiple platforms:
89+
90+
### Machine Learning Pipeline
91+
92+
| Component | Technology |
93+
|---|---|
94+
| Landmark Extraction | MediaPipe Tasks Vision (Hand, Pose, Face Landmarker) |
95+
| Video Processing | OpenCV-Python, FFmpeg |
96+
| Model Architecture | Gated Recurrent Unit (GRU) — TensorFlow / Keras |
97+
| Augmentation | FFmpeg (brightness ±0.1, horizontal flip, combinations — 6× per video) |
98+
99+
### Mobile Application
100+
101+
| Component | Technology |
102+
|---|---|
103+
| Cross-Platform UI | Flutter / Dart |
104+
| Native Camera | Kotlin — CameraX |
105+
| On-Device ML | MediaPipe Tasks via Android PlatformView |
106+
| Communication | MethodChannel (Dart ↔ Kotlin) |
107+
108+
### Web Platform
109+
110+
| Component | Technology |
111+
|---|---|
112+
| Frontend | NextJS |
113+
| Backend | Go (Golang) |
114+
115+
---
116+
117+
## Architecture Decisions
118+
119+
Key rationale documented in `docs/adr/`:
120+
121+
| ADR | Decision |
122+
|---|---|
123+
| [ADR-001](docs/adr/adr-001-pengambilan-video.md) | 5-day distributed video recording over single-day marathon to reduce participant fatigue |
124+
| [ADR-002](docs/adr/adr-002-pilih-gru.md) | **GRU over LSTM/RNN** — better temporal sequence modeling for gesture recognition with fewer parameters |
125+
| [ADR-003](docs/adr/adr-003-jumlah-gesture.md) | **20 gestures** — balanced between research significance and dataset feasibility |
126+
| [ADR-004](docs/adr/adr-004-mitra-gerkatin.md) | **GERKATIN Solo** as community partner over PUSBISINDO for regional dialect alignment |
127+
| [ADR-005](docs/adr/adr-005-laporan-kemajuan.md) | **70% stage-1 spending** to mitigate bureaucratic delays in grant fund disbursement |
128+
| [ADR-006](docs/adr/adr-006-extraction-analysis.md) | **Drop face landmarks** — 84.5% NaN detection rate, reducing input from 252 → 153 dims |
129+
130+
---
131+
132+
## Getting Started
133+
134+
### Prerequisites
135+
136+
- [Obsidian](https://obsidian.md) (recommended for browsing/editing)
137+
138+
### Open the Vault
139+
140+
```bash
141+
git clone git@github.com:williamu04/temanisyarat-manager.git
142+
cd temanisyarat-manager
143+
# Open this folder as an Obsidian vault
144+
```
145+
146+
### Browse Documentation
147+
148+
Start with the architecture decisions to understand *why* key choices were made, then move to specs for *how* each component is built.
149+
150+
---
151+
152+
## Contributing
153+
154+
We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) (in Indonesian) for:
155+
156+
- Git workflow (fork → feature branch → PR)
157+
- Commit message conventions (`feat`, `fix`, `docs`, etc.)
158+
- Branch naming (`feature/`, `fix/`, `docs/`, etc.)
159+
- PR template and review process
160+
- **Mandatory:** update `logs/{nama}.md` with every pull request
161+
162+
---
163+
164+
## Team
165+
166+
| Name | Role |
167+
|---|---|
168+
| Dunwill William | Project Lead |
169+
| Fredy Ramadhan | R&D |
170+
| Hany Wachidatul Aisyah | R&D |
171+
| Ian | R&D |
172+
| Ivan | R&D |
173+
| Kevin | R&D |
174+
| Mutia | R&D |
175+
| Saidah | R&D |
176+
177+
---
178+
179+
## Acknowledgments
180+
181+
- **GERKATIN Solo** — Dataset collection, validation, and field testing partnership
182+
- **Universitas Sebelas Maret (UNS)** — Hibah Jarprak funding and academic support
183+
- **Google MediaPipe Team** — Open-source landmark detection models
184+
185+
---
186+
187+
## License
188+
189+
This project is developed for academic purposes under the Hibah Jarprak program at Universitas Sebelas Maret.

0 commit comments

Comments
 (0)