Skip to content

Add 43 missing Selangor cities to Malaysia dataset #1330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 4 commits into
base: master
Choose a base branch
from
+1,288 βˆ’4
Draft

Add 43 missing Selangor cities to Malaysia dataset #1330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1b1e925
Initial plan
Copilot Jan 11, 2026
2fa7dd4
Initial analysis: Plan to update Malaysia Selangor cities data
Copilot Jan 11, 2026
0f5b0a6
feat: Add 43 new Selangor cities to Malaysia data
Copilot Jan 11, 2026
dbc3424
feat: Complete Malaysia Selangor cities update with translations
Copilot Jan 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 237 additions & 0 deletions .github/fixes-docs/MALAYSIA_SELANGOR_CITIES_UPDATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
# Malaysia Selangor Cities Update

## Issue Reference
**Issue:** [Bug]: Update Malaysia state and Selangor towns data
**Problem:** The current Malaysia Selangor data was incomplete. Missing 43 towns/cities from Selangor state.

## Countries/Regions Addressed
- **Country:** Malaysia (MY)
- **State:** Selangor (state_code: 10)

## Changes Made

### Cities Added
- **Before count:** 25 Selangor cities
- **After count:** 68 Selangor cities
- **Cities added:** 43 new cities
- **Fields added:** timezone, translations (for 25 cities)
- **Coordinate updates:** 3 cities (Banting, Puchong, Sungai Besar)

### New Cities List
1. Ampang Jaya
2. Assam Jawa
3. Bandar Baru Selayang
4. Bandar Kundang
5. Bangi
6. Batang Kali
7. Batu Caves
8. Beranang
9. Bestari Jaya
10. Broga
11. Bukit Rotan
12. Cyberjaya
13. Gombak
14. Jenjarom
15. Kajang
16. Kampung Kuantan
17. Kapar
18. Kuala Kubu Bharu
19. Kuala Sungai Buloh
20. Lagong
21. Meru
22. Mutiara Damansara
23. Padang Jawa
24. Pandamaran
25. Paya Jaras
26. Port Klang
27. Sabak
28. Salak Tinggi
29. Sekinchan
30. Selayang
31. Sepang
32. Seri Kembangan
33. Sungai Air Tawar
34. Sungai Buaya
35. Sungai Burong
36. Sungai Choh
37. Sungai Pelek
38. Sungai Pelong
39. Sungai Tengi
40. Taman Melawati
41. Taman Tasik Semenyih
42. Teluk Datok
43. Ulu Yam

### Coordinate Updates
Three existing cities had their coordinates updated to match the source data:

1. **Banting**
- Old: 2.81360000, 101.50185000
- New: 2.81209800, 101.50255500

2. **Puchong**
- Old: 3.03270000, 101.61880000
- New: 3.00980000, 101.61800000

3. **Sungai Besar**
- Old: 3.67460000, 100.98670000
- New: 3.35000000, 101.25000000

### Existing Cities (No Changes)
The following 5 cities already existed with matching coordinates:
- Kuala Selangor
- Kuang
- Rawang
- Semenyih
- Tanjung Sepat

## Validation Steps

### 1. Data Import and Timezone Enrichment
```bash
# Import JSON to MySQL
python3 bin/scripts/sync/import_json_to_mysql.py --host localhost --user root --password root

# Add timezones to all new cities
python3 bin/scripts/validation/add_timezones.py --table cities --user root --password root
```

**Result:** Successfully added timezone `Asia/Kuala_Lumpur` to all 43 new cities.

### 2. Translation Enrichment
```bash
# Add translations from Wikipedia
python3 bin/scripts/validation/translation_enricher.py --file contributions/cities/MY.json --type city
```

**Result:**
- Translations added: 25 cities (58% success rate)
- No translations found: 18 cities (smaller towns without Wikipedia articles)
- Languages covered: ar, bn, de, es, fr, hi, id, it, ja, ko, nl, pl, pt, ru, tr, uk, vi, zh

### 3. Sync Back to JSON
```bash
# Sync MySQL to JSON
python3 bin/scripts/sync/sync_mysql_to_json.py --host localhost --user root --password root
```

**Result:** All cities now have assigned IDs, timezones, and available translations.

### 4. Final Verification
```bash
# Count Selangor cities
jq '[.[] | select(.state_code == "10" and .country_code == "MY")] | length' contributions/cities/MY.json
# Result: 68

# Verify timezone for new cities
jq '[.[] | select(.state_code == "10" and .country_code == "MY" and .timezone == "Asia/Kuala_Lumpur")] | length' contributions/cities/MY.json
# Result: 68 (all have timezone)

# Check translation coverage
jq '[.[] | select(.state_code == "10" and .country_code == "MY" and (.translations != null and .translations != {}))] | length' contributions/cities/MY.json
# Result: 50 (73% have translations)
```

## Data Samples

### City Entry with Full Data (Kajang)
```json
{
"id": 160571,
"name": "Kajang",
"state_id": 1944,
"state_code": "10",
"country_id": 132,
"country_code": "MY",
"latitude": "2.98330000",
"longitude": "101.78780000",
"timezone": "Asia/Kuala_Lumpur",
"translations": {
"fr": "Kajang (Selangor)",
"id": "Kajang, Malaysia",
"it": "Kajang",
"ko": "μΉ΄μž₯",
"nl": "Kajang",
"pl": "Kajang-Sungai Chua",
"ru": "КадТанг",
"tr": "Kajang",
"vi": "Kajang",
"zh": "加影"
}
}
```

### City Entry (Port Klang)
```json
{
"id": 160584,
"name": "Port Klang",
"state_id": 1944,
"state_code": "10",
"country_id": 132,
"country_code": "MY",
"latitude": "3.00507000",
"longitude": "101.40887000",
"timezone": "Asia/Kuala_Lumpur",
"translations": {
"ar": "Ω…ΩŠΩ†Ψ§Ψ‘ ΩƒΩ„Ψ§ΨΊ",
"bn": "ΰ¦ͺΰ§‹ΰ¦°ΰ§ΰ¦Ÿ ক্লাং",
"de": "Port Klang",
"es": "Puerto Klang",
"fr": "Port Klang",
"hi": "ΰ€ͺΰ₯‹ΰ€°ΰ₯ΰ€Ÿ ΰ€•ΰ₯ΰ€²ΰ₯ˆΰ€‚ΰ€—",
"ja": "γƒγƒΌγƒˆγ‚―γƒ©γƒ³",
"ko": "ν΄λž‘ν•­",
"pt": "Port Klang",
"ru": "ΠŸΠΎΡ€Ρ‚-Кланг",
"tr": "Klang LimanΔ±",
"zh": "ε·΄η”ŸζΈ―"
}
}
```

### City Entry (Cyberjaya)
```json
{
"id": 160568,
"name": "Cyberjaya",
"state_id": 1944,
"state_code": "10",
"country_id": 132,
"country_code": "MY",
"latitude": "2.92000000",
"longitude": "101.65000000",
"timezone": "Asia/Kuala_Lumpur"
}
```

## References
- **Source:** [Wikipedia - Category:Towns in Selangor](https://en.wikipedia.org/wiki/Category:Towns_in_Selangor)
- **State Information:** [Selangor on Wikipedia](https://en.wikipedia.org/wiki/Selangor)
- **WikiData:** [Selangor (Q189710)](https://www.wikidata.org/wiki/Q189710)

## Impact

### Data Quality Improvements
- βœ… **172% increase** in Selangor city coverage (25 β†’ 68 cities)
- βœ… All new cities have timezone information
- βœ… 58% of new cities have translations (25/43)
- βœ… Coordinates updated for 3 existing cities to match authoritative source

### Coverage by District
The updated data now includes cities from various Selangor districts:
- **Gombak:** Batu Caves, Selayang, Rawang, Kuala Sungai Buloh
- **Hulu Selangor:** Kuala Kubu Bharu, Batang Kali
- **Kuala Selangor:** Kuala Selangor, Bestari Jaya, Sekinchan
- **Petaling:** Cyberjaya, Puchong, Seri Kembangan, Subang Jaya
- **Sepang:** Sepang, Salak Tinggi
- **Klang:** Port Klang, Kapar
- **Hulu Langat:** Kajang, Bangi, Semenyih, Beranang

### API Changes
- No breaking changes
- New cities are added with proper foreign key relationships
- All data follows existing schema

### No Known Issues
All cities imported successfully with proper validation.
4 changes: 2 additions & 2 deletions bin/db/schema.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ CREATE TABLE `cities` (
KEY `cities_test_ibfk_2` (`country_id`),
CONSTRAINT `cities_ibfk_1` FOREIGN KEY (`state_id`) REFERENCES `states` (`id`),
CONSTRAINT `cities_ibfk_2` FOREIGN KEY (`country_id`) REFERENCES `countries` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=160557 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=COMPACT;
) ENGINE=InnoDB AUTO_INCREMENT=162557 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=COMPACT;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

Expand All @@ -198,6 +198,6 @@ CREATE TABLE `cities` (
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2025-12-13 9:08:52
-- Dump completed on 2026-01-11 7:21:08

SET FOREIGN_KEY_CHECKS=1;
2 changes: 1 addition & 1 deletion contributions/cities/BQ.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,4 +327,4 @@
"flag": 1,
"wikiDataId": "Q2511504"
}
]
]
Loading