Skip to content

Commit 197aeeb

Browse files
haowu77claude
haowu77
and
claude
committed
Add NER detection engine (GLiNER) and community infrastructure
- Add GLiNER zero-shot NER as alternative detection engine (default), with automatic fallback to regex when gliner is not installed - Add --detection-engine CLI flag to switch between ner/regex - Add NerConfig/DetectionConfig dataclasses and config.json sections - Add pip install privacy-mask[ner] optional dependency - Add issue templates (bug, feature, new-rule), PR template, Code of Conduct - Update architecture diagram to reflect dual-engine design - Update README with detection engine docs - Bump version to 0.3.0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent cab53fb commit 197aeeb

18 files changed

Lines changed: 735 additions & 15 deletions

‎.github/ISSUE_TEMPLATE/bug.yml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
name: Bug Report
2+
description: Report a bug in privacy-mask
3+
labels: ["bug"]
4+
body:
5+
- type: markdown
6+
attributes:
7+
value: |
8+
Thanks for reporting a bug! Please fill in the details below.
9+
10+
- type: textarea
11+
id: description
12+
attributes:
13+
label: Description
14+
description: What happened? What did you expect to happen?
15+
placeholder: Describe the bug clearly
16+
validations:
17+
required: true
18+
19+
- type: textarea
20+
id: reproduce
21+
attributes:
22+
label: Steps to reproduce
23+
description: How can we reproduce this bug?
24+
placeholder: |
25+
1. Run `privacy-mask mask screenshot.png`
26+
2. ...
27+
validations:
28+
required: true
29+
30+
- type: textarea
31+
id: image
32+
attributes:
33+
label: Sample image (if applicable)
34+
description: >
35+
If the bug relates to detection or masking, attach a sample image
36+
(redact any real sensitive data first!).
37+
38+
- type: input
39+
id: version
40+
attributes:
41+
label: privacy-mask version
42+
description: Output of `privacy-mask --version`
43+
placeholder: "0.2.0"
44+
validations:
45+
required: true
46+
47+
- type: dropdown
48+
id: os
49+
attributes:
50+
label: Operating System
51+
options:
52+
- macOS
53+
- Ubuntu / Debian
54+
- Windows
55+
- Other Linux
56+
validations:
57+
required: true
58+
59+
- type: input
60+
id: python
61+
attributes:
62+
label: Python version
63+
placeholder: "3.12"
64+
validations:
65+
required: true
66+
67+
- type: textarea
68+
id: logs
69+
attributes:
70+
label: Error output / logs
71+
description: Paste any error messages or JSON output
72+
render: shell

‎.github/ISSUE_TEMPLATE/feature.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: Feature Request
2+
description: Suggest a new feature or improvement
3+
labels: ["enhancement"]
4+
body:
5+
- type: markdown
6+
attributes:
7+
value: |
8+
Thanks for suggesting an improvement! Please describe your idea below.
9+
10+
- type: textarea
11+
id: problem
12+
attributes:
13+
label: Problem
14+
description: What problem does this feature solve?
15+
placeholder: I'm always frustrated when...
16+
validations:
17+
required: true
18+
19+
- type: textarea
20+
id: solution
21+
attributes:
22+
label: Proposed solution
23+
description: How would you like this to work?
24+
validations:
25+
required: true
26+
27+
- type: textarea
28+
id: alternatives
29+
attributes:
30+
label: Alternatives considered
31+
description: Any other approaches you've thought about?
32+
33+
- type: textarea
34+
id: context
35+
attributes:
36+
label: Additional context
37+
description: Screenshots, links, or anything else relevant

‎.github/ISSUE_TEMPLATE/new-rule.yml‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: New Detection Rule
2+
description: Propose a new regex detection rule for a sensitive data pattern
3+
labels: ["new-rule"]
4+
body:
5+
- type: markdown
6+
attributes:
7+
value: |
8+
Adding detection rules is the most common contribution to privacy-mask.
9+
Please provide the details below so we can evaluate and add it.
10+
11+
- type: input
12+
id: name
13+
attributes:
14+
label: Rule name
15+
description: Short identifier (e.g. `US_DRIVER_LICENSE`, `FRENCH_SSN`)
16+
placeholder: COUNTRY_TYPE
17+
validations:
18+
required: true
19+
20+
- type: textarea
21+
id: description
22+
attributes:
23+
label: What does this rule detect?
24+
description: Describe the sensitive data type and which countries/regions it applies to
25+
placeholder: "French social security number (numéro de sécurité sociale) — 15 digits"
26+
validations:
27+
required: true
28+
29+
- type: input
30+
id: pattern
31+
attributes:
32+
label: Regex pattern
33+
description: >
34+
The regex pattern to match. Use Python regex syntax.
35+
Remember to test for false positives against common English words.
36+
placeholder: '(?<!\d)\d{3}-\d{2}-\d{4}(?!\d)'
37+
validations:
38+
required: true
39+
40+
- type: textarea
41+
id: positive
42+
attributes:
43+
label: Positive test cases (should match)
44+
description: List strings that should be detected by this rule
45+
placeholder: |
46+
- 1 85 07 91 234 089 72
47+
- 2 99 05 12 345 678 23
48+
validations:
49+
required: true
50+
51+
- type: textarea
52+
id: negative
53+
attributes:
54+
label: Negative test cases (should NOT match)
55+
description: >
56+
List strings that look similar but should NOT trigger this rule.
57+
Think about common OCR artifacts and English words.
58+
placeholder: |
59+
- ORGANIZATION
60+
- REQUIRED
61+
- 123456
62+
validations:
63+
required: true
64+
65+
- type: textarea
66+
id: references
67+
attributes:
68+
label: References
69+
description: Links to documentation about this data format

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
## What
2+
3+
<!-- Brief description of the change -->
4+
5+
## Why
6+
7+
<!-- Why is this change needed? Link related issues with "Closes #NNN" -->
8+
9+
## Type of change
10+
11+
- [ ] Bug fix
12+
- [ ] New detection rule
13+
- [ ] New feature
14+
- [ ] Refactoring (no behavior change)
15+
- [ ] Documentation
16+
17+
## Checklist
18+
19+
- [ ] I have read [CONTRIBUTING.md](../CONTRIBUTING.md)
20+
- [ ] Tests pass locally: `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest tests/ -v`
21+
- [ ] New detection rules have both positive AND negative test cases
22+
- [ ] No real secrets or PII in test data (use runtime string construction)

‎CODE_OF_CONDUCT.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Contributor Covenant Code of Conduct
2+
3+
## Our Pledge
4+
5+
We as members, contributors, and leaders pledge to make participation in our
6+
community a friendly, safe, and welcoming experience for everyone.
7+
8+
## Our Standards
9+
10+
Examples of behavior that contributes to a positive environment:
11+
12+
- Using welcoming and inclusive language
13+
- Being respectful of differing viewpoints and experiences
14+
- Gracefully accepting constructive criticism
15+
- Focusing on what is best for the community
16+
- Showing empathy towards other community members
17+
18+
Examples of unacceptable behavior:
19+
20+
- Trolling, insulting or derogatory comments, and personal attacks
21+
- Public or private harassment
22+
- Publishing others' private information without explicit permission
23+
- Other conduct which could reasonably be considered inappropriate
24+
25+
## Enforcement Responsibilities
26+
27+
Community leaders are responsible for clarifying and enforcing our standards of
28+
acceptable behavior and will take appropriate and fair corrective action in
29+
response to any behavior that they deem inappropriate.
30+
31+
## Scope
32+
33+
This Code of Conduct applies within all community spaces, and also applies when
34+
an individual is officially representing the community in public spaces.
35+
36+
## Enforcement
37+
38+
Instances of unacceptable behavior may be reported to the project team at
39+
**fullstackcrew-alpha@users.noreply.github.com**.
40+
41+
All complaints will be reviewed and investigated promptly and fairly.
42+
43+
## Attribution
44+
45+
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org),
46+
version 2.1, available at
47+
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html).

‎README.md‎

Lines changed: 49 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,12 @@ This matters for compliance too: **GDPR**, **HIPAA**, and other regulations requ
5353
## Quick Start
5454

5555
```bash
56-
# Install
56+
# Install (regex engine only)
5757
pip install privacy-mask
5858

59+
# Install with NER engine (recommended)
60+
pip install privacy-mask[ner]
61+
5962
# Mask a screenshot
6063
privacy-mask mask screenshot.png
6164

@@ -96,8 +99,44 @@ privacy-mask follows the [agentskills.io](https://agentskills.io) SKILL.md stand
9699
97100
---
98101

102+
## Detection Engines
103+
104+
privacy-mask supports two detection engines, switchable via config or CLI:
105+
106+
| Engine | Description | Install |
107+
|--------|-------------|---------|
108+
| **NER** (default) | Zero-shot Named Entity Recognition via [GLiNER](https://github.com/urchade/GLiNER). Detects person names, addresses, organizations, dates of birth, medical conditions, and more — without regex. | `pip install privacy-mask[ner]` |
109+
| **Regex** | 47 hand-tuned regex rules covering 15+ countries. No extra dependencies. | `pip install privacy-mask` |
110+
111+
```bash
112+
# Default: NER engine (requires privacy-mask[ner])
113+
privacy-mask mask screenshot.png
114+
115+
# Switch to regex engine
116+
privacy-mask mask screenshot.png --detection-engine regex
117+
```
118+
119+
You can also set the default engine in `config.json`:
120+
121+
```json
122+
{
123+
"detection": { "engine": "ner" }
124+
}
125+
```
126+
127+
---
128+
99129
## What It Detects
100130

131+
### NER Engine
132+
133+
Configurable entity types (zero-shot, no training needed):
134+
- Person names, street addresses, organization names
135+
- Dates of birth, medical conditions, license plate numbers
136+
- Custom entity types via `config.json` `ner.entity_types`
137+
138+
### Regex Engine
139+
101140
**47 regex rules** covering **15+ countries**:
102141

103142
| Category | Rules |
@@ -119,9 +158,13 @@ privacy-mask follows the [agentskills.io](https://agentskills.io) SKILL.md stand
119158

120159
1. **OCR** — Dual-engine: Tesseract + RapidOCR extract text with bounding boxes. Multi-strategy preprocessing (grayscale, binarization, contrast enhancement) with confidence-based merge for maximum accuracy.
121160

122-
2. **Detect** — 47 compiled regex rules scan OCR results. Overlapping detections on nearby bounding boxes are merged automatically.
161+
2. **Line Grouping** — OCR results are grouped into logical text lines using vertical overlap analysis.
123162

124-
3. **Mask** — Matched regions are blurred (default) or filled with solid color. Output is saved as a new file or overwrites the original.
163+
3. **Detect** — Switchable engine:
164+
- **NER** (default) — GLiNER zero-shot NER identifies entities (names, addresses, etc.) without regex
165+
- **Regex** — 47 compiled regex rules scan for structured patterns (IDs, phone numbers, API keys)
166+
167+
4. **Mask** — Matched regions are blurred (default) or filled with solid color. Output is saved as a new file or overwrites the original.
125168

126169
---
127170

@@ -143,6 +186,9 @@ privacy-mask mask screenshot.png --method fill
143186
# Choose OCR engine (tesseract, rapidocr, or combined)
144187
privacy-mask mask screenshot.png --engine tesseract
145188

189+
# Choose detection engine (ner or regex)
190+
privacy-mask mask screenshot.png --detection-engine regex
191+
146192
# Custom config
147193
privacy-mask mask screenshot.png --config my_rules.json
148194

‎SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ description: >-
88
leaves your machine. Use when receiving screenshots that may contain private
99
data, or when the user mentions privacy / masking / redacting / PII removal /
1010
sensitive data protection.
11-
version: 0.2.4
11+
version: 0.3.0
1212
license: MIT
1313
compatibility: Requires tesseract OCR and Python 3.10+. All processing is local and offline.
1414
metadata:

‎examples/architecture.svg‎

Lines changed: 1 addition & 1 deletion
Loading

‎marketplace.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
"name": "privacy-mask",
55
"repo": "fullstackcrew-alpha/privacy-mask",
66
"description": "Auto-mask sensitive info in images before AI upload. 100% local, 47 regex rules, dual OCR engine.",
7-
"version": "0.2.4",
7+
"version": "0.3.0",
88
"keywords": ["privacy", "pii", "masking", "redaction", "security", "ocr", "screenshot", "sensitive-data", "data-protection", "offline", "local", "claude-code", "api-keys", "secrets"]
99
}
1010
]

‎mask_engine/__init__.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
"""Local Image Privacy Mask Engine."""
22

3-
__version__ = "0.1.0"
3+
__version__ = "0.3.0"
44

55
from .ocr import OcrResult, run_ocr
66
from .detector import Detection, detect_sensitive
77
from .masker import apply_mask
88
from .pipeline import run_pipeline, MaskResult
9-
from .config import Config, load_config
9+
from .config import Config, load_config, NerConfig, DetectionConfig
1010

1111
__all__ = [
1212
"OcrResult",
@@ -18,4 +18,6 @@
1818
"MaskResult",
1919
"Config",
2020
"load_config",
21+
"NerConfig",
22+
"DetectionConfig",
2123
]

0 commit comments

Comments
 (0)