Skip to content

Commit 175904f

Browse files
author
Halvani
committed
Updated README.md + changed default spaCy model to "Large".
1 parent 488e362 commit 175904f

3 files changed

Lines changed: 45 additions & 48 deletions

File tree

Demo_POSNoise.ipynb

Lines changed: 20 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -2,48 +2,16 @@
22
"cells": [
33
{
44
"cell_type": "markdown",
5-
"id": "1a592652",
5+
"id": "50297305",
66
"metadata": {},
77
"source": [
88
"## POSNoise"
99
]
1010
},
1111
{
1212
"cell_type": "code",
13-
"execution_count": 1,
14-
"id": "f3b4b9ca",
15-
"metadata": {},
16-
"outputs": [
17-
{
18-
"name": "stdout",
19-
"output_type": "stream",
20-
"text": [
21-
"[POSNoise] Loading spaCy model 'en_core_web_sm' (disabled: parser, ner)...\n",
22-
"[POSNoise] Loaded 'en_core_web_sm'.\n",
23-
"The original dataset contains two partitions comprising sockpuppets and non-sockpuppets cases.\n",
24-
"The @ # Ø two # Ø # and @@@ #.\n"
25-
]
26-
}
27-
],
28-
"source": [
29-
"import posnoise\n",
30-
"from posnoise import POSNoise"
31-
]
32-
},
33-
{
34-
"cell_type": "code",
35-
"execution_count": 2,
36-
"id": "9555ed7f",
37-
"metadata": {},
38-
"outputs": [],
39-
"source": [
40-
"posnoise_instance = POSNoise(spacy_model_size=posnoise.core.SpacyModelSize.Large)"
41-
]
42-
},
43-
{
44-
"cell_type": "code",
45-
"execution_count": 4,
46-
"id": "48f4addb",
13+
"execution_count": 5,
14+
"id": "f7dc478d",
4715
"metadata": {
4816
"scrolled": true
4917
},
@@ -52,18 +20,22 @@
5220
"name": "stdout",
5321
"output_type": "stream",
5422
"text": [
55-
"I love python !\n",
56-
"I Ø # !\n"
23+
"Fitzgerald made her first tour of Australia in July 1954 for the Australian-based American promoter Lee Gordon.\n",
24+
"§ made her first # of § in § µ for the §-Ø @ # § §.\n"
5725
]
5826
}
5927
],
6028
"source": [
6129
"import posnoise\n",
6230
"from posnoise import POSNoise\n",
6331
"\n",
64-
"posnoise_instance = POSNoise(spacy_model_size=posnoise.core.SpacyModelSize.Large)\n",
32+
"# By default POSNoise loads the \"Large\" spaCy model\n",
33+
"posnoise_instance = POSNoise()\n",
34+
" \n",
35+
"# In case you want specify another model, just set it accordingly e.g. \n",
36+
"posnoise_instance = POSNoise(spacy_model_size=posnoise.core.SpacyModelSize.Medium)\n",
6537
"\n",
66-
"document = \"I love python !\"\n",
38+
"document = \"Fitzgerald made her first tour of Australia in July 1954 for the Australian-based American promoter Lee Gordon.\"\n",
6739
"posnoised_doc = posnoise_instance.pos_noise(document)\n",
6840
"\n",
6941
"print(document)\n",
@@ -73,7 +45,15 @@
7345
{
7446
"cell_type": "code",
7547
"execution_count": null,
76-
"id": "6e92a348",
48+
"id": "dfa9dad7",
49+
"metadata": {},
50+
"outputs": [],
51+
"source": []
52+
},
53+
{
54+
"cell_type": "code",
55+
"execution_count": null,
56+
"id": "b153a880",
7757
"metadata": {},
7858
"outputs": [],
7959
"source": []

README.md

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,22 +24,39 @@ The latter will pull and install the latest commit from this repository as well
2424

2525
## Quickstart
2626
```python
27-
import posnoise
2827
from posnoise import POSNoise
2928

30-
posnoise_instance = POSNoise(spacy_model_size=posnoise.core.SpacyModelSize.Large)
29+
# By default POSNoise loads the "Large" spaCy model
30+
posnoise_instance = POSNoise()
31+
32+
# In case you want specify another model, just set it accordingly e.g.
33+
# posnoise_instance = POSNoise(spacy_model_size=posnoise.core.SpacyModelSize.Medium)
3134

32-
document = "I love python !"
35+
document = "Fitzgerald made her first tour of Australia in July 1954 for the Australian-based American promoter Lee Gordon."
3336
posnoised_doc = posnoise_instance.pos_noise(document)
3437

3538
print(document)
39+
# Fitzgerald made her first tour of Australia in July 1954 for the Australian-based American promoter Lee Gordon.
40+
3641
print(posnoised_doc)
37-
38-
# I love python !
39-
# I Ø # !
42+
# § made her first # of § in § µ for the §-Ø @ # § §.
4043
```
4144

42-
A detailed explanation of the **POS placeholders** (Ø, #, etc.) used in POSNoise can be found in Table 2 in the paper. Here, the arXiv version for open access: https://arxiv.org/abs/2005.06605
45+
All part-of-speech (POS) placeholders used in POSNoise to replace topic-related words or tokens are listed below:
46+
47+
| **Category** | Tag | Examples |
48+
|----------------|-----|-----------------------------------------------|
49+
| **Noun** | # | { house, music, bird, tree, air, … } |
50+
| **Proper noun** | § | { David, Vivien, London, USA, COVID-19, … } |
51+
| **Verb** | Ø | { eat, laugh, dance, travel, hiking, … } |
52+
| **Adjective** | @ | { red, shiny, fascinating, phenomenal, … } |
53+
| **Adverb** | © | { financially, foolishly, angrily, … } |
54+
| **Numeral** | μ | { 0, 5, 2013, 3.14159, III, IV, MMXIV, … } |
55+
| **Symbol** | $ | { £, ©, §, %, #, … } |
56+
| **Other** | ¥ | { xfgh, pdl, jklw, … } |
57+
58+
59+
_Source_: [POSNoise: An Effective Countermeasure Against Topic Biases in Authorship Analysis](https://arxiv.org/abs/2005.06605) ➜ Table 2.
4360

4461
## Features
4562
- Effective way to mask topic-relatd content with **custom POS placeholders**

posnoise/core.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ def __init__(
118118
self,
119119
nlp_model: Optional["spacy.language.Language"] = None,
120120
abbrev_pos_tags: Optional[dict] = None,
121-
spacy_model_size: Union[SpacyModelSize, str] = SpacyModelSize.Small,
121+
spacy_model_size: Union[SpacyModelSize, str] = SpacyModelSize.Large,
122122
disable: Iterable[str] = ("parser", "ner"),
123123
verbose: bool = False,
124124
log_fn: Optional[Callable[[str], None]] = None,

0 commit comments

Comments
 (0)