Skip to content

Commit 56cd59e

Browse files
shanestshanest
authored
Merge pull request #3 from CLMBRs/colors
Add Files Relating to Color Convexity Paper
2 parents d1d807d + d9874b6 commit 56cd59e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+267696
-0
lines changed

‎src/examples/colors/README.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
## Colors
2+
3+
Based on the paper [TODO]
4+
5+
## Setup
6+
After setting up the `eff_conv` environment, install the proper prerequisites:
7+
```sh
8+
python -m pip install -r requirements.txt
9+
```
10+
11+
## How To Run
12+
13+
### Installing The Data
14+
15+
Please install the `model.pkl` following the direction from [here](https://github.com/nogazs/ib-color-naming/tree/master). Place the `model.pkl` file in the `data` folder
16+
17+
Please additionally install the `WCS-Data-20110316.zip` file from the World Color Survey data from [here](https://linguistics.berkeley.edu/wcs/data.html). Make sure the unzipped folder is in `data`. Make sure to also install the mappings (`cnum-vhcm-lab-new.txt`) and place them in the `data` folder as well.
18+
19+
### Creating the WCS and CIELab visualizations
20+
21+
TODO: The stimulus grid needs to be updated
22+
23+
In the `examples` directory, run the following commands to get CIELab visualization (Figure 2 in the paper):
24+
```sh
25+
python -m colors.visualization.cloud color_reg
26+
python -m colors.visualization.cloud color_rot 20
27+
python -m colors.visualization.cloud color_diff 20 diff
28+
```
29+
30+
This will create the files `color_reg.png`, `color_rot.png`, and `color_diff.png` in the `output/color` folder.
31+
32+
33+
### Minimizing Data
34+
35+
The raw model files for each experiement are very large (ranging from 250 MB for artifical models to over 14 GB for the color model). In order to distribute the data and for ease of iteration, we provide minimized `.csv` files containing the relevant data in the GitHub repository. We also include the raw model files we generated [here](https://osf.io/x3dkz/).
36+
37+
If you wish to run this experiment with your own data, after doing the data generation for experiment 1 and 2 you will need to generate the `.csv` files using the following command:
38+
39+
```sh
40+
python -m colors.utils.minimize_models
41+
```
42+
43+
### Experiment 1
44+
45+
#### Data Generation
46+
47+
To generate the color model run the following command in the `examples` directory:
48+
```sh
49+
python -m colors.exp1.generate_color_model
50+
```
51+
52+
#### Data Analysis
53+
54+
Please ensure you have the minimized `.csv` files before doing analysis.
55+
56+
You can generate the plots and correlations using the respective commands:
57+
```sh
58+
# Plots
59+
python -m colors.exp1.generate_color_plot
60+
python -m colors.exp1.gen_whisker_plots
61+
# Correlations
62+
python -m colors.exp1.generate_color_correlation
63+
python -m colors.utils.ols_model color_model
64+
```
65+
66+
TODO: ADD OTHER CORRELATION MODEL AND OTHER ANALYSIS
67+
68+
### Experiments 2 & 3
69+
70+
#### Data Generation
71+
72+
To generate the test models for experiments 2 and 3, you can run the followign command in the `examples` directory:
73+
```sh
74+
python -m colors.exp2_3.generate_models
75+
```
76+
77+
### Data Analysis
78+
79+
Please ensure you have the minimized `.csv` files before doing analysis.
80+
81+
After this you can generate the plots and correlations using the respective commands:
82+
```sh
83+
python -m colors.exp2_3.generate_plots
84+
python -m colors.exp2_3.calculate_correlation
85+
```
86+
87+
TODO: ADD OTHER CORRELATION MODEL AND OTHER ANALYSIS
88+
89+
## Additional Documentation
90+
- `exp1.check_diff`
91+
- Experimental: Used to get the data regarding the differences with nearby neighbors
92+
- `exp1.gen_whisker_plots`
93+
- The script to generate the whisker plots in the paper. The output will be in `output/convexity/whisker.png`
94+
- `exp1.generate_color_correlation`
95+
- The script to generate the color correlation file and table. The correlation file will be in `output/convexity/color_coefficients.csv`
96+
- `exp1.generate_color_model`
97+
- The script to generate the color model. The model file will be in `data/convexity/color_model.pkl`. This script will take a few days to run.
98+
- `exp1.generate_color_plot`
99+
- The script to generate the color plot. The model file will be in `output/output/color.png`.
100+
- `exp2_3.models.*`
101+
- The scaffolding for the various artifical models showcased in the paper.
102+
- `exp2_3.calculate_correlation`
103+
- The script to generate the convexity correlation files and tables. The correlation files will be in `output/convexity/qmw_coefficients.csv` and `output/convexity/quw_coefficients.csv`
104+
- `exp2_3.generate_models`
105+
- The script to generate the artificial model files. The model files will be in `data/convexity/model-*.pkl`
106+
- `exp2_3.generate_plots`
107+
- The script to generate the artificial model plots. The plots will be in `output/convexity/tile_qmw.png` and `output/convexity/tile_quw.png`
108+
- `utils.correlation`
109+
- This contains a function to convert the correlation tables into a formatted LaTeX table
110+
- `utils.migrate`
111+
- This was a script to fix discrepencies between an old format of models and a more organized format
112+
- `utils.minimize_models`
113+
- This minimizes the models in the `data/convexity/` folder into `.csv` files and outputs them in `data/minimized/`
114+
- `utils.ols_model`
115+
- Takes in an argument, that being the name of a minimized `.csv` file without the extension. Creates an OLS model for the given model.
116+
- `visualization.cloud`
117+
- Takes in 1 - 3 arguemnts, the first being the name of the output file and the second (optional) being a rotation value, the final can argument is `diff` and toggles whether or not to show difference point cloud or the regular point cloud.
118+
- `visualization.gen_wcs_grid`
119+
- Generates a very simplistic WCS stimulus grid. To change the rotation a constant must be manually edited in the file.
120+
121+
## Notes
122+
123+
If you found this useful please cite the following paper:
124+
[TODO]

‎src/examples/colors/data/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
model.pkl
2+
cnum-vhcm-lab-new.txt
3+
WCS-Data-20110316/*
4+
!.gitignore

‎src/examples/colors/data/convexity/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
*
2+
!.gitignore

‎src/examples/colors/data/download.sh‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/bin/bash
2+
3+
# WCS Data
4+
curl -O https://linguistics.berkeley.edu/wcs/data/cnum-maps/cnum-vhcm-lab-new.txt
5+
curl -O https://linguistics.berkeley.edu/wcs/data/20041016/WCS-Data-20110316.zip
6+
unzip WCS-Data-20110316.zip
7+
rm WCS-Data-20110316.zip
8+
9+
# IB Color Naming Model
10+
curl -L -O https://www.dropbox.com/s/70w953orv27kz1o/IB_color_naming_model.zip?dl=1
11+
unzip IB_color_naming_model.zip
12+
mv IB_color_naming_model/IB_color_naming.pkl model.pkl
13+
rm -r IB_color_naming_model
14+
rm IB_color_naming_model.zip
15+
16+
# TODO: OSF Data

0 commit comments

Comments
 (0)