Skip to content

Commit 17b849e

Browse files
juanis2112juanis2112
committed
Add CURIOSS config file and modify readme
1 parent 68f0da9 commit 17b849e

23 files changed

Lines changed: 213 additions & 14 deletions

‎README.md‎

Lines changed: 37 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# Repository Finder
22

33
## Overview
4-
Repository Finder is a tool that identifies and analyzes open-source repositories affiliated with universities using GitHub metadata and contributor analysis. The pipeline is split into four modular scripts:
4+
Repository Finder is a tool that identifies and analyzes open-source repositories affiliated with universities using GitHub metadata and contributor analysis. The institutions covered by this project are members of [CURIOSS](https://curioss.org/) (Community for University and Research Institution OSPOs). The pipeline is split into four modular scripts:
55

66
- **main_scraping.py**: Fetches and stores raw repository, organization, and contributor data.
77

88
- **main_filtering.py**: Filters repositories based on affiliation with a specific university.
99

1010
- **main_analysis.py**: Analyzes and visualizes filtered repository data, including license usage, language distribution, and community practices.
1111

12-
- **main_analysis_combined.py**: Analyzes and visualizes aggregated repository data by type. We use this script to analyze the data of the 10 University of California campuses.
12+
- **main_analysis_combined.py**: Analyzes and visualizes aggregated repository data by type. We use this script to analyze data across all 32 supported institutions.
1313

1414
## Installation
1515
1. **Clone the repository:**
@@ -35,21 +35,44 @@ This step collects raw data from GitHub, including repositories, organizations,
3535
Data/db/repository_data_{ACRONYM}_database.db
3636
```
3737
38-
There are already configuration files available for ten universities from the University of California System:
39-
- UCB (University of California, Berkeley)
40-
- UCD (University of California, Davis)
41-
- UCI (University of California, Irvine)
42-
- UCLA (University of California, Los Angeles)
43-
- UCM (University of California, Merced)
44-
- UCR (University of California, Riverside)
45-
- UCSD (University of California, San Diego)
46-
- UCSB (University of California, Santa Barbara)
47-
- UCSC (University of California, Santa Cruz)
38+
There are already configuration files available for **32 institutions** in `config/`:
39+
40+
- BIOHUB (Biohub)
41+
- CMU (Carnegie Mellon University)
42+
- ETH (ETH Zürich)
43+
- GEORGIA_TECH (Georgia Institute of Technology)
44+
- GWU (George Washington University)
45+
- JHU (Johns Hopkins University)
46+
- LERO (Lero — The Science Foundation Ireland Research Centre for Software)
47+
- MGB (Mass General Brigham)
48+
- MSU (Mississippi State University)
49+
- OSU (Oregon State University)
50+
- RIT (Rochester Institute of Technology)
51+
- SLU (Saint Louis University)
52+
- SNT (SnT – Université du Luxembourg)
53+
- STANFORD (Stanford University)
54+
- SYRACUSE (Syracuse University)
55+
- TCD (Trinity College Dublin)
56+
- UCB (University of California, Berkeley)
57+
- UCD (University of California, Davis)
58+
- UCI (University of California, Irvine)
59+
- UCL (University College London)
60+
- UCLA (University of California, Los Angeles)
61+
- UCM (University of California, Merced)
62+
- UCR (University of California, Riverside)
63+
- UCSB (University of California, Santa Barbara)
64+
- UCSC (University of California, Santa Cruz)
65+
- UCSD (University of California, San Diego)
4866
- UCSF (University of California, San Francisco)
67+
- UGA (Université Grenoble Alpes)
68+
- UMICH (University of Michigan)
69+
- UT_AUSTIN (University of Texas at Austin)
70+
- UVM (University of Vermont)
71+
- UWMADISON (University of Wisconsin, Madison)
4972
50-
For a simple test case, replace `university_acronyms = ['UCSD']` in `repofinder/main_scraping.py` with the acronym of the university you would like to collect data from.
73+
For a simple test case, replace `university_acronyms = ['UCSD']` in `repofinder/main_scraping.py` with the acronym of the institution you would like to collect data from (use the config slug, e.g. `georgia_tech`, `ut_austin`).
5174
52-
For any other university, create a configuration file inside the config folder and update the path accordingly.
75+
For any additional institution, create a configuration file inside the `config/` folder (e.g. `config/config_{acronym}.json`) and update the path accordingly.
5376
5477
5578
Run the scraping script:

‎config/config_biohub.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Biohub",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "Biohub",
5+
"UNIVERSITY_EMAIL_DOMAIN": "biohub.org",
6+
"UNIVERSITY_WEBSITE_URL": "biohub.org",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_cmu.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Carnegie Mellon University",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "CMU",
5+
"UNIVERSITY_EMAIL_DOMAIN": "cmu.edu",
6+
"UNIVERSITY_WEBSITE_URL": "cmu.edu",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_eth.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "ETH Zürich",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "ETH",
5+
"UNIVERSITY_EMAIL_DOMAIN": "ethz.ch",
6+
"UNIVERSITY_WEBSITE_URL": "ethz.ch",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_georgia_tech.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Georgia Institute of Technology",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "Georgia Tech",
5+
"UNIVERSITY_EMAIL_DOMAIN": "gatech.edu",
6+
"UNIVERSITY_WEBSITE_URL": "gatech.edu",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_gwu.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "George Washington University",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "GWU",
5+
"UNIVERSITY_EMAIL_DOMAIN": "gwu.edu",
6+
"UNIVERSITY_WEBSITE_URL": "gwu.edu",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_jhu.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Johns Hopkins University",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "JHU",
5+
"UNIVERSITY_EMAIL_DOMAIN": "jhu.edu",
6+
"UNIVERSITY_WEBSITE_URL": "jhu.edu",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_lero.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Lero The Science Foundation Ireland Research Centre for Software",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "Lero",
5+
"UNIVERSITY_EMAIL_DOMAIN": "sfi.ie",
6+
"UNIVERSITY_WEBSITE_URL": "sfi.ie",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_mgb.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Mass General Brigham",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "MGB",
5+
"UNIVERSITY_EMAIL_DOMAIN": "massgeneralbrigham.org",
6+
"UNIVERSITY_WEBSITE_URL": "massgeneralbrigham.org",
7+
"ADDITIONAL_QUERIES": []
8+
}

‎config/config_msu.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"UNIVERSITY_NAME": "Mississippi State University",
3+
"UNIVERSITY_LOCATION": "",
4+
"UNIVERSITY_ACRONYM": "MSU",
5+
"UNIVERSITY_EMAIL_DOMAIN": "msstate.edu",
6+
"UNIVERSITY_WEBSITE_URL": "msstate.edu",
7+
"ADDITIONAL_QUERIES": []
8+
}

0 commit comments

Comments
 (0)