You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: vignettes/fivethirtyeight.Rmd
+7-6
Original file line number
Diff line number
Diff line change
@@ -18,11 +18,11 @@ library(knitr)
18
18
19
19
## Motivation
20
20
21
-
We are all involved in statistics and data science education, in particular at the
21
+
We are involved in statistics and data science education, in particular at the
22
22
introductory undergraduate level. As such, we are always looking for data sets that
23
23
balance being
24
24
25
-
1.**Rich enough** to answer meaningful questions with, **real enough** to ensure that there is context, and **realistic enough** to convey to students that data as it exists "in the wild" needs processing.
25
+
1.**Rich enough** to answer meaningful questions with, **real enough** to ensure that there is context, and **realistic enough** to convey to students that data as it exists "in the wild" often needs processing.
26
26
1. Easily and quickly accessible to novices, so that we [minimize the prerequisites to research](https://arxiv.org/abs/1507.05346).
27
27
28
28
It has been our experience that many data sets that exist in R packages, such as the
@@ -31,15 +31,15 @@ It has been our experience that many data sets that exist in R packages, such as
31
31
packages, are of great pedagogical value as they:
32
32
33
33
* Satisfy the above two goals.
34
-
* Are in standardized format as they fit into the [tidy tools](https://cran.r-project.org/web/packages/tidyverse/vignettes/manifesto.html) ecosystem.
34
+
* Are in standardized format as they fit into the [tidy tools](https://CRAN.R-project.org/package=tidyverse/vignettes/manifesto.html) ecosystem.
35
35
* Are really fun to play with!
36
36
37
37
It is along these lines that we present `fivethirtyeight`: an R package of data and code behind the stories and interactives at [FiveThirtyEight.com](http://fivethirtyeight.com/), a data-driven journalism website founded by Nate Silver and owned by ESPN. FiveThirtyEight has been very forward thinking in making the data used in many of their articles open and accessible on [GitHub](https://github.com/fivethirtyeight/data), a web-based repository for collaboration on code and data.
38
38
39
39
With consultation from [Andrew Flowers](http://fivethirtyeight.com/contributors/andrew-flowers/) and [Andrei Scheinkman](http://fivethirtyeight.com/contributors/andrei-scheinkman/) of FiveThirtyEight, we go one step further by:
40
40
41
-
1. Doing just enough pre-processing so that statistics and data science novices can sink their teeth into the data.
42
-
2. Packaging it all in an easy to load format: package installation instead of working with CSV's.
41
+
1. Doing just enough pre-processing so that statistics and data science novices can sink their teeth into the data right away.
42
+
2. Packaging it all in an easy to load format: package installation instead of working with CSV files.
43
43
3. Providing easily accessible documentation: The help file for each data set includes a thorough description of the observational unit and all variables, a link to the original article, and (if listed) the data sources.
44
44
45
45
@@ -63,9 +63,10 @@ guidelines:
63
63
+ Factors vs characters:
64
64
+ Ordinal categorical variables are `factor` with the intuitive ordering of `levels`. We did this to ensure barplots and boxplots would display in an intuitive order.
65
65
+ Regular categorical variables are left as `character` vectors.
66
-
+ Convert date variables that are beyond just `year` to [POSIX date](http://www.epochconverter.com/) objects using the [lubridate](https://cran.r-project.org/web/packages/lubridate/vignettes/lubridate.html) package. That way users can easily create time series plots. Example:
66
+
+ Convert date variables that are beyond just `year` to [POSIX date](http://www.epochconverter.com/) objects using the [lubridate](https://CRAN.R-project.org/package=lubridate/vignettes/lubridate.html) package. That way users can easily create time series plots. Example:
67
67
+ If only a `year` variable exits, then we leave it as is.
68
68
+ If there are `year` and `month` variables, we convert them POSIX date objects as `year-month-01`.
69
+
+ If there are `year`, `month`, and `day` variables, we convert them POSIX date objects as `year-month-day`.
69
70
70
71
**Note**: The code used to pre-process the data can be found on the [GitHub repository](https://github.com/rudeboybert/fivethirtyeight/tree/master/data-raw) for the package in the `process_data_sets.R` files. These can serve as data manipulation/wrangling examples and exercises for more advanced students.
<h4class="author"><em>Albert Y. Kim, Chester Ismay, and Jennifer Chunn</em></h4>
35
-
<h4class="date"><em>2017-01-07</em></h4>
35
+
<h4class="date"><em>2017-01-08</em></h4>
36
36
37
37
38
38
39
39
<divid="motivation" class="section level2">
40
40
<h2>Motivation</h2>
41
-
<p>We are all involved in statistics and data science education, in particular at the introductory undergraduate level. As such, we are always looking for data sets that balance being</p>
41
+
<p>We are involved in statistics and data science education, in particular at the introductory undergraduate level. As such, we are always looking for data sets that balance being</p>
42
42
<olstyle="list-style-type: decimal">
43
-
<li><strong>Rich enough</strong> to answer meaningful questions with, <strong>real enough</strong> to ensure that there is context, and <strong>realistic enough</strong> to convey to students that data as it exists “in the wild” needs processing.</li>
43
+
<li><strong>Rich enough</strong> to answer meaningful questions with, <strong>real enough</strong> to ensure that there is context, and <strong>realistic enough</strong> to convey to students that data as it exists “in the wild” often needs processing.</li>
44
44
<li>Easily and quickly accessible to novices, so that we <ahref="https://arxiv.org/abs/1507.05346">minimize the prerequisites to research</a>.</li>
45
45
</ol>
46
46
<p>It has been our experience that many data sets that exist in R packages, such as the <ahref="https://github.com/hadley/nycflights13"><code>nycflights13</code></a>, <ahref="https://github.com/hadley/babynames"><code>babynames</code></a>, and <ahref="https://github.com/jennybc/gapminder"><code>gapminder</code></a> packages, are of great pedagogical value as they:</p>
47
47
<ul>
48
48
<li>Satisfy the above two goals.</li>
49
-
<li>Are in standardized format as they fit into the <ahref="https://cran.r-project.org/web/packages/tidyverse/vignettes/manifesto.html">tidy tools</a> ecosystem.</li>
49
+
<li>Are in standardized format as they fit into the <ahref="https://CRAN.R-project.org/package=tidyverse/vignettes/manifesto.html">tidy tools</a> ecosystem.</li>
50
50
<li>Are really fun to play with!</li>
51
51
</ul>
52
52
<p>It is along these lines that we present <code>fivethirtyeight</code>: an R package of data and code behind the stories and interactives at <ahref="http://fivethirtyeight.com/">FiveThirtyEight.com</a>, a data-driven journalism website founded by Nate Silver and owned by ESPN. FiveThirtyEight has been very forward thinking in making the data used in many of their articles open and accessible on <ahref="https://github.com/fivethirtyeight/data">GitHub</a>, a web-based repository for collaboration on code and data.</p>
53
53
<p>With consultation from <ahref="http://fivethirtyeight.com/contributors/andrew-flowers/">Andrew Flowers</a> and <ahref="http://fivethirtyeight.com/contributors/andrei-scheinkman/">Andrei Scheinkman</a> of FiveThirtyEight, we go one step further by:</p>
54
54
<olstyle="list-style-type: decimal">
55
-
<li>Doing just enough pre-processing so that statistics and data science novices can sink their teeth into the data.</li>
56
-
<li>Packaging it all in an easy to load format: package installation instead of working with CSV’s.</li>
55
+
<li>Doing just enough pre-processing so that statistics and data science novices can sink their teeth into the data right away.</li>
56
+
<li>Packaging it all in an easy to load format: package installation instead of working with CSV files.</li>
57
57
<li>Providing easily accessible documentation: The help file for each data set includes a thorough description of the observational unit and all variables, a link to the original article, and (if listed) the data sources.</li>
58
58
</ol>
59
59
</div>
@@ -74,10 +74,11 @@ <h2>Guidelines</h2>
74
74
<li>Ordinal categorical variables are <code>factor</code> with the intuitive ordering of <code>levels</code>. We did this to ensure barplots and boxplots would display in an intuitive order.</li>
75
75
<li>Regular categorical variables are left as <code>character</code> vectors.</li>
76
76
</ul></li>
77
-
<li>Convert date variables that are beyond just <code>year</code> to <ahref="http://www.epochconverter.com/">POSIX date</a> objects using the <ahref="https://cran.r-project.org/web/packages/lubridate/vignettes/lubridate.html">lubridate</a> package. That way users can easily create time series plots. Example:
77
+
<li>Convert date variables that are beyond just <code>year</code> to <ahref="http://www.epochconverter.com/">POSIX date</a> objects using the <ahref="https://CRAN.R-project.org/package=lubridate/vignettes/lubridate.html">lubridate</a> package. That way users can easily create time series plots. Example:
78
78
<ul>
79
79
<li>If only a <code>year</code> variable exits, then we leave it as is.</li>
80
80
<li>If there are <code>year</code> and <code>month</code> variables, we convert them POSIX date objects as <code>year-month-01</code>.</li>
81
+
<li>If there are <code>year</code>, <code>month</code>, and <code>day</code> variables, we convert them POSIX date objects as <code>year-month-day</code>.</li>
0 commit comments