You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
where R is the number of Required checks that pass or fail, and O is the number of Optional checks that pass or fail. This ensures that datasets are not penalized for failing optional checks. This system allows the Arctic Data Center to evaluate FAIR-ness of its datasets over time. The figure below shows the quality scores of the repository through time for metadata FAIRness.
To evaluate data quality over a wide range of data types and disciplines, the Arctic Data Center first conceptualized four categories of data quality checks. These categories are: congruency, accessibility, validity, and accuracy.
19
19
20
20
#### Congruency {.unnumbered}
21
21
22
22
**Is the file what it says it is?**
23
23
24
-
Examples:
25
24
- data format matches format listed
26
25
- checksum of file matches documentation
27
26
- variable names in file match documentation
@@ -30,7 +29,6 @@ Examples:
30
29
31
30
**Can the file be accessed and read correctly?**
32
31
33
-
Examples:
34
32
- text formatted files valid according to subtype (eg: csv files match format)
35
33
- binary files validate against declared type (eg: hdf5 is valid against HDF5 standard)
36
34
- characters in text files are valid against declared encoding
@@ -83,15 +81,17 @@ Warnings are issued for checks that failed, but are optional. Within each row, o
At the Arctic Data Center we can use these checks to gain insights about our data holdings. TODO: maybe insert figures here?
88
88
89
89
### Checking Data Quality in R
90
90
91
91
Poll:
92
92
93
-
- What are the most common quality issues you see? (free text)
94
-
- What kind of data quality checks do you use on your own data? (free text)
93
+
- What are the most common quality issues you see?
94
+
- What kind of data quality checks do you use on your own data?
95
95
- Do you have ideas for checks you think we should add to the ADC quality suite?
96
96
97
97
Although the Arctic Data Center has implemented data quality checks that are automated, the vast majority of data quality work still falls on researchers doing analysis. In this section we will discuss a few tools that researchers can use in R to do general data quality work.
0 commit comments