Soil moisture in wrong units for some datasets

[arthur-e]

According to the 2020 paper in Global Change Biology, soil moisture in the "data" tables should be in volumetric m^3/m^3 but this seems to vary by dataset. "d20200114_CARBONE_BURNS", "d20190526_RAICH_survey", and "d20200328_UEYAMA_HOKUROKU" are three (there may be more) that seem to have soil moisture in percent (%) units.

> csr_table('data', 'd20190526_RAICH_survey')
d20190526_RAICH_survey	Reading standardized data
# A tibble: 4,858 x 10
   CSR_PORT CSR_TIMESTAMP_BEGIN CSR_TIMESTAMP_END   CSR_FLUX_CO2 CSR_DRY_CO2 CSR_RH CSR_SM5 CSR_T5 CSR_TAIR_AMB
      <int> <dttm>              <dttm>                     <dbl>       <dbl>  <dbl>   <dbl>  <dbl>        <dbl>
 1        3 2004-09-29 10:08:30 2004-09-29 10:10:00         7.92        423    78.1    24.3   25           26.6
 2        3 2004-09-29 10:14:30 2004-09-29 10:16:00         6.94        399    74.3    37.8   24.7         26.6
 3        3 2004-09-29 10:18:30 2004-09-29 10:20:00         7.67        417.   77.2    36.9   25           26.7

For now, I'm going to assume datasets with a maximum soil moisture value < 1 are in m^3/m^3 units and those with a max. > 1 are in percent units.

[arthur-e]

Also, at least one dataset has negative soil moisture values!

> csr_table('data', 'd20200212_KAYE_UNW')
d20200212_KAYE_UNW	Reading standardized data
# A tibble: 14,707 x 7
   CSR_PORT CSR_TIMESTAMP_BEGIN CSR_TIMESTAMP_END   CSR_FLUX_CO2 CSR_SM5 CSR_T5 CSR_DATASET       
      <int> <dttm>              <dttm>                     <dbl>   <dbl>  <dbl> <chr>             
 1        0 2015-07-13 23:51:30 2015-07-13 23:53:00         5.6   -0.840   19.0 d20200212_KAYE_UNW
 2        0 2015-07-14 00:51:30 2015-07-14 00:53:00         5.77  -0.837   18.9 d20200212_KAYE_UNW

[bpbond]

Thanks @arthur-e ! Great to identify. I will investigate about the fraction vs. percentage thing—ugh, tests should catch this.

The negative issue, hmm, separate but good to know. Thanks.

[arthur-e]

I hate to pile on here, but there are a couple of other datasets to note. Below are percentiles for the CSR_SM5 field (5-cm soil moisture). The TP39 dataset has the correct units (m3 m-3) but the other two do not seem to be correct, i.e., there is too little variation around (and above) 1.0 for these to be in percent units.

Dataset	Port	Flux	1%	10%	50%	90%	99%
d20200417_ARAIN_TP39	1	Rs	0.06	0.06	0.10	0.15	0.19
d20200417_ARAIN_TP39	2	Rs	0.06	0.06	0.11	0.15	0.20
d20200417_ARAIN_TP39	3	Rh	0.06	0.06	0.11	0.15	0.21
d20200417_ARAIN_TP39	4	Rs	0.06	0.06	0.10	0.15	0.20
d20200417_ARAIN_TP39	5	Rs	0.06	0.06	0.10	0.14	0.19
d20200417_ARAIN_TP39	6	Rs	0.06	0.06	0.10	0.14	0.19
d20200417_ARAIN_TP39	7	Rs	0.06	0.06	0.10	0.14	0.18
d20200417_ARAIN_TP39	8	Rh	0.06	0.06	0.09	0.14	0.18
d20200417_ARAIN_TP74	1	Rs	1.09	1.15	1.21	1.45	1.63
d20200417_ARAIN_TP74	2	Rs	1.19	1.21	1.32	1.56	1.65
d20200417_ARAIN_TP74	3	Rs	1.19	1.21	1.38	1.59	1.71
d20200417_ARAIN_TP74	4	Rs	1.18	1.25	1.48	1.59	1.66
d20200417_ARAIN_TPD	1	Rs	0.19	0.76	0.89	1.11	2.04
d20200417_ARAIN_TPD	2	Rs	0.78	0.88	1.05	1.15	1.50
d20200417_ARAIN_TPD	3	Rs	0.03	0.17	0.91	1.12	3.47
d20200417_ARAIN_TPD	4	Rs	0.77	0.81	1.03	1.11	1.18
d20200417_ARAIN_TPD	5	Rs	0.73	0.75	1.00	1.09	1.15

I might reach out to the authors and see if they can clarify. I assume they would have the option of uploading correcting data to COSORE.