MetaProViz::Preprocessing - CoRE=TRUE: Media blank samples - Enable removal of single metabolite values?

[ChristinaSchmidt1]

We measure the mean and sd of the blank samples and return a message with the persentage of high variable metabolites in the blank samples.

I am not sure what would be a decent cut-off for SD to take actions, meaning give error message asking the user to check certain blanks versus warning message. You are right that there should not be any variance really.

You said: "for nowI used coefficient of variation CV>1 to give a warning of high variance based on this. I will come back at this."

[dprymidis]

As it is now, we check each metabolite one at a time and report the persentage of metabolites that have "high" variance with respect to the "arbitrary" measure we applied (the CV>1). This has the problem that the CV (relative variability) is a relative measure and is a bit arbitrary and that also the threshold is arbitrary.

What we want to do is to identify whether the blank samples are "the same" or not.

What we could do is pairwise t-tests or ANOVA to find out if there are any samples that are different from the rest. But we cant really use a t -test on the samples as the means are the same because the data are TIC normalized.

So, to check if the samples are the same or if any is different we can do the following:

1. One sample a t-test to compare one sample against the group for each feature for all samples.

This results in a dataframe with a pvalue for each sample against the rest for each metabolite.

2. From here we can measure how many metabolites in a sample are statistically different (p<0.05) from the rest and get the percentage (rowSums divided by the number of metabolites).

This results in a value for each blank sample. This value represents the percentage of metabolites of this sample that are statistically different from the rest of the sample. But again here we have the same problem with CV. Where to put the threshold.

3. Here we do another t-test to see if a sample has a statistically different amount of statistically different metabolites. Because you would expect some metabolites to be different because for example of the detection limit and the adundance of the other metabolites in the samples but these differences should be small and roughly the same. So if a sample is indeed different it would have many more differences from the rest of the samples than any other sample in general against the group.

In this example I have 5 blank samples and 1 sample which is not. We can see that the % of metabolites that are different fro MS51-06 is 80% while the rest is ~10-20%. The final t-test shows the p.val of MS51-06 to be smaller than 0.05 while the rest is higher.

What do you think? Does this make sence?

[dprymidis]

Another thought is that we could use again the HotellingT2 here

[dprymidis]

About the " We cant really use a t -test on the samples as the means are the same because the data are TIC normalized."

I tested it again using the code in preprocessing and I do get the same rowmeans for the data_TIC (for each sample after TIC normalization).

I am not sure what went wrong before, I think in excel they were different because in excel it only has the geometric mean which uses the product instead of the sum.

[ChristinaSchmidt1]

ok, thanks for checking!
In this case we probably should just calculate the standard error (SE) of the replicates for each metabolites. I am not sure if here we could set a good threshold (?)

[dprymidis]

After todays meeting we decided to do:

1. If number of media samples ==0 then error("we need to have at least 1 media sample").
2. If ==1 then warning" we dont do statistics because we have only 1 media. We assume they calculated the mean beforehand") adn skip the next steps.
3. if == 2 or 3 then skip the variance testing and "outlier" sample testing (the next steps)..
4. IF >3 then Do the same as the pooled sampled and report the high variant metabolites. But do not exclud anything. Give a warning" these metabolties were found as having high variance, please check the pooled samples if you need to remove these metabolites".
5. Also take log, shapiro test and then ANOVA to test if any sample is different from the rest. IF a sample is identified as different report it and do not include it in the subsequent statistics.

[ChristinaSchmidt1]

To add for 3.: With 2 or 3 we can do a variance testing, to understand if there are features (metabolites) that appear to be really variable. If so we can report those. Otherwise, I completely agree we can not do any outlier testing (as in weather a media sample is an outlier).

[dprymidis]

There is a issue.

1. for feature filtering we ignore the blank samples.
2. Then for MVI we do half minimum based on Conditions.

It can happen (and it happens is the toy data) that we keep a metabolite that has NA values for all the blank samples because of 1. Then in 2 we get inf and thigs dont work.

What do we do here? I think we should use the half minimum of all the dataset to fill these NAs is the blank samples.

[ChristinaSchmidt1]

Just to be sure, blank samples as in the media control samples for the CoRe:

I agree thats something we should change. The problem is that the user can of course choose not to do any feature filtering or MVI for many reasons. I think here we should just treat the blank samples in the best way possible irrespectively what the user choose to do for the rest of the matrix. Meaning:

1. We process all samples as the user requested (feature filtering, MVI,...)
2. If features are removed, we remove them from the blank.
3. We do QC of the blank samples as you noted down above, but additionally we will check for NAs. Here of course it depends how many samples (%) are NAs. Over a certain threshold we should at least report those. Yet with media it can happen that we have NAs in all samples because the metabolite is not part of the media, but the cells have secreted it and hence it is in the other samples. Because of this, I would set NAs to 0 in the blanks and continue calculation. But of course if not all blanks are NA we need to consider that the metabolite is close to the detection limit and hence I would do MVI per feature here, just focusing on the blank values. As mentioned we can then reprot the metabolites where the blanks have >20% of NAs (80%-fitering rule). And in general if there are any NAs in the blank we can message the user to inform them.

Would this make sense to you?