Confounder function not available

[sophieleech]

Hi Jakob,

Just wondering if there is a reason why the create.biased.idx function has been left as an internal function rather than exported. I do notice I do get an error if I forcefully run it by simbaR:::create.biased.idx

Error in table(sim$labels, sim$conf_label) : 
  all arguments must have the same length 

Is this just incomplete currently and if so - will it be completed? I see benchmarking with confounding was included in the paper and I'd be keen to be able to create simulations with confounding.

Cheers,
Sophie

[sophieleech]

Okay I think I have answered my own question but keen for some clarification to make sure I have interpreted this correctly.

So I discovered that confounders can actually be specified with 'create.data.simulation' using the options within 'sim.params', 'conf' and 'conf.params' (see example below - apologies for the horrific formatting). Which I believe then nullifies the need for 'create.biased.idx'. Instead 'create.test.idx' can be used, as the specified bias should be roughly preserved following random subsampling (right?).

I just want to check my understanding however of these parameters as there doesn't seem to be documentation for these yet.

so 'conf' can be = to either 'artificial', 'global', 'batch' or a binary column of the metadata. I think I understand what artificial is doing - an additional resampling step on features not effected by the main group of interest, which essentially generates an additional label marker (but without swapping of zero and non-zero values between groups?) - but I am not sure what the other options do?

As what I think I understand is 'artificial', which allows for specification of 'bias', 'prop', 'feat.prop' and 'feat.effect' I want to just double check my understanding of each of these parameters as well:

'bias' represents the overlap between the confounder and group of interest, where bal = 0.5 results in an even distribution of the confounding variable between groups e.g. if prop = 0.5 and bal =0.5, group of interest + 1 confounder results in 4 equal groups (1/1, -1/1, 1/-1, -1/-1) while bal = 1 would result in all confounder_label = 1 being in label = 1 (if prop = 0.5, this results in 2 groups, 1/1 and -1/-1).

'prop' is equivalent to 'class.balance' for the main label and determines the proportion of subjects in each confounder group. This interacts with bias to determine the overlap between the confounder groups and the main interest groups.

'feat.prop' is equivalent to 'prop.markers' and determines the proportion of features that are differentially abundant.

'feat.effect' is equivalent to 'ab.scale' and determines the scaling of features (limited between 2 and 20).

##set up for loop 
ab.scale = c(1.0, 1.25, 1.5, 2.0, 5.0, 10.0, 20.0)
prev.scale = c(0.0, 0.1, 0.2, 0.3)
prop.markers = c(0.01, 0.05, 0.1, 0.2, 0.3)
class.balance = c(0.5, 0.6, 0.7, 0.8)
feature.type = c('all', 'low')
bias.con <- c(0.5, 0.6, 0.7, 0.8, 0.9)
prop.con <- c(0.2, 0.3, 0.4, 0.5)
feat.prop.con <- c(0.05, 0.1, 0.2, 0.3)
feat.effect.con <- c(2, 5, 10, 20)

for (ab in ab.scale){
	for (prev in prev.scale){
		if (ab == 1.0 & prev == 0.0){
			next
		}
		for (prop in prop.markers){
			for (class.bal in class.balance){
				for (feat in feature.type){
					for (bias in bias.con){
						for (prop.c in prop.con) {
							for (feat.c in feat.prop.con) {
								for (feat.eff.c in feat.effect.con){
									#simulating for bunya 
									sim.implant <- paste0('/Volumes/MIM022-Q5375/SIMBA_simulations/confounded/implantation_ab', ab, '_prev', prev,
											'_prop', prop, '_bal', class.bal, '_type.', feat, '_rep20_sub_24_50_100_200_rep10_conf', bias,
											'_prop.con', prop.c, '_feat.prop.con', feat.c, '_feat.effect.con', feat.eff.c ,'_230425.h5')
									
									print(paste0('ab.scale = ', ab, ' prev.scale = ', prev, ' prop.markers = ', prop,
									' class.balance = ', class.bal, ' feature.type = ', feat, 
                                                                           ' bias = ', bias, ' prop.con = ', prop.c, ' feat.prop.con = ', feat.c, 
                                                                            ' feat.effect.con = ', feat.eff.c))
									# create simulations
									system.time(
										create.data.simulation(feat = toy.feat,
												meta = toy.meta,
												sim.location = sim.implant,
												sim.type ='cross-section',
												sim.method ='resampling',
										filt.params = list(ab.cutoff=0, prev.cutoff=0, log.n0=1e-05), 
                                                                                   sim.params = list(ab.scale=ab, prev.scale=prev, 	prop.markers=prop, 																			 																								 									 																					 																											 								 																		 									 
                                                                                    conf=c('artificial'), 
                                                                                   conf.params = list(bias = bias, prop = prop.c, feat.prop = feat.c, feat.effect = feat.eff.c),																			 																														 																															 																																	 																																				 									 
                                                                                   class.balance=class.bal, 																		 									 
                                                                                   feature.type=feat,																				 									 
                                                                                   repeats=20)
									))
								h5closeAll()
	if (class.bal == 0.8){
	     system.time(
             simbaR::create.test.idx(sim.location=sim.implant,																				 
             subsets=c(50,100,200), 																					 
             repetitions=10))
	} else {
	system.time(
         simbaR::create.test.idx(sim.location=sim.implant.																						 
         subsets=c(24,50,100,200), 																					 
         repetitions=10))
	}
										
       h5closeAll()
								}
							}
						}
					}
				}
			}
		}
	}
}

[jakob-wirbel]

Hi @sophieleech
many apologies for the late answer!!! The last couple of days had been a bit busy on my side

Apologies, the documentation is not really in great shape in some instances...

In principle, we envisioned three different kinds of confounders. All of them suppose that there is a real label parameter that you are interested in. Let's say that this would be a disease label (cancer vs control, for example).

• artificial: This creates an artificial confounder label and will implant features into this label. Think about a medication that affects some microbes. Then the testing indices are created so that you sample from the four groups (label+, confounder+;label-,confounder-;label+,confounder-;label-,confounder-) with a specific bias parameter. This one was the one that we explored the most in the paper, because in this one you have the most control about the implanted features (effect size, proportion of affected features, etc.)

• batch: This simulates batch effects between studies. So you will have to submit several studies as a list for the features and the metadata. So in this case, it will create re-sample from these groups: (label+,study1;label+,study2;label-,study1;label-,study2). If there is a big batch effect between studies, lots of features will be different between the batches. So, de-convoluting which one is a "real" implanted feature or just background will be a bit more complicated.

• global: This one is similar to the batch in that it tries to have a confounder affecting lots of different features, but if you have only a single study. It will do a PCoA and the proportion of a sample to be in the positive/negative label group will depend on the first two principal coordinates. That means that there should be large compositional differences between groups for testing again. Similar to batch, there is very little control about which features will be affected by the confounder: globally, there should be differences between groups.

I hope this makes sense! Let me know if you have any other questions. Depending on your research question, i think the artificial confounder situation is probably the most applicable since you have the most control about the "confounded" features.

[sophieleech]

Great thanks Jakob that is very helpful!

[sophieleech]

Is there any prevalence shifting applied to the confounding groups? I assumed there was and it was just the same as what is applied to the main group but now I am thinking maybe there isn't any at all.

[jakob-wirbel]

It's doing both prevalence shift and abundance shift to the confounder. I think it mirrors exactly the same parameters as the main label, unless otherwise specified.
let me double-check, though

[jakob-wirbel]

If you specify an abundance shift for the confounder, it will use only that value and only the abundance shift. If you don't specify anything, it will mirror the same effect size as the normal label.

[sophieleech]

Great thanks!