tileMethylCounts Execution halted

[yhj-j]

Hi Dr. Akalin,

Here is the code I ran, and I submitted the command via sbatch on a cluster. The error is as follows, and my file leaf_WGBS_merge_loci is 30GB. How should I resolve this issue?

I really appreciate your time and help!

########################
library(methylKit)

file.list = list("/home/xueyan/my_data/workspace/WGBS/6.DMR/CHH/leaf_WGBS_merge_loci.CHH.txt",
"/home/xueyan/my_data/workspace/WGBS/6.DMR/CHH/Root_WGBS_merge_loci.CHH.txt")

myobj=methRead(file.list,
sample.id=list("leaf_WGBS_merge_loci","Root_WGBS_merge_loci"),
assembly="mygenome",
treatment=c(0,1),
context="CHH",
mincov = 10,
dbtype = "tabix",
dbdir = "methylDB"
)

tiles = tileMethylCounts(myobj,win.size=1000,step.size=1000,cov.bases = 10)
mb_region=methylKit::unite(tiles)

dm_region = calculateDiffMeth(mb_region, mc.cores = 10)

write.table(dm_region,
file = 'Root_vs_leaf.dmr',
sep = "\t", quote = F, row.names = F)

#################################
Received list of locations.
creating directory methylDB ...
Reading file.
compressing the file with bgzip...
making tabix index...
Reading file.
compressing the file with bgzip...
making tabix index...

checking wether tabix file already exists:
/data/xueyan/workspace/WGBS/6.DMR/methylDB/leaf_WGBS_merge_loci_tiled.txt.bgz
tabix file is new.
continuing now ...

Error in paste(tabixRes[[1]], collapse = "\n") :
result would exceed 2^31-1 bytes
Calls: tileMethylCounts ... getTabixByChr -> tabix2df -> fread -> paste0 -> paste
Execution halted

[alexg9010]

Hi @yhj-j ,

Thanks for reporting your error and sharing your code.

The error indicates that the data supposed to be loaded from the tabix file would exceed the character length limit (2^31-1 bytes), we dealt with similar issues before (see #141).
This error is tricky because you are analyzing non-CpG loci, which are way more abundant than CpG loci. The current implementation of tiling was done per chromosome, which is fine for small number of sites as CpGs but would not work for the larger number of non-CpG sites.

A fix for tabix would require adapting the loading strategy in the code.

For now, if you are working in an hpc environment and have enough memory available, you could try to switch to the in-memory object processing by setting save.db = FALSE and save intermediate results via RDS files or as Tabix file via makeMethylDB.

Please let us know if that helped, or if you need more help.

Best,
Alex