Question regarding parallelisation in IsoQuant

[andr-kun]

Hi there,

I noticed that IsoQuant provide a parallelisation option within the DatasetProcessor class (more specifically in collect_reads method) but it is not entirely clear to me if the parallelisation is done at the read/read group level or at the chromosome level.

I was wondering if you can please clarify which parallelisation strategy was used? I tried to increase the number of threads in my IsoQuant run (using aligned BAM files as input and with a single chromosome only in the BAM, FASTA and GTF files), but when running htop, it looks like only one thread is running while the others stays suspended.

Thanks and kind regards,
Andrian

[andrewprzh]

Hi @andr-kun

IsoQuant does per-chromosome parallelization - extracting reads from a sorted BAM file is the easiest (and still more or less balanced) way to do it (https://github.com/ablab/IsoQuant/blob/master/src/dataset_processor.py#L528).

I tried running IsoQuant quite a lot of time and I have not noticed anything suspicious. Could you send me the log file to check?

Best
Andrey

[andr-kun]

Hi Andrey,

Thanks for getting back to me so quickly! I am running IsoQuant as part of the scnanoseq nextflow pipeline and one of their recommendation for processing PromethION data was to increase the CPU and RAM for IsoQuant. I did so following their recommendation, but I still noticed that the IsoQuant step was still taking some time to run, even with the pipeline splitting the aligned BAM, GTF and FASTA into individual chromosome (so IsoQuant only process a single chromosome) to parallelise the analysis. I was curious why IsoQuant was taking so long so I decided to ssh to the node running the job and found out that only a single thread was active.

I unfortunately do not have a log file handy at the moment, but there are some samples being processed by the pipeline currently, and I can extract the log file once the pipeline is finished.

Is it possible at all to parallelise the processing of the reads further (rather than just by chromosome) as some of the chromosome is taking 7+ hour to finish?

Kind regards,
Andrian

[sklages]

@andr-kun - you may also take a look at I/O . This may slow down processing significantly. Actually reducing the number of cores used may speed up things in the end (I reduced the cores from 40 to 24 and finally to 12 for a bunch of isoquant quant jobs reading/writing data from/to the same storage)..