Make a module for multi-qc

[sergpolly]

Multi-qc is great!
Well supported , maintained and documented
https://multiqc.info/docs/#introduction-2

We just need to make multi-qc plugin for pairtools (would be part of pairtools) to make multi-qc understand our .stats -> make all kinds of beautiful and interactive plots and tables browseable along with e.g. fastqc report
It would end up in distiller afterwards of course...

@golobor @nvictus - have anyone been doing anything like that yet?

[sergpolly]

relevant:
https://github.com/4dn-dcic/pairsqc
open2c/distiller-nf#96
issue #68
issue #59

[sergpolly]

preliminary stuff:

[golobor]

whoa, this is so useful and pretty!

…

On Sat, 1 Feb 2020 at 20:37, Sergey Venev ***@***.***> wrote: preliminary stuff: [image: Screenshot from 2020-02-01 14-36-46] <https://user-images.githubusercontent.com/6790270/73597986-6d0b6000-4500-11ea-92d5-27ba9c23e0a6.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG64CRDQYNSIB72UKYCMHLRAXFRFANCNFSM4KM6NSOA> .

[sergpolly]

First draft is ready https://github.com/dekkerlab/MultiQC/tree/pairtools-module ...

Check out:

multiqc --outdir ~/blah --module pairtools /path/to/distiller-results

[sergpolly]

here is a clickable example:
http://ummsres37.ad.umassmed.edu:8080/mqc/multiqc_report.html

[sergpolly]

question regarding the types of pairs pairtools parse can generate:
is there a place somewhere where those are exhaustively enumerated, it is not obvious from the code ...
by looking at some "old"-ish distiller stats we have only NU NM MU there , are we always guaranteed to have those and not UN, MN, UM ?

I guess if one uses --no-flip option - than It is not guaranteed, otherwise it does - am I right ?
https://github.com/mirnylab/pairtools/blob/d1ddf9c39a336662f7fc725fa5a70ec68df9ba95/pairtools/pairtools_parse.py#L802

Should I account for X type of alignments ? XX pairs ?

We've also seen strange pair-type MR ? is it now the XR ? - i don't fully understand the meaning, but it's a separate question i guess

here is how the barchart "pairs by alignment status" looks at the moment:

[mimakaev]

There may be more types soon as Sasha finishes the walk rescuer...

[sergpolly]

ok - i'll try to make it more flexible than

[sergpolly]

here are the keys that I've included and assigned "nice" colors to ...
known_keys = ['UU', 'RU', 'UR', 'WW', 'DD', 'MR', 'MU', 'MM', 'NM', 'NU', 'NN', 'XX']

barchart would show them in the this order as well .

Any extra keys that are not form this list are going to be displayed after these ones, in a "random" order and with auto-coloring by MultiQC itself - i.e. it might look ugly at the end.

but before we submit everyhting I'd like to hear your input @golobor @mimakaev @agalitsyna - whoever it might concern - on the groupping of pairt-types, potentially missing categories, collapsing existing categories into 1 (UR+RU-> RU), XR vs MR still unclear to me, MN vs NM , etc

[golobor]

This looks awesome!!

…

-- the list of all possible pair types can be found here: https://pairtools.readthedocs.io/en/latest/formats.html#pair-types -- "MR" indeed can occur. Check the illustration at https://pairtools.readthedocs.io/en/latest/parsing.html#rescuing-single-ligations, "MR" would happen if the read alignment was a multimapper. -- "XX" is a corrupt pair. Currently, we apply this label to reads missing a pair, i.e. https://github.com/mirnylab/pairtools/blob/1579c6f1b3b3566ca95e10ae3a8fe6023408309c/tests/data/mock.sam#L56 -- collapsing UR+RU would be great! The only reason I kept them separate is that I wanted to retain information on which side was rescued. -- as long as pairs are flipped, we are guaranteed not to have UN, MN, UM.

On Tue, 3 Mar 2020 at 03:42, Sergey Venev ***@***.***> wrote: here are the keys that I've included and assigned "nice" colors to ... known_keys = ['UU', 'RU', 'UR', 'WW', 'DD', 'MR', 'MU', 'MM', 'NM', 'NU', 'NN', 'XX'] barchart would show them in the this order as well . Any extra keys that are not form this list are going to be displayed after these ones, in a "random" order and with auto-coloring by MultiQC itself - i.e. it might look ugly at the end. but before we submit everyhting I'd like to hear your input @golobor <https://github.com/golobor> @mimakaev <https://github.com/mimakaev> @agalitsyna <https://github.com/agalitsyna> - whoever it might concern - on the groupping of pairt-types, potentially missing categories, collapsing existing categories into 1 (UR+RU-> RU), XR vs MR still unclear to me, MN vs NM , etc — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG64CWWDMZ5CFCGQPIS5KLRFRVA3ANCNFSM4KM6NSOA> .

[sergpolly]

fix scalings-report section to use actual chromsizes instead of a fake 2_000_000 - which is happening now ...

[Phlya]

I didn't manage to say that during the talk, but do we actually need to know chromsizes? Knowing bins fixes the ratios of areas between them (doesn't it?), and in the end we can ensure area under curve equals 1. So we can fake areas to ensure the right shape of the curve, and then rescale it to get correct Y axis.

(assuming we ignore the last bin issue)