Resampling p-values and Multiplicity correction

[elikling]

Hi,
I have been following the DoWhy with much interest and have used it a little. It inspired me to start writting a paper (I hope to submit) about how to co-design a causlity graph with an SME while informing the dicussion using multiplicty adjusted p-values (which could be obtained via resampling or theoretically). I do find it easier to discuss that in the context of SCMs. - i.e. all continous variables.

It has also caused me to try and formulate what should be comunicated on the arcs and a gave it a go in graphviz - the look it not amazing. I agree with a comment I saw in your summary that the graphical representation needs good attention.

Will you be:

1. Interested in co-authering the paper (I have alreadya 16 page draft including a toy expample ;-) )
2. Colaborate on introducing the approach into the packde or guild me on how to do so (my knoledge of Python is probably not up to scratch)

https://www.linkedin.com/in/elikling/

Eli Kling

[emrekiciman]

Hi Eli, Thanks for getting in touch. Indeed, co-designing a causality graph with a subject matter expert is an interesting process. Best practices on this would be welcome. I would be curious to learn more about what you have learned through your experiences. Regarding a paper / package inclusion --- I am not sure what might be the right way to go about this. It probably depends on the specifics of your approach. Maybe it as simple as linking to a blog post or tutorial where you describe lessons you have learned? Or, what did you have in mind? Thanks, Emre From: Eli Y. Kling ***@***.***> Sent: Wednesday, June 8, 2022 7:54 AM To: py-why/dowhy ***@***.***> Cc: Subscribed ***@***.***> Subject: [py-why/dowhy] Resampling p-values and Multiplicity correction (Discussion #457) Hi, I have been following the DoWhy with much interest and have used it a little. It inspired me to start writting a paper (I hope to submit) about how to co-design a causlity graph with an SME while informing the dicussion using multiplicty adjusted p-values (which could be obtained via resampling or theoretically). I do find it easier to discuss that in the context of SCMs. - i.e. all continous variables. It has also caused me to try and formulate what should be comunicated on the arcs and a gave it a go in graphviz - the look it not amazing. I agree with a comment I saw in your summary that the graphical representation needs good attention. Will you be: 1. Interested in co-authering the paper (I have alreadya 16 page draft including a toy expample ;-) ) 2. Colaborate on introducing the approach into the packde or guild me on how to do so (my knoledge of Python is probably not up to scratch) https://www.linkedin.com/in/elikling/<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Felikling%2F&data=05%7C01%7Cemrek%40microsoft.com%7C28272b9d671043121a6508da495eb6a8%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637902968390615665%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=b8tfi2NqkwzDivA5RyyENRUB8BlVpnxRskw6HmKVyX4%3D&reserved=0> Eli Kling - Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpy-why%2Fdowhy%2Fdiscussions%2F457&data=05%7C01%7Cemrek%40microsoft.com%7C28272b9d671043121a6508da495eb6a8%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637902968390615665%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=c4IQwoiCCjggICn2NpmIAhoXsUmoiLl5ItohkKXRwGA%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABNUPUFT7KJVHJQDPVL6IP3VOCXYJANCNFSM5YG25GBA&data=05%7C01%7Cemrek%40microsoft.com%7C28272b9d671043121a6508da495eb6a8%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637902968390615665%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VlCiYRNZCUl8KZpZ4m859Mb0GxAlbfTisxXISwH9epc%3D&reserved=0>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.******@***.***>>

[elikling]

Thanks for getting back to me :-)
Actually I have two things in mind that boild down to looging for a thought partenr:

1. Finding a co-auther for the paper - I would like to try and submit it to the Journal of the Operatiosn Research Society.
2. Figure out how to wrap my apprach around (or within) DoWhy - it requires obtaining p-values, correcting for maltiplicty and comunicatin g in the right way - I will spend some time trying to develop this and if I have something I will put it in a separate pull request. - again - it would be nice to colaborate with someoen - I always work better in co-creation settings (not delegation ;-) )

Eli [https://www.linkedin.com/in/elikling/]

[bloebp]

Hey, I saw you were referring to https://github.com/py-why/dowhy/blob/master/dowhy/gcm/stats.py#L13 once. Lets have a chat on discord, would be curious to hear what you have in mind.

[emrekiciman]

Hi Eli, For #1: I'd recommend searching for a co-author who is familiar with your target journal. Unfortunately, I don't have experience with OR. For #2: I'd recommend posting a request for collaboration on our discord community channel. Thanks, Emre From: Eli Y. Kling ***@***.***> Sent: Friday, June 10, 2022 2:54 AM To: py-why/dowhy ***@***.***> Cc: Emre Kiciman ***@***.***>; Comment ***@***.***> Subject: Re: [py-why/dowhy] Resampling p-values and Multiplicity correction (Discussion #457) Thanks for getting back to me :-) Actually I have two things in mind that boild down to looging for a thought partenr: 1. Finding a co-auther for the paper - I would like to try and submit it to the Journal of the Operatiosn Research Society. 2. Figure out how to wrap my apprach around (or within) DoWhy - it requires obtaining p-values, correcting for maltiplicty and comunicatin g in the right way - I will spend some time trying to develop this and if I have something I will put it in a separate pull request. - again - it would be nice to colaborate with someoen - I always work better in co-creation settings (not delegation ;-) ) Eli [https://www.linkedin.com/in/elikling/] - Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpy-why%2Fdowhy%2Fdiscussions%2F457%23discussioncomment-2921484&data=05%7C01%7Cemrek%40microsoft.com%7C80e6637ede804b19bbe308da4ac73311%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637904516661422832%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FIUHbUSxkN1mSnHEpo4euEkvEdPXTlO64%2B%2FO3x8tWrE%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABNUPUCIWPAAOLZE7D34GRTVOMGE7ANCNFSM5YG25GBA&data=05%7C01%7Cemrek%40microsoft.com%7C80e6637ede804b19bbe308da4ac73311%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637904516661422832%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rtdKyb%2FApfgS0kitxCGCeW2wrG%2FoNgxyMhTX7RSv1YM%3D&reserved=0>. You are receiving this because you commented.Message ID: ***@***.******@***.***>>