Skip to content

dummy PR for carrot testing **DO NOT MERGE** #7651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

dummy PR for carrot testing **DO NOT MERGE** #7651

wants to merge 2 commits into from

Conversation

jamesemery
Copy link
Collaborator

No description provided.

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "847e7d4f-7906-4e81-b34e-b5593784d0b0",
  "test_id": "acce36f4-84fb-4770-b82a-a758dbb79bc8",
  "name": "GATK HaplotypeCaller Test_run_2022-01-27 19:46:31.156656310 UTC",
  "status": "testfailed",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 10,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "Benchmark.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "Benchmark.evalLabel": "TESTSNAPSHOT2018HG002",
    "Benchmark.evalVcf": "test_output:VariantCalling.output_vcf",
    "Benchmark.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "Benchmark.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "Benchmark.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "Benchmark.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "Benchmark.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "Benchmark.referenceVersion": "HG38",
    "Benchmark.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "Benchmark.stratLabels": [
      "exome",
      "LCR"
    ],
    "Benchmark.truthLabel": "HG002_GRCh38_GIAB",
    "Benchmark.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "Benchmark.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "dbe17be2-3639-40ef-a107-b65193aee07a",
  "eval_cromwell_job_id": null,
  "created_at": "2022-01-27T19:46:31.156714",
  "created_by": null,
  "finished_at": "2022-01-28T08:58:14.576",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller Test_run_2022-01-31 15:46:11.708783346 UTC 
 
 {
  "run_id": "ca06f9ae-eecd-4168-9abe-50cd10e03db8",
  "test_id": "acce36f4-84fb-4770-b82a-a758dbb79bc8",
  "name": "GATK HaplotypeCaller Test_run_2022-01-31 15:46:11.708783346 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 10,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "Benchmark.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "Benchmark.evalLabel": "TESTSNAPSHOT2018HG002",
    "Benchmark.evalVcf": "test_output:VariantCalling.output_vcf",
    "Benchmark.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "Benchmark.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "Benchmark.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "Benchmark.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "Benchmark.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "Benchmark.referenceVersion": "HG38",
    "Benchmark.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "Benchmark.stratLabels": [
      "exome",
      "LCR"
    ],
    "Benchmark.truthLabel": "HG002_GRCh38_GIAB",
    "Benchmark.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "Benchmark.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-01-31T15:46:11.708929",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "ca06f9ae-eecd-4168-9abe-50cd10e03db8",
  "test_id": "acce36f4-84fb-4770-b82a-a758dbb79bc8",
  "name": "GATK HaplotypeCaller Test_run_2022-01-31 15:46:11.708783346 UTC",
  "status": "testfailed",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 10,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "Benchmark.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "Benchmark.evalLabel": "TESTSNAPSHOT2018HG002",
    "Benchmark.evalVcf": "test_output:VariantCalling.output_vcf",
    "Benchmark.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "Benchmark.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "Benchmark.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "Benchmark.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "Benchmark.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "Benchmark.referenceVersion": "HG38",
    "Benchmark.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "Benchmark.stratLabels": [
      "exome",
      "LCR"
    ],
    "Benchmark.truthLabel": "HG002_GRCh38_GIAB",
    "Benchmark.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "Benchmark.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "fa37ab73-c780-43b3-8799-9c9c33791403",
  "eval_cromwell_job_id": null,
  "created_at": "2022-01-31T15:46:11.708929",
  "created_by": null,
  "finished_at": "2022-02-01T05:22:24.782",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-02-01 19:09:10.650558739 UTC 
 
 {
  "run_id": "869dc901-0125-41f2-8977-19fbf51bdafd",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-01 19:09:10.650558739 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 10,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCalling.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCalling.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCalling.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-01T19:09:10.650676",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "869dc901-0125-41f2-8977-19fbf51bdafd",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-01 19:09:10.650558739 UTC",
  "status": "testfailed",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 10,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCalling.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCalling.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCalling.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "8e99e013-02e9-43f8-84e6-c2910a554eef",
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-01T19:09:10.650676",
  "created_by": null,
  "finished_at": "2022-02-02T06:43:19.685",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

1 similar comment
@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-02-23 19:59:31.084286796 UTC 
 
 {
  "run_id": "e296eb66-95b2-428a-a31e-3118047b197d",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-23 19:59:31.084286796 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://emeryj-testing/CarrotData/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://emeryj-testing/CarrotData/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://emeryj-testing/CarrotData/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://emeryj-testing/CarrotData/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://emeryj-testing/CarrotData/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://emeryj-testing/CarrotData/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCalling.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCalling.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCalling.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-23T19:59:31.084338",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "e296eb66-95b2-428a-a31e-3118047b197d",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-23 19:59:31.084286796 UTC",
  "status": "testfailed",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://emeryj-testing/CarrotData/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://emeryj-testing/CarrotData/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://emeryj-testing/CarrotData/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://emeryj-testing/CarrotData/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://emeryj-testing/CarrotData/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://emeryj-testing/CarrotData/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCalling.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCalling.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCalling.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "055d7bde-345f-43d4-9581-5f527b6fdbd1",
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-23T19:59:31.084338",
  "created_by": null,
  "finished_at": "2022-02-23T20:05:18.021",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-02-23 21:13:30.630066805 UTC 
 
 {
  "run_id": "d1c6dc68-9474-4803-89f3-7e22c52f2fd6",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-23 21:13:30.630066805 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCalling.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCalling.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCalling.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-23T21:13:30.630132",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "d1c6dc68-9474-4803-89f3-7e22c52f2fd6",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-23 21:13:30.630066805 UTC",
  "status": "carrotfailed",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCalling.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCalling.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCalling.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCalling.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "4d7d4a30-2d60-4093-939a-5c0eb6ea5486",
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-23T21:13:30.630132",
  "created_by": null,
  "finished_at": "2022-02-24T01:13:28.324278",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-02-24 15:51:30.766998897 UTC 
 
 {
  "run_id": "49a3a4ec-c92e-4181-b681-9dd431472165",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-24 15:51:30.766998897 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-24T15:51:30.767060",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "49a3a4ec-c92e-4181-b681-9dd431472165",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-24 15:51:30.766998897 UTC",
  "status": "evalfailed",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://gcp-public-data--broad-references/hg38/v0/exome_evaluation_regions.v1.interval_list",
      "gs://broad-dsp-spec-ops/scratch/mshand/EvaluationStratIntervalLists/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "6c6bbb52-5649-478b-8d95-e50381bcec5f",
  "eval_cromwell_job_id": "d56ac6eb-dab1-48e2-b080-bd5f86a0751b",
  "created_at": "2022-02-24T15:51:30.767060",
  "created_by": null,
  "finished_at": "2022-02-24T16:04:03.018",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-02-24 19:51:30.490690587 UTC 
 
 {
  "run_id": "88f12a77-2c28-4def-9ba3-1d88b80f037c",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-24 19:51:30.490690587 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_evaluation_regions.v1.interval_list",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-02-24T19:51:30.490746",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "88f12a77-2c28-4def-9ba3-1d88b80f037c",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-02-24 19:51:30.490690587 UTC",
  "status": "succeeded",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_evaluation_regions.v1.interval_list",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "test_cromwell_job_id": "d7213827-7fd0-4d4d-9a2d-332e6652f625",
  "eval_cromwell_job_id": "c2f81051-c34c-4cf1-a3f5-3f80b78959c2",
  "created_at": "2022-02-24T19:51:30.490746",
  "created_by": null,
  "finished_at": "2022-02-24T23:06:04.523",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-03-01 18:37:45.839345051 UTC 
 
 {
  "run_id": "202a0396-1aaa-4fc4-b59e-5f068894e566",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-03-01 18:37:45.839345051 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_evaluation_regions.v1.interval_list",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-01T18:37:45.839419",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "202a0396-1aaa-4fc4-b59e-5f068894e566",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-03-01 18:37:45.839345051 UTC",
  "status": "succeeded",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_evaluation_regions.v1.interval_list",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "eval_options": null,
  "test_cromwell_job_id": "f32b1f78-7094-4a8a-a0f1-fb107184a1c1",
  "eval_cromwell_job_id": "9f7e732d-f92f-4abd-a5e4-32f73e3ab388",
  "created_at": "2022-03-01T18:37:45.839419",
  "created_by": null,
  "finished_at": "2022-03-01T19:14:26.348",
  "results": {
    "control indel f1 score": "0.9903",
    "control indel precision score": "0.9906",
    "control indel recall score": "0.99",
    "control snp f1 score": "0.9901",
    "control snp precision score": "0.989",
    "control snp recall score": "0.9911",
    "indel f1 score": "0.9902",
    "indel precision score": "0.9903",
    "indel recall score": "0.9901",
    "snp f1 score": "0.9899",
    "snp precision score": "0.9887",
    "snp recall score": "0.9911"
  }
}

@jamesemery
Copy link
Collaborator Author

#carrot(GATK HaplotypeCaller vs Nightly Test, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : GATK HaplotypeCaller vs Nightly Test_run_2022-03-01 22:04:38.405022793 UTC 
 
 {
  "run_id": "074e8b3b-5149-4e7d-8dcd-0aca2643b625",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-03-01 22:04:38.405022793 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_evaluation_regions.v1.interval_list",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-01T22:04:38.405077",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "074e8b3b-5149-4e7d-8dcd-0aca2643b625",
  "test_id": "15973a95-055a-4540-9590-ce40f34411f5",
  "name": "GATK HaplotypeCaller vs Nightly Test_run_2022-03-01 22:04:38.405022793 UTC",
  "status": "succeeded",
  "test_input": {
    "VariantCallingCarrot.agg_preemptible_tries": 3,
    "VariantCallingCarrot.base_file_name": "NA24385",
    "VariantCallingCarrot.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrot.calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrot.contamination": 0.0383312,
    "VariantCallingCarrot.final_vcf_base_name": "NA24385",
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrot.haplotype_scatter_count": 50,
    "VariantCallingCarrot.input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrot.input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrot.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrot.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrot.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrot.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkComparison.confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkComparison.controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkComparison.controlVcf": "test_output:VariantCallingCarrot.control_vcf",
    "BenchmarkComparison.controlVcfIndex": "test_output:VariantCallingCarrot.control_vcf_index",
    "BenchmarkComparison.evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkComparison.evalVcf": "test_output:VariantCallingCarrot.output_vcf",
    "BenchmarkComparison.evalVcfIndex": "test_output:VariantCallingCarrot.output_vcf_index",
    "BenchmarkComparison.hapMap": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkComparison.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkComparison.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkComparison.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkComparison.referenceVersion": "HG38",
    "BenchmarkComparison.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_evaluation_regions.v1.interval_list",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCRFromHengHg38.interval_list"
    ],
    "BenchmarkComparison.stratLabels": [
      "exome",
      "LCR"
    ],
    "BenchmarkComparison.truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkComparison.truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkComparison.truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi"
  },
  "eval_options": null,
  "test_cromwell_job_id": "e28f2184-ba82-40b1-8b0a-5cf8ddb05449",
  "eval_cromwell_job_id": "2cf6650a-7089-4ef2-9b98-2224afbd3d9c",
  "created_at": "2022-03-01T22:04:38.405077",
  "created_by": null,
  "finished_at": "2022-03-02T04:46:20.806",
  "results": {
    "control indel f1 score": "0.9902",
    "control indel precision score": "0.9903",
    "control indel recall score": "0.9901",
    "control snp f1 score": "0.9899",
    "control snp precision score": "0.9887",
    "control snp recall score": "0.9911",
    "indel f1 score": "0.9902",
    "indel precision score": "0.9903",
    "indel recall score": "0.9901",
    "snp f1 score": "0.9899",
    "snp precision score": "0.9887",
    "snp recall score": "0.9911"
  }
}

@jamesemery
Copy link
Collaborator Author

#carrot(HaplotypeCaller CARROT Regression Tests, VariantCallingCarrot.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : HaplotypeCaller CARROT Regression Tests_run_2022-03-15 02:45:40.427499888 UTC 
 
 {
  "run_id": "7d96ffaf-77c1-440b-89c0-9cd08a352840",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-15 02:45:40.427499888 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-15T02:45:40.427752",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "7d96ffaf-77c1-440b-89c0-9cd08a352840",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-15 02:45:40.427499888 UTC",
  "status": "testfailed",
  "test_input": {
    "VariantCallingCarrot.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": "922f0be2-d538-47f7-991c-f036e6034792",
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-15T02:45:40.427752",
  "created_by": null,
  "finished_at": "2022-03-15T02:46:45.881",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(HaplotypeCaller CARROT Regression Tests, VariantCallingCarrotOrchestrated.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : HaplotypeCaller CARROT Regression Tests_run_2022-03-15 18:03:52.615116907 UTC 
 
 {
  "run_id": "05eb46d2-1bd0-4796-9df2-9a4d0fee745e",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-15 18:03:52.615116907 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-15T18:03:52.615170",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "05eb46d2-1bd0-4796-9df2-9a4d0fee745e",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-15 18:03:52.615116907 UTC",
  "status": "evalfailed",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": "d38de90b-8608-4043-b9e4-1fbc729ad064",
  "eval_cromwell_job_id": "89eec570-7bd0-4abb-8510-7ebc27f803c4",
  "created_at": "2022-03-15T18:03:52.615170",
  "created_by": null,
  "finished_at": "2022-03-16T02:53:33.530",
  "results": null
}

@jamesemery
Copy link
Collaborator Author

#carrot(HaplotypeCaller CARROT Regression Tests, VariantCallingCarrotOrchestrated.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : HaplotypeCaller CARROT Regression Tests_run_2022-03-16 14:20:46.087542573 UTC 
 
 {
  "run_id": "81cc98a3-f4cc-4e96-aa4a-493a718eb5a0",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-16 14:20:46.087542573 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-16T14:20:46.087600",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "81cc98a3-f4cc-4e96-aa4a-493a718eb5a0",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-16 14:20:46.087542573 UTC",
  "status": "succeeded",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": "5f0f8f34-cdc7-46ff-a59d-2368edcdf007",
  "eval_cromwell_job_id": "e6f57e40-2025-46fd-9aa0-d591a3799007",
  "created_at": "2022-03-16T14:20:46.087600",
  "created_by": null,
  "finished_at": "2022-03-16T17:21:08.639",
  "results": {
    "CHM controlHCprocesshours": "75.88741944444445",
    "CHM controlHCsystemhours": "0.1663777777777778",
    "CHM controlHCwallclockhours": "52.24009722222222",
    "CHM controlHCwallclockmax": "2.852152777777778",
    "CHM controlMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-CHMSampleHeadToHead/BenchmarkComparison/f65a7960-7b66-4a5d-a346-bd188a1b3830/call-CONTROLRuntimeTask/cacheCopy/monitoring.pdf",
    "CHM controlindelF1Score": "0.8724",
    "CHM controlindelPrecision": "0.8814",
    "CHM controlsnpF1Score": "0.9784",
    "CHM controlsnpPrecision": "0.9706",
    "CHM controlsnpRecall": "0.9863",
    "CHM controlsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-CHMSampleHeadToHead/BenchmarkComparison/f65a7960-7b66-4a5d-a346-bd188a1b3830/call-BenchmarkVCFControlSample/Benchmark/8d0e47ca-66f5-42a0-8785-6aa8d2db2663/call-CombineSummaries/summary.csv",
    "CHM evalHCprocesshours": "80.5165222222222",
    "CHM evalHCsystemhours": "0.1713305555555555",
    "CHM evalHCwallclockhours": "53.10978888888891",
    "CHM evalHCwallclockmax": "2.7458416666666667",
    "CHM evalMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-CHMSampleHeadToHead/BenchmarkComparison/f65a7960-7b66-4a5d-a346-bd188a1b3830/call-EVALRuntimeTask/cacheCopy/monitoring.pdf",
    "CHM evalindelF1Score": "0.8724",
    "CHM evalindelPrecision": "0.8814",
    "CHM evalsnpF1Score": "0.9784",
    "CHM evalsnpPrecision": "0.9706",
    "CHM evalsnpRecall": "0.9863",
    "CHM evalsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-CHMSampleHeadToHead/BenchmarkComparison/f65a7960-7b66-4a5d-a346-bd188a1b3830/call-BenchmarkVCFTestSample/Benchmark/96c96714-3ac6-4d2b-a79c-57086cda6373/call-CombineSummaries/summary.csv",
    "EXOME1 controlindelF1Score": "0.727",
    "EXOME1 controlindelPrecision": "0.632",
    "EXOME1 controlsnpF1Score": "0.9878",
    "EXOME1 controlsnpPrecision": "0.9815",
    "EXOME1 controlsnpRecall": "0.9941",
    "EXOME1 controlsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-EXOME1SampleHeadToHead/BenchmarkComparison/4803682b-a3c6-46d6-924b-dbc96a877e16/call-BenchmarkVCFControlSample/Benchmark/dd059ca4-251d-4793-bbff-10dd76123882/call-CombineSummaries/summary.csv",
    "EXOME1 evalindelF1Score": "0.727",
    "EXOME1 evalindelPrecision": "0.632",
    "EXOME1 evalsnpF1Score": "0.9878",
    "EXOME1 evalsnpPrecision": "0.9815",
    "EXOME1 evalsnpRecall": "0.9941",
    "EXOME1 evalsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-EXOME1SampleHeadToHead/BenchmarkComparison/4803682b-a3c6-46d6-924b-dbc96a877e16/call-BenchmarkVCFTestSample/Benchmark/e3563584-017d-476b-bbca-775128c80272/call-CombineSummaries/summary.csv",
    "NIST controlHCprocesshours": "90.94291388888888",
    "NIST controlHCsystemhours": "0.182125",
    "NIST controlHCwallclockhours": "63.56370277777778",
    "NIST controlHCwallclockmax": "3.701625",
    "NIST controlMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-NISTSampleHeadToHead/BenchmarkComparison/ccdb901c-fb8f-49e4-b542-cf42e011a623/call-CONTROLRuntimeTask/cacheCopy/monitoring.pdf",
    "NIST controlindelF1Score": "0.9902",
    "NIST controlindelPrecision": "0.9903",
    "NIST controlsnpF1Score": "0.9899",
    "NIST controlsnpPrecision": "0.9887",
    "NIST controlsnpRecall": "0.9911",
    "NIST controlsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-NISTSampleHeadToHead/BenchmarkComparison/ccdb901c-fb8f-49e4-b542-cf42e011a623/call-BenchmarkVCFControlSample/Benchmark/6d64f12a-ca50-4ecd-8608-93dc53d241bb/call-CombineSummaries/summary.csv",
    "NIST evalHCprocesshours": "95.62183055555556",
    "NIST evalHCsystemhours": "0.18361111111111117",
    "NIST evalHCwallclockhours": "64.22846111111112",
    "NIST evalHCwallclockmax": "3.3683277777777776",
    "NIST evalMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-NISTSampleHeadToHead/BenchmarkComparison/ccdb901c-fb8f-49e4-b542-cf42e011a623/call-EVALRuntimeTask/cacheCopy/monitoring.pdf",
    "NIST evalindelF1Score": "0.9902",
    "NIST evalindelPrecision": "0.9903",
    "NIST evalsnpF1Score": "0.9899",
    "NIST evalsnpPrecision": "0.9887",
    "NIST evalsnpRecall": "0.9911",
    "NIST evalsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/e6f57e40-2025-46fd-9aa0-d591a3799007/call-NISTSampleHeadToHead/BenchmarkComparison/ccdb901c-fb8f-49e4-b542-cf42e011a623/call-BenchmarkVCFTestSample/Benchmark/f0709402-e72d-4013-a781-e50d8d46e2c3/call-CombineSummaries/summary.csv"
  }
}

@jamesemery
Copy link
Collaborator Author

#carrot(HaplotypeCaller CARROT Regression Tests, VariantCallingCarrotOrchestrated.gatk_docker, )

1 similar comment
@jamesemery
Copy link
Collaborator Author

#carrot(HaplotypeCaller CARROT Regression Tests, VariantCallingCarrotOrchestrated.gatk_docker, )

@CarrotBroadBot
Copy link

Started a CARROT test run : HaplotypeCaller CARROT Regression Tests_run_2022-03-16 19:52:46.276710483 UTC 
 
 {
  "run_id": "50da448b-32dd-4f51-bf84-79b351ccb980",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-16 19:52:46.276710483 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-03-16T19:52:46.276978",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

CARROT test run finished 
 
 {
  "run_id": "50da448b-32dd-4f51-bf84-79b351ccb980",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-03-16 19:52:46.276710483 UTC",
  "status": "succeeded",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|ce0a811376d45c6ea62b40e78abffcd37c0317b4",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": "54997ade-421d-439f-acc9-abf50b3f9cb5",
  "eval_cromwell_job_id": "6ea2705f-a3fa-41fc-8d17-a2c55d875eab",
  "created_at": "2022-03-16T19:52:46.276978",
  "created_by": null,
  "finished_at": "2022-03-17T00:13:17.198",
  "results": {
    "CHM controlHCprocesshours": "75.88741944444445",
    "CHM controlHCsystemhours": "0.1663777777777778",
    "CHM controlHCwallclockhours": "52.24009722222222",
    "CHM controlHCwallclockmax": "2.852152777777778",
    "CHM controlMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-CHMSampleHeadToHead/BenchmarkComparison/1fb97a8b-caee-4184-8e36-be21e6c43549/call-CONTROLRuntimeTask/cacheCopy/monitoring.pdf",
    "CHM controlindelF1Score": "0.8724",
    "CHM controlindelPrecision": "0.8814",
    "CHM controlsnpF1Score": "0.9784",
    "CHM controlsnpPrecision": "0.9706",
    "CHM controlsnpRecall": "0.9863",
    "CHM controlsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-CHMSampleHeadToHead/BenchmarkComparison/1fb97a8b-caee-4184-8e36-be21e6c43549/call-BenchmarkVCFControlSample/Benchmark/3b068fb2-7140-4c1e-8860-df8df21821ec/call-CombineSummaries/summary.csv",
    "CHM evalHCprocesshours": "80.5165222222222",
    "CHM evalHCsystemhours": "0.1713305555555555",
    "CHM evalHCwallclockhours": "53.10978888888891",
    "CHM evalHCwallclockmax": "2.7458416666666667",
    "CHM evalMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-CHMSampleHeadToHead/BenchmarkComparison/1fb97a8b-caee-4184-8e36-be21e6c43549/call-EVALRuntimeTask/cacheCopy/monitoring.pdf",
    "CHM evalindelF1Score": "0.8724",
    "CHM evalindelPrecision": "0.8814",
    "CHM evalsnpF1Score": "0.9784",
    "CHM evalsnpPrecision": "0.9706",
    "CHM evalsnpRecall": "0.9863",
    "CHM evalsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-CHMSampleHeadToHead/BenchmarkComparison/1fb97a8b-caee-4184-8e36-be21e6c43549/call-BenchmarkVCFTestSample/Benchmark/7f7c4522-e293-4a03-ada8-9541a585250b/call-CombineSummaries/summary.csv",
    "EXOME1 controlindelF1Score": "0.727",
    "EXOME1 controlindelPrecision": "0.632",
    "EXOME1 controlsnpF1Score": "0.9878",
    "EXOME1 controlsnpPrecision": "0.9815",
    "EXOME1 controlsnpRecall": "0.9941",
    "EXOME1 controlsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-EXOME1SampleHeadToHead/BenchmarkComparison/3ba68beb-5853-4beb-b31c-cbef12825001/call-BenchmarkVCFControlSample/Benchmark/18840f82-6653-4365-8e02-daf8790ea4f0/call-CombineSummaries/summary.csv",
    "EXOME1 evalindelF1Score": "0.727",
    "EXOME1 evalindelPrecision": "0.632",
    "EXOME1 evalsnpF1Score": "0.9878",
    "EXOME1 evalsnpPrecision": "0.9815",
    "EXOME1 evalsnpRecall": "0.9941",
    "EXOME1 evalsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-EXOME1SampleHeadToHead/BenchmarkComparison/3ba68beb-5853-4beb-b31c-cbef12825001/call-BenchmarkVCFTestSample/Benchmark/194337cf-f57b-46fa-812c-e6510f51fd8d/call-CombineSummaries/summary.csv",
    "NIST controlHCprocesshours": "90.94291388888888",
    "NIST controlHCsystemhours": "0.182125",
    "NIST controlHCwallclockhours": "63.56370277777778",
    "NIST controlHCwallclockmax": "3.701625",
    "NIST controlMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-NISTSampleHeadToHead/BenchmarkComparison/a39481f5-0969-4891-a843-f3c3fd7437d1/call-CONTROLRuntimeTask/cacheCopy/monitoring.pdf",
    "NIST controlindelF1Score": "0.9902",
    "NIST controlindelPrecision": "0.9903",
    "NIST controlsnpF1Score": "0.9899",
    "NIST controlsnpPrecision": "0.9887",
    "NIST controlsnpRecall": "0.9911",
    "NIST controlsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-NISTSampleHeadToHead/BenchmarkComparison/a39481f5-0969-4891-a843-f3c3fd7437d1/call-BenchmarkVCFControlSample/Benchmark/0c99102a-bca1-4426-97c6-5a311ace93c1/call-CombineSummaries/summary.csv",
    "NIST evalHCprocesshours": "95.62183055555556",
    "NIST evalHCsystemhours": "0.18361111111111117",
    "NIST evalHCwallclockhours": "64.22846111111112",
    "NIST evalHCwallclockmax": "3.3683277777777776",
    "NIST evalMonitoringLogs": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-NISTSampleHeadToHead/BenchmarkComparison/a39481f5-0969-4891-a843-f3c3fd7437d1/call-EVALRuntimeTask/cacheCopy/monitoring.pdf",
    "NIST evalindelF1Score": "0.9902",
    "NIST evalindelPrecision": "0.9903",
    "NIST evalsnpF1Score": "0.9899",
    "NIST evalsnpPrecision": "0.9887",
    "NIST evalsnpRecall": "0.9911",
    "NIST evalsummary": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-NISTSampleHeadToHead/BenchmarkComparison/a39481f5-0969-4891-a843-f3c3fd7437d1/call-BenchmarkVCFTestSample/Benchmark/a3925c8a-7e0a-4fec-8507-f885061b69c3/call-CombineSummaries/summary.csv",
    "ROC_Plots_Reported": "gs://dsde-methods-carrot-prod-cromwell/BenchmarkVCFsHeadToHeadOrchestrated/6ea2705f-a3fa-41fc-8d17-a2c55d875eab/call-CreateHTMLReport/cacheCopy/report.html"
  }
}

@jamesemery jamesemery force-pushed the je_testCarrotInfrastructure branch from ce0a811 to 9b1a9c1 Compare June 14, 2022 15:53
@jamesemery
Copy link
Collaborator Author

#carrot(HaplotypeCaller CARROT Regression Tests, VariantCallingCarrotOrchestrated.gatk_docker, )

@CarrotBroadBot
Copy link

🥕CARROT🥕 run started

Test: HaplotypeCaller CARROT Regression Tests | Status: building

Run: HaplotypeCaller CARROT Regression Tests_run_2022-06-14 15:54:21.685287323 UTC

Full details 
 
 {
  "run_id": "f41c307f-57fd-4cd5-8c35-38b1fab53118",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-06-14 15:54:21.685287323 UTC",
  "status": "building",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|9b1a9c1a631809800debcd83466556397a54e798",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-06-14T15:54:21.685372",
  "created_by": null,
  "finished_at": null
}

@CarrotBroadBot
Copy link

🥕CARROT🥕 run finished

Test: HaplotypeCaller CARROT Regression Tests | Status: build_failed

Run: HaplotypeCaller CARROT Regression Tests_run_2022-06-14 15:54:21.685287323 UTC

Full details 
 
 {
  "run_id": "f41c307f-57fd-4cd5-8c35-38b1fab53118",
  "test_id": "c3de522b-7ce5-4a51-8b57-1cea628dd93a",
  "name": "HaplotypeCaller CARROT Regression Tests_run_2022-06-14 15:54:21.685287323 UTC",
  "status": "buildfailed",
  "test_input": {
    "VariantCallingCarrotOrchestrated.CHM_base_file_name": "CHM113",
    "VariantCallingCarrotOrchestrated.CHM_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.CHM_contamination": 0.0,
    "VariantCallingCarrotOrchestrated.CHM_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm1_chm13_hiseqx_sm_hf3mo.bam",
    "VariantCallingCarrotOrchestrated.CHM_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.NIST_base_file_name": "NA24385",
    "VariantCallingCarrotOrchestrated.NIST_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/wgs_calling_regions.hg38.interval_list",
    "VariantCallingCarrotOrchestrated.NIST_contamination": 0.0383312,
    "VariantCallingCarrotOrchestrated.NIST_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bam",
    "VariantCallingCarrotOrchestrated.NIST_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA24385_NA24385_O1D1_SM-G947H_v1_NS.bai",
    "VariantCallingCarrotOrchestrated.agg_preemptible_tries": 3,
    "VariantCallingCarrotOrchestrated.break_bands_at_multiples_of": 100000,
    "VariantCallingCarrotOrchestrated.contamination": 0.0,
    "VariantCallingCarrotOrchestrated.exome1_base_file_name": "NA12878Exome1",
    "VariantCallingCarrotOrchestrated.exome1_calling_interval_list": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/exome_calling_regions.v1.interval_list",
    "VariantCallingCarrotOrchestrated.exome1_input_bam": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram",
    "VariantCallingCarrotOrchestrated.exome1_input_bam_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/NA12878_forPCRplus_1.cram.crai",
    "VariantCallingCarrotOrchestrated.gatk_control_docker": "broadinstitute/gatk-nightly:2022-03-04-4.2.5.0-9-gb097f75c5-NIGHTLY-SNAPSHOT",
    "VariantCallingCarrotOrchestrated.gatk_docker": "image_build:gatk|9b1a9c1a631809800debcd83466556397a54e798",
    "VariantCallingCarrotOrchestrated.haplotype_scatter_count": 50,
    "VariantCallingCarrotOrchestrated.monitoring_script": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/cromwell_monitoring_script.sh",
    "VariantCallingCarrotOrchestrated.ref_dict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "VariantCallingCarrotOrchestrated.ref_fasta": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "VariantCallingCarrotOrchestrated.ref_fasta_index": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "VariantCallingCarrotOrchestrated.use_gatk3_haplotype_caller": true
  },
  "test_options": null,
  "eval_input": {
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.CHM_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.CHM_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcf": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.CHM_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthLabel": "CHM_GRCh38_SYNDIPv20180222",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.CHM_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/chm.full.m38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.twist_exome.interval_list",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.EXOME1_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcf": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.EXOME1_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthLabel": "NA12878_GRCh38_TWISTExome",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.EXOME1_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/GIAB_v3.3.2_NA12878_hg38.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_confidenceInterval": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark_noinconsistent.bed",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlLabel": "CONTROLSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_control_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_control_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_controlVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_control_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalLabel": "TESTSNAPSHOT2018HG002",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalMonitoringExample": "test_output:VariantCallingCarrotOrchestrated.NIST_representative_benchmarking",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalRuntimeSummaries": "test_output:VariantCallingCarrotOrchestrated.NIST_output_runtimes",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcf": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_evalVcfIndex": "test_output:VariantCallingCarrotOrchestrated.NIST_output_vcf_index",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthLabel": "HG002_GRCh38_GIAB",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcf": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz",
    "BenchmarkVCFsHeadToHeadOrchestrated.NIST_truthVcfIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HG002_GRCh38_GIAB_1_22_v4.2.1_benchmark.broad-header.vcf.gz.tbi",
    "BenchmarkVCFsHeadToHeadOrchestrated.hapMap": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.haplotype_database.txt",
    "BenchmarkVCFsHeadToHeadOrchestrated.refDict": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.dict",
    "BenchmarkVCFsHeadToHeadOrchestrated.refIndex": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta.fai",
    "BenchmarkVCFsHeadToHeadOrchestrated.reference": "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/Homo_sapiens_assembly38.fasta",
    "BenchmarkVCFsHeadToHeadOrchestrated.referenceVersion": "HG38",
    "BenchmarkVCFsHeadToHeadOrchestrated.stratIntervals": [
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/HCR_hg38.bed",
      "gs://dsp-methods-carrot-data/test_data/haplotypecaller_tests/LCR_Hg38.interval_list"
    ],
    "BenchmarkVCFsHeadToHeadOrchestrated.stratLabels": [
      "HCR",
      "LCR"
    ]
  },
  "eval_options": null,
  "test_cromwell_job_id": null,
  "eval_cromwell_job_id": null,
  "created_at": "2022-06-14T15:54:21.685372",
  "created_by": null,
  "finished_at": "2022-06-14T16:26:19.183470",
  "results": null,
  "errors": null
}

@codecov
Copy link

codecov bot commented Jun 14, 2022 •
edited
Loading

Codecov Report

Merging #7651 (d22b752) into master (d22b752) will not change coverage.
The diff coverage is n/a.

❗ Current head d22b752 differs from pull request most recent head 663214b. Consider uploading reports for the commit 663214b to get more accurate results

@@             Coverage Diff             @@
##              master     #7651   +/-   ##
===========================================
  Coverage     86.933%   86.933%           
  Complexity     36950     36950           
===========================================
  Files           2221      2221           
  Lines         173833    173833           
  Branches       18778     18778           
===========================================
  Hits          151118    151118           
  Misses         16083     16083           
  Partials        6632      6632

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jamesemery @CarrotBroadBot