v17.0.0

Highlights

• Infrastructure Updates:

  • Deprecated Python 3.8 and added support for Python 3.10 and 3.11. Updated pandas, synapseclient, and Ubuntu versions.
  • Updated Java version to support latest annotator.jar.
  • Resolved version inconsistencies and cache-related issues in synapseclient affecting consortium and public releases.
  • Resolved datatype error for upgraded pandas.

• Data Processing Updates:

  • Unmasked pediatric patient data and year fields.

• Improved testing infrastructure:

  • Enabled dashboard generation for testing mode to catch issues earlier.

Data Processing Changes

• [GEN-1892] Unmask pediatric patient data (@danlu1 in #604, #611)

Technical Features

• [GEN-1789] [GEN-1775] Update to support Python 3.10, 3.11, newer pandas, synapseclient and Ubuntu 22.04 (@rxu17 in #559)
• [GEN-1262] Enable dashboard generation for testing mode (@danlu1 in #602)
• [GEN-1821] Provide Genie user-agent string to Synapse Python client for usage tracking (@danlu1 in #610)
• [GEN-2242] Update Java version to support newest annotator.jar (@rxu17 in #609)

Bug Fixes

• [GEN-2224] Fix dashboard bugs in consortium and public release steps (@rxu17 in #607)
• [GEN-2285] Resolve data type validation issue for mutation data (@danlu1 in #615)

Documentation Changes

• [GEN-2283] Improve docstring for validation function regarding text consistency (@rxu17 in #613)
• Update release procedure documentation (@rxu17 in #605)
• Update README for Python 3.10 updates (@rxu17 in #608)

Full Changelog: v16.6.0...v17.0.0

v16.6.0

Highlights

• Automated the updating of Genome Nexus truststore used in maf processing to prevent ALL failed annotations

  • This greatly reduces the likelihood of having to start the pipeline all over again (maf processing is the 1st step of the pipeline)

• Improving cBioportal integration

  • Added structural variant (SV) column to gene panel matrix file

Data Processing Changes

• [GEN-1332] Added structural variant (SV) column to gene panel matrix files by @danlu1 in #587
• [GEN-1954] Added Transcript_Exon as new additional MAF column by @rxu17 in #592

Technical Features

• [GEN-1616] Automated update of Genome Nexus truststore by @rxu17 in #588
• [GEN-2027] Enabled mutation in cis process for staging by @rxu17 in #597
• Updated truststore update cadence by @rxu17 in #600

Bug Fixes

• [GEN-1759] Fixed bug with deploy GitHub job by @rxu17 in #585
• [GEN-1953] Resolved issue with moving MAF table to less restrictive project by @danlu1 in #595
• [GEN-2001] Set synapse query timeout for MAF table upload by @rxu17 in #596
• [GEN-2094] Used customized _copyRecursive to enable copying files with access restrictions by @danlu1 in #598
• [GEN-2210] Fixed setting of skip_mutationsincis flag for staging by @rxu17 in #603

Other

• Updated release package guidelines by @rxu17 in #586
• Updated local validator README by @rxu17 in #589
• [DPE-1253] Added PR description template by @danlu1 in #590
• Added mermaid flowchart diagram for the input to database / database to staging steps by @thomasyu888 in #591
• [GEN-1969] Updated retraction policy by @danlu1 in #593

Full Changelog: v16.5.0...v16.6.0

v16.5.0

Highlights

• Improved tracking of validation status and error tracking

  • There is now a version column available for tracking down files that failed validation

• Improved testing infrastructure

  • Set up staging pipeline for testing changes
  • Added integration tests to GitHub Actions workflow
  • Fixed issue with building Docker image versions for package releases

Validation Rule Updates

• [GEN-1545] Removed GERMLINE from valid SV_STATUS values to prevent release of germline variants (@danlu1 in #579)
• [GEN-1021] Added validation to prevent release of samples with non-targeted sequencing or WXS library strategy values (@danlu1 in #580)

Data Processing Changes

• [GEN-1622] Removed sample class filter to enable release of cfDNA samples (@danlu1 in #581)
• [GEN-1704] Filtered out germline variants from SV files before release (@rxu17 in #583)

Technical Features

• [GEN-1330] Set up staging pipeline for testing changes (@rxu17 in #568)
• [GEN-703] Added version number to validation status and error tracking tables (@thomasyu888 in #574)
• [GEN-1653] Added integration tests to GitHub Actions workflow (@rxu17 in #582)

Bug Fixes

• [GEN-1442] Fixed issue with building Docker image versions for package releases (@rxu17 in #584)

Tech Debt

• [GEN-1325] Removed section on flagged mutations from dashboard summary (@thomasyu888 in #569)
• [GEN-1326] Removed panel overlap charts from dashboard summary (@thomasyu888 in #570)
• [GEN-1299] Aligned column names in dashboard (@thomasyu888 in #571)

Documentation Changes

• Added details about maf file validation (@rxu17 in #577)
• Added details about germline variant filtering during consortium release (@rxu17 in #578)

Full Changelog: v16.4.0...v16.5.0

v16.4.0

What's Changed

New Features

• [GEN-1164] add workflow to build docker image and push it to ghcr by @danlu1 in #556
• [GEN-868] add year death validation by @danlu1 in #560
• [GEN-867] Add validation rule to check if INT_DOD >= INT_CONTACT by @danlu1 in #561
• [GEN-1109] Setup base mkdocs for genie by @rxu17 in #566
• [GEN-1313] Export detailed columns for NAACCR codes by @danlu1 in #567
• [GEN-1348] Allow in mapping tables by @rxu17 in #572

Bug Fixes

• [GEN-1285] Remove row-based validation error message by @danlu1 in #563
• [GEN-1294] pin r-base version and upgrade to Node 20 by @danlu1 in #565
• [GEN-1285] Remove row based validation by @danlu1 in #564
• [GEN-1263] Fix bed file duplicates by @rxu17 in #562
• [GEN-1429] Add variant_classification as known string col by @rxu17 in #576

Documentation Changes

• [GEN-1237] Add documentation to maf allele validation by @rxu17 in #557
• Update docstring for cna's validateSymbol function by @rxu17 in #575

Full Changelog: v16.3.0...v16.4.0

v16.3.0

What's Changed

• [GEN-1163] Hotfix for annotation-tools by @rxu17 in #551
• [GEN-1191] Add file logging by @rxu17 in #552
• [GEN-1067] add functionality to warn for identical ref and tsa2 by @danlu1 in #553

New Contributors

• @danlu1 made their first contribution in #553

Full Changelog: v16.2.0...v16.3.0

v16.2.0

What's Changed

• [GEN-973] Fix bed file-clinical file cross-validation by @rxu17 in #540
• [GEN-977] update annotation-tools version by @rxu17 in #541
• Update docs by @rxu17 in #542
• [GEN-1026] Output annotation error report by @rxu17 in #544
• [GEN-1018] Update main.py and validate.py by @Chelsea-Na in #543
• [GEN-1078] HOTFIX: Add in new known string columns as MSK added new columns into maf file by @thomasyu888 in #547
• [GEN-1027] Store failed annotations error report by @rxu17 in #545
• [GEN-1076] Exclude genomic_location_explanation from release by @rxu17 in #548
• [GEN-1028] Generate failed annotations report dashboard by @rxu17 in #546
• [GEN-974] Allow NaN, nan and NA strings for mutation data by @rxu17 in #549
• [GEN-1065] Use OIDC with pypi in genie package publishing by @rxu17 in #550

New Contributors

• @Chelsea-Na made their first contribution in #543

Full Changelog: v16.1.0...v16.2.0

v16.1.0

What's Changed

• [GEN-851] Hotfix pin cbioportal repo version by @rxu17 in #535
• [GEN-846] & [GEN-845] Ignore case and allow underscores in cross-validate by @rxu17 in #536
• [GEN-834] Disable data guide generation by @rxu17 in #537
• [GEN-905] - Change error of start position > end position to a warning by @thomasyu888 in #538
• [GEN-809] Validate allele columns by @rxu17 in #539

Full Changelog: v16.0.0...v16.1.0

v16.0.0

What's Changed

• [GEN-81] Deprecate support for passwords, and authentication parameters in cli by @thomasyu888 in #518
• Add in pre-commit by @thomasyu888 in #517
• [GEN-560] Add ancillary files for cross-validation by @rxu17 in #520
• [GEN-562] Cross-validate maf with clinical sample files by @rxu17 in #522
• [GEN-561] Add cross validation to check for seq-assay-id bed file pair by @rxu17 in #523
• [GEN-706] Cross-validate clinical assay information by @rxu17 in #524
• Do not cross-file validate if ancillary files not present by @thomasyu888 in #526
• Add mypy pre-commit by @thomasyu888 in #521
• [GEN-498] Remove fusion files from public release by @rxu17 in #532
• [GEN-738] Add variant position validation for maf file by @rxu17 in #533
• [GEN-833] Remove support for python 3.7, pin synapseclient by @rxu17 in #534

Full Changelog: v15.4.0...v16.0.0

v15.4.0

What's Changed

• [GEN-694] Remove fusions from dashboarding by @thomasyu888 in #519
• [GEN-720] support new SV primary keys by @thomasyu888 in #527
• [GEN-756] Fix maf validation by @rxu17 in #530

Full Changelog: v15.3.0...v15.4.0

v15.3.0

What's Changed

• Dashboard code should support multiple major public releases, but always new the latest by @thomasyu888 in #513
• [GEN-493] Accept Unknown for SV dna and rna support columns by @rxu17 in #514
• [GEN-564] Add support for more values for site1 site2 col in SV file by @rxu17 in #515
• [GEN-636] Allow NAs/blanks for unrequired columns in SV by @rxu17 in #516
• [GEN-497] Remove fusion file from consortium release by @rxu17 in #505

Full Changelog: v15.2.0...v15.3.0