0.20.1

Changes:

• c0a8d76 Update java/spark for version 0.20.1 (#465)
• 52221c8 Create new query after finalize in stats ingestion tasks (#464)

This list of changes was auto generated.

0.20.0

Changes:

• 9f93e75 Merge pull request #463 from TileDB-Inc/gspowley/sc-22800/skip-test-with-tiledb-py
• 31d620e Skip test using tiledb-py
• de8e663 Merge pull request #462 from TileDB-Inc/adamwenocur/sc-22758
• b903480 Update java/spark for version 0.19.0

This list of changes was auto generated.

0.19.2

Changes:

• 6dd03ca Merge pull request #460 from TileDB-Inc/gspowley/init-vfs-for-create
• 5b6b677 Update java/spark for version 0.19.2
• a3158b9 update ci to macos-11
• bbadde4 add comment
• ecd300f call init_htslib
• a95f1b0 initialize vfs in the create path

This list of changes was auto generated.

0.19.1

What's Changed

• Update superbuild to TileDB 2.11.3 by @Shelnutt2 in #459
• Check for an exact sample range match when resuming ingestion by @gspowley in #456
• Bump java/spark version for pending release of 0.19.1 by @Shelnutt2 in #458
• Wrap URIs with VFS scheme as necessary by @awenocur in #457

Full Changelog: 0.19.0...0.19.1

0.19.0

What's Changed

• Fix broken doc subsection link by @leipzig in #450
• Upgrade TileDB-VCF to 2.11 by @awenocur and @gspowley in #452

Full Changelog: 0.18.0...0.19.0

0.18.0

Changes:

• 973fb6a Update TileDB and java/spark for version 0.18.0 (#451)
• bc44a82 Merge pull request #447 from TileDB-Inc/awenocur/sc-19637/port-catch-search
• e089dfc Add optional ingestion tasks (#448)
• b3aa808 port FindCatch_EP from core
• f03ad02 Fix global order violation (#446)

Ingestion Tasks

This release adds optional ingestion tasks to efficiently compute allele count and allele frequency. Ingestion tasks create arrays containing useful statistics at ingestion time. This approach is very efficient since the ingestion process is already reading every VCF record.

The ingestion task must be enabled at array creation time.

Python

ds = tiledbvcf.Dataset(uri, mode="w")

# enable both ingestion tasks
ds.create_dataset(enable_allele_count=True, enable_variant_stats=True)

CLI

# enable both ingestion tasks
tiledbvcf create -u vcf.tdb --enable-allele-count --enable-variant-stats

# enable the allele_count ingestion task only
tiledbvcf create -u vcf.tdb --enable-allele-count

Ingestion task overview

• The allele_count task creates an array of called allele counts, for example:

         pos                    ref                                                alt filter   gt  count
0    5041703                     TA                                  T,TAAAAAAAAAAAAAA   PASS  1,2      1
1    5041703                     TA                                T,TAAAAAAAAAAAAAAAA   PASS  1,2      1
2    5041703                    TAA                                               T,TA   PASS  1,2      1
3    5046753                      G                                              C,GCC   PASS  1,2     11

• The variant_stats task creates an array used to compute allele frequencies, for example:

pos	       allele   ac          an          af
43036308       C        781         1026        0.761209
               T        245         1026        0.238791
43036324       A        1218        3708        0.328479
               G        2490        3708        0.671521

This list of changes was auto generated.

0.17.0

What's Changed

• Silence unused variable warning and bump TileDB version by @gspowley in #441
• Upgrade TileDB-VCF to 2.10 by @Shelnutt2 in #442

Full Changelog: 0.16.0...0.17.0

0.16.0

Changes:

• 08ff0cc Updates for version 0.16.0 (#439)
• 632cfdd Add RLE compression and dictionary encoding (#438)
• 754a441 Add support for the new TileDB group API (#437)

This list of changes was auto generated.

0.15.0

Changes:

• d3b614d Update java/spark for version 0.15.0 (#435)
• 066c979 Update resume ingestion (#434)
• ed993c6 consolidate commits and fragment metadata (#433)
• c6409b5 disable non-empty domain preload (#432)
• 8f8fefd write anchors in a separate fragment (#431)
• c59030b consolidate and vacuum commits (#430)
• c4823f4 Create variant stats array during ingestion (#426)

This list of changes was auto generated.

0.14.0

Changes:

• 8202f98 Update to TileDB 2.8.0 (#429)
• 528e92f Support VCF export in python (#414)
• 13f919b Remove lower bound limit for buffer sizes (#422)
• 1e196cf Remove error when incomplete query has 0 results; Add more native libraries to spark jar (#420)
• baef9b6 Merge pull request #417 from TileDB-Inc/ss/add-backport-bot
• 35fbfce Merge pull request #418 from TileDB-Inc/ss/tiledb-2.6.3
• 7112479 Update to TileDB 2.6.3
• b9c14fc Add backport bot for release branching support
• 043f09b Add option to ingest chromosomes and pseudo-contigs separately (#416)
• 955a55b Add default case to switch statement (#415)

See More

• 06cd4ee Update to TileDB-2.6.2
• 0b579d8 Adding Spark3 jars to releases
• 5f37696 Spark 3 support
• 679166b Combined VCF export feature (#413)

This list of changes was auto generated.