0.9.2

Patch release 0.9.2

• Fixes parsing of ellipse gates in FlowJo 10 workspace files (see #120)

0.9.1

Dependencies:

• FlowIO requirement updated to v1.0.1 (fixes issue exporting some FCS files)

API changes:

• new GatingStrategy method find_matching_gate_paths to find gate paths given a gate name
• new Session method find_matching_gate_paths & get_child_gate_ids

Bug fixes:

• Corrected error types (several instances of ValueError changed to TypeError)
• BUGFIX: Fixed issue where adding new gates to a Session where a WSP was imported would cause an error (issue #110)

Documentation:

• Several docstrings updated and fixed for ReadTheDocs
• Example notebooks moved to docs folder & added to ReadTheDocs

0.9.0

New features:

• Support for Python 3.10 (drops support for Python 3.6)
• All C extensions moved to FlowUtils (and FlowUtils now provides pre-built wheels - should avoid the need for a compiler for most users)
• New function extract_wsp_sample_data to parse FlowJo 10 wsp & extract Sample metadata (keywords) to a Python dictionary
• New GatingStrategy method get_max_depth to get max depth of gate tree
• New Session method summary for a quick summary of what the session contains
• Sample class now supports exporting new FCS files with metadata (#89)

API changes:

• Sample attribute channel_lin_log removed, lin / log metadata now available as "pne" column in Sample.channels attribute
• Sample class: export method has new kwarg include_metadata for including all key/value pairs in self.metadata in the output FCS file

Bug fixes:

• Fixed typo in FCS_STANDARD_KEYWORDS for 'date' keyword
• Fixed parsing of FlowJo workspaces where sample groups have "gate-less" samples (#91)
• Replace special quote characters that caused a UnicodeDecodeError on some Windows installs (#97)

Dependencies:

• FlowIO requirement updated to 1.0.0
• FlowUtils requirement updated to 1.0.0 (with pre-built binaries for most platforms)
• NumPy minimum requirement of 1.20 when using pre-built FlowUtils binaries (required for NumPy C API compatibility)
• Fix deprecation warnings from Pandas 1.4 (squeeze kwarg removed from read_csv)

Acknowledgements:

• Special thanks to @sbenz for help testing this release on arm64 Macs (Apple Silicon)

0.8.2

• Sample Class
  • Error is raised when retrieving compensated or transformed events prior to applying a compensation matrix or a transform (used to issue just a warning)
  • plot_scatter_matrix argument order changed to be more consistent with other plot methods
• Session Class
  • Analyzing multiple samples using multiprocessing now uses a more conservative estimate of memory to avoid running out of system memory
• BUGFIX: Fixed failure to properly parse FlowJo workspaces containing an ellipse gate.
• Documentation was updated to include a table of contents with links to methods for the complex classes: Sample, GatingStrategy, GatingResults, & Session

0.8.1

• GatingResults get methods now accept a gate_path argument to disambiguate re-used gates
• BUGFIX: fixed issue parsing FlowJo 10 wsp files with reused gates that contain a child gate (issue #84)
• Suppressed output for analyzing Session samples when verbose=False
• Suppress FlowKit generated warnings when running tests

0.8.0

New features:

• Support for Python 3.9
• FlowKit version now available via top-level attribute __version__
• GatingStrategy:
  • optimized processing order of gates to avoid re-processing
  • optional caching of pre-processed events to speed up processing of gates
• Matrix class now has an inverse method
• WSPBiexTransform now has an inverse method
• Significant speed increase for parsing FlowJo 10 WSP files with lots of bi-ex transforms
• Sample method plot_contour now has optional fill argument to created filled contours
• Sample now re-applies a transform when Sample is given a new compensation
• Analyzing samples in a Session using multiprocessing will now try to dynamically allocate CPUs based on available memory to avoid out of memory errors

Removed features:

• Anomalous event detection removed from FlowKit, now lives in separate FlowQC project

API changes:

• gates module:
  • All gate types now take a pandas DataFrame as input (possible to use gates independently now)
• Sample class:
  • subsample argument added back to Sample constructor (better to have subsample ready for plot methods)
  • Sample plot methods (except histogram and plot channel) now default to subsample=True
  • The various methods for getting NumPy arrays of events are now encapsulated into the single get_events methods that takes a source argument (consistent w/ other Sample methods)
  • Method get_channel_data renamed to get_channel_events
  • Added option to cache original events, they are no longer stored by default (saves memory)
  • Attribute anomalous_indices renamed to flagged_indices, and now set by user for any purpose using set_flagged_events method
  • Attribute channels changed from a dictionary to a more readable pandas DataFrame
  • New plotting method plot_channel for plotting channel data with "events" as x-axis (similar to plotting vs Time)
• GatingStrategy class:
  • gate_path is now always a tuple (not a list)
  • Uses of gate_id replaced with gate_name: a gate ID is a tuple of (gate_name, gate_path)
  • Method get_gate_indices renamed to get_gate_membership (since it returns a boolean array)
• Session class:
  • Plotting methods no longer analyze samples, analyze_samples method must be called prior
  • Method get_gate_indices renamed to get_gate_membership (since it returns a boolean array)
• Dimension class
  • Attribute label renamed to id
  • Method get_gate now returns the containing QuadrantGate when a Quadrant ID is given
• Function calculate_extent removed from public API (was never intended for external use)

Bug fixes:

• Improved support for Windows
  • multiprocessing now working
• fixed issue where parent gate results were not being properly retrieved, causing lots of re-processing
• fixed memory leak in points_in_polygon C extension (used for processing PolygonGate instances)
• fixed issue in C extension code that would sometimes cause mis-allocation of NumPy array sizes

Documentation:

• New & comprehensive tutorial notebook series for new FlowKit users
• New notebook demonstrating how to create a custom Sample gate in a Session
• More docstrings for classes and methods

Dependencies:

• FlowIO requirement updated to 0.9.12 (significantly faster reading of FCS files)
• FlowUtils requirement updated to 0.9.5
• new requirement networkx >= 2.3
• new requirement psutil ~= 5.8.0

0.7.0

New features:

• It's now easier to assign multiple samples to a Session group
  • Session.assign_samples can take either a Sample ID or list of Sample IDs
  • Session.add_samples now takes an optional group argument for assigning to a group when loading Samples
• New Session method get_wsp_gated_events for conveniently getting a list of DFs of compensated & transformed gated events
• More informative error message when reloading the same samples (include file name in error message)
• Transforms now support handling of 1-D data arrays

API changes:

• Session: renamed method assign_sample to assign_samples now that it supports a list of sample IDs
• Session methods get_sample_ids() & get_group_sample_ids() now have a loaded_only option to only return sample IDs of loaded samples
• Session: method get_gate now has a sample_id option allowing retrieval of custom sample gates in a sample group's gating strategy

Bug fixes:

• Eliminate Pandas PerformanceWarning when creating GatingResults instances (occurred when getting gate indices)
• Fixed issue where Session.export_wsp fails to export rectangle gates that include transforms
• Fixed issue where Session.get_group_sample_ids() incorrectly reports samples as members when that are not part of that group
• Test data and code are no longer included in packaged

Documentation:

• Showcase exporting to wsp in replicate flowjo example notebook
• More Session methods documented with docstrings
• Cleaned up examples folder so it is easier for users to browse
• New example notebooks for:
  • Demonstrating a comparison of clustering methods (Leiden vs Louvain)
  • Demonstrating dimension reduction on processed event data

Dependencies:

• FlowUtils requirement updated to 0.9.4
• NumPy requirement updated to >=1.19
• Seaborn requirement updated to >=0.11
• Pandas requirement updated to >=1.1

0.6.2

BUGFIX: Fixed case where some transforms fail when given a 1-D numpy array of events (issue #71)

Round and Round

New features:

• Experimental support for exporting FlowJo 10 workspace files (limited to single sample groups)
• Added support for FlowJo 10 biex transform
• Added inverse transform methods for most transformations
• implement ignore_transforms option for parsing FJ10 workspaces
• Sample.apply_transform supports customized transforms per channel via dictionary
• Create Sample instances from Pandas DataFrame w/MultiIndex columns
• Sample: store original events unprocessed

API changes:

• Sample: added attributes scatter_indices & time_index
• Sample: apply_transform method now has option to include scatter channels
• GatingStrategy: get_gate_ids now returns a list of tuples containing the gate ID & list of ancestors (gate path)
• GatingStrategy: new method get_root_gates for ...
• GatingStrategy: simplify GatingStrategy methods for retrieving gates,
• GatingStrategy: added new convenience method get_child_gates
• Session: new method get_group_comp_matrices
• Session: all method arguments referencing a sample group name are now consistent ('group_name')
• Session: new method to get all gate IDs & paths for a sample
• Session: new method get_group_transforms for retrieving group transformations
• Session: new method export_wsp for exporting a sample group's gating strategy as a FlowJo 10 workspace
• Session: new methods for retrieving Sample specific comp matrices & transforms for a group
• Session: new method for retrieving gated events for a Sample
• Compensation utility functions moved to FlowUtils
• make sample utils functions public
• expose calculate_extent function for flowkit-extras

Bug fixes:

• Clearer error message when requesting sub-sampled events from get_channel_data when Sample has not been sub-sampled
• Fixed issue where gain was incorrectly applied to time channel data
• Allow re-use of gates on different branches at any level
• Applying re-used gates requires the Gate instance to know the full path
• Fixed plot_scatter for cases where the event arrays are empty
• Session.get_group_samples() now only returns loaded samples

Documentation:

• Added more docstrings for Sample, Session, Transform methods
• Updated tutorial notebooks
• Added notebook for calculating mean fluorescence intensity (MFI) per channel for gated populations

Dependencies:

• Support for Python 3.8
• Removed MulticoreTSNE dependency
• Removed sklearn dependency
• Most transform implementations moved to FlowUtils

Branching Out

New features:

• GatingStrategy & Session classes now support re-using gate IDs in different gate hierarchy branches (i.e. having a child gate "IFNg+" under both a "CD4+" and "CD8+" gate)
• Added support for FlowJo 10 ellipsoid gates
• Plotting of gates in Session class now plots only events from the parent gate
• Added plotting of Quadrant gates
• Added bokeh hover tool to scatter plots
• Re-enabled multiprocessing by default for faster processing of gates in a Session
• Updated plot colormap for better contrast with gate boundaries

API changes:

• Major cleanup of public/private modules for cleaner API
• New Quadrant class for cleaner API when creating QuadrantGate instances
• New wsp_transforms module containing FlowJo specific Transform sub-classes
• GatingStrategy, Session - added get_transform and get_comp_matrix methods
• GatingStrategy, Session - added gate_path kwarg for ambiguous gates in get_gate & get_gate_indices
• Session: add sample_id option to analyze_samples to process a single Sample
• Session: added ignore_missing_files option to import_flowjo_workspace method for suppressing warnings about missing FCS files when importing a FlowJo WSP file
• Gate class - added get_dimension method
• Sample: rename Sample method to as_dataframe (consistent with Matrix class), & added options to re-order and rename columns in output DataFrame
• Sample: 'fcs' is no longer the default ID when adding a compensation matrix (it is reserved for the embedded comp matrix from the spill metadata value)
• Matrix: fluorochromes argument is now optional
• Matrix: new method as_dataframe to get comp matrix as a Pandas DataFrame
• Dimension: 'uncompensated' is now the default value when creating Dimension instances

Bug fixes:

• Properly support time channel data when timestep keyword is present in FCS metadata
• Fixed display of plotted ellipses (corrected angle)
• Fixed parsing of compensated dimension references in FlowJo WSP files
• Fixed import of CSV compensation files with commented header
• Fixed creation of Sample instances from Pandas DataFrame
• Fixed parsing of FlowJo gate IDs for some WSP variants

Documentation:

• Documentation now on ReadTheDocs
• Many more docstrings for classes and methods
• Added contribution guidelines
• New tutorial notebook to demonstrate loading of FlowJo workspaces & programmatically replicating the gating strategy
• New example notebook showing compensation and Matrix class usage