- added support for block comments using /* ... */ and /+ ... +/ - anything between these will get removed (new method: remove_comments)
- added all_plugins() method to ClassRegistry class
- load_user_defined_placeholders now stores the placeholders correctly (fixed incorrect late lambda binding)
- placeholders got rebranded as variables (seppl.placeholders -> seppl.variables)
- added @abc.abstractmethod decorator where appropriate
- added stopped flag to Session to indicate that the execution is over, which is monitored by the seppl.io.execute(...) function
- added wai_logging>=0.0.5 as dependency
- added constants for meta-data types: METADATA_TYPE_STRING, METADATA_TYPE_BOOL, METADATA_TYPE_NUMERIC
- added load_args and save_args methods for loading args from/saving args to files
- the execute(...) method now queries the reader whether it has finished after the first read/yield to allow for dynamically locating files during first call to read() method after initializing the reader
- the placeholders() function (package: seppl.placeholders) now returns the non-input-based ones when outputting input-based ones
- added DataCollector writer, which just collects all the data and makes it accessible
- introduced StreamFilter filters to allow for 1-to-m processing (with m>=0)
- the MultiFilter is now a StreamFilter
- added filter_data generator function and FilterPipelineIterator iterator class for efficiently process data
- the classes_to_str and get_class_name methods can remove builtins. via the clean=True parameter now
- introduced the InifiniteReader mixin which automatically disables batch mode
- added support for caching plugins managed by the ClassRegistry via the ClassCache class (one for each class hierarchy)
- the execute method now supports custom pre-initialization and post-finalization method hooks
- filters output the object ID before processing the incoming data when logging level is set to DEBUG
- the ClassRegistry now supports class listers that list class names (not modules!) to be ignored, i.e., ones that should be returned for their class hierarchies; useful when excluding inherited plugins that are not applicable in the concrete application
- the is_help_requested method can now pinpoint whether global help or help for a specific plugin was requested (requires supplying a list of valid handler names and whether to use partial matching)
- the args_to_objects method now resolves the plugin name and throws an error if it is flagged as unknown
- the write methods of DirectStreamWriter and DirectBatchWriter now have the additional as_bytes parameter, to indicate whether whether to write as bytes or str
- formalized support for direct read from/write to file-like objects with the DirectReader, DirectStreamWriter, DirectBatchWriter mixins
- replaced deprecated pkg_resources with importlib.metadata-based approach (based on: https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/#using-package-metadata)
- filters and writers can be skipped now via the --skip flag, making it easy for external scripts to enable/disable pipeline components
- added support to the seppl.io.Splitter class for keeping item/sample groups together via a split_group regular expression
- backported helper methods for seppl.io.Writer classes for managing splitting
- added resume_from parameter to seppl.io.locate_files method which allows to skip all files preceding this glob
- the resolve_handler and split_args methods now have the partial boolean parameter which determines whether partial matches are accepted or not; off by default as it can interfere with parameters from plugins
- moved placeholder functionality from seppl to seppl.placeholders
- load_user_defined_placeholders now ignores lines that start with #
- added support for placeholders, which can be expanded via the Session object
- plugins supporting placeholders should import the PlaceholderSupporter indicator mixin for automatically adding help on placeholders to the help screen; plugins that support placeholders based on the current input should import the InputBasedPlaceholderSupporter indicator mixin
- placeholder-supporting plugins can use the placeholder_list method in their argparse options
- the load_user_defined_placeholders method allows incorporating custom placeholders for directories
- added alias support to the ClassRegistry class
- added method is_alias(...) and property all_aliases to the Registry and ClassRegistry classes
- extended the enumerate_plugins method to allow flagging of aliases (default: *)
- added support for using partial handler/plugin names (as long as they are unique)
- added experimental support for aliases with AliasSupporter mixin
- added setuptools as dependency
- the seppl.io.locate_files method can support recursive globs now (default is no)
- reworked the execute method, properly distinguishing between stream/batch mode now
- the seppl.io.locate_files method can take a default glob now, which gets appended to inputs that point to directories
- reworked excluding of classes
- _determine_from_entry_points method of ClassListerRegistry class now checks whether there the attributes tuple has any elements (i.e., whether the optional :function_name was provided)
- message X records processed in total now only output at the end
- ClassListerRegistry now safely removes any excluded class listers before locating the classes
- ClassListerRegistry now removes any excluded class listers before locating the classes
- the execute method no longer counts None items returned by the reader
- added the seppl.ClassListerRegistry class that offers a more convenient way of discovering classes via a function that returns a dictionary of superclasses and the associated modules to inspect; with this approach only a single entry_point has to be defined in setup.py, pointing to the class lister module/function
- added the dummy type AnyData which is used by default in the check_compatibility method for a match all (ie can be used for general purpose plugins)
- added methods escape_args and unescape_args (and corresponding command-line tools seppl-escape and seppl-unescape) for escaping/unescaping unicode characters in command-lines to make them copyable across ssh sessions
- check_compatibility method now also checks whether generated class is subclass of accepted classes, to allow for broader accepts() methods
- gcd method now creates a copy of the integer ratio list before processing it
- added basic support for meta-data: MetaDataHandler, get_metadata, add_metadata
- added support for splitting sequences using supplied (int) split ratios
- added session support: Session, SessionHandler
- added I/O super classes: Reader, Writer, StreamWriter, BatchWriter, Filter, MultiFilter
- added support for executing I/O pipelines: Reader, [Filter...], [Writer]
- the DEFAULT placeholder in the environment variable listing the modules now gets expanded to the default modules, making it easier to specify modules in derived projects
- added excluded_modules and excluded_env_modules to Registry class initializer to allow user to specify modules (explicit list or list from env variable) to be excluded from being registered; useful when outputting help for derived modules that shouldn't output all the base plugins as well.
- the registry now inspects modules when environment modules are present even when it already found plugins (eg default ones)
- the registry now inspects modules when custom modules were supplied even when it already found plugins (eg default ones)
- suppressing help output for unknown args now
- Plugin.parse_args now returns any unparsed arguments that were found
- the args_to_objects method now raises an Exception by default when unknown arguments are encountered for a plugin (can be controlled with the allow_unknown_args parameter)
- enforcement of uniqueness is now checking whether the class names differ before raising an exception.
- added OutputProducer and InputConsumer mixins that can be use for checking the compatibility between pipeline components using the check_compatibility function.
- added support for dynamic mode which only requires listing the superclass of a plugin and the module in which to look for these plugins (slower, but more convenient)
- added generate_entry_points helper method to easily generate the entry_points section for plugins, rather than manually maintaining it
- added generate_help and generate_plugin_usage methods for generating documentation for plugins
- removed old, logging-related code from Plugin class
- added args_to_objects to quickly instantiate plugins from parsed arguments
- added example to README.md and example library (https://github.com/waikato-datamining/seppl-example)
- initial release