Skip to content

Releases: AbsaOSS/cobrix

Minor bugfix release

04 Mar 12:25
92bc156
Compare
Choose a tag to compare
  • #474 Fix numeric decoder of unsigned DISPLAY format. The decoder made more strict and does not allow sign
    overpunching for unsigned numbers.
  • #477 Fixed NotSerializableException when using non-default logger implementations
    (Thanks @joaquin021).

Minor feature release

04 Feb 10:05
fae87b9
Compare
Choose a tag to compare
  • Improved schema flattening method SparkUtils.flattenSchema() for dataframes that have arrays. Array size metadata is used to determine maximum array elements, making it much faster for dataframes produced from mainframe files.
  • #324 Allow removing of FILLERs from AST when parsing using 'parseSimple()'. The signature of the method has
    changed. The boolean arguments now reflect more clearly what they do.
  • #466 Added maxElements and minElements to Spark schema metadata for
    array fields created from fields with OCCURS. This allows knowing the maximum number of elements in arrays when flattening the schema.

Minor bugfix release

11 Jan 07:27
172eee2
Compare
Choose a tag to compare

Minor bugfix release

22 Dec 09:28
816fad3
Compare
Choose a tag to compare
  • #451 Fixed COMP-9 (Cobrix extension for little-endian binary fields).

Minor bugfix release

16 Nov 07:31
d4fe309
Compare
Choose a tag to compare
  • #435 Fixed 'INDEXED BY' clause followed by multiple identifiers.
  • #437 Added support for '@' characters inside identifier names.

Minor feature release

26 Oct 10:40
2a0dbf0
Compare
Choose a tag to compare
  • #430 Added support for 'twisted' RDW headers when big-endian or little-endian RDWs use unexpected RDW bytes.

Minor feature release

23 Sep 09:07
ccdfc01
Compare
Choose a tag to compare
  • #420 Add experimental support for fixed blocked (FB) record format.
  • #422 Fixed decoding of 'broken pipe' (¦) character from EBCDIC.
  • #424 Fixed an ASCII reader corner case.

Feature Release

07 Sep 12:08
af950af
Compare
Choose a tag to compare
  • #412 Add support for variable block (VB aka VBVR) record format.
    Options to adjust BDW settings are added:
    • is_bdw_big_endian - specifies if BDW is big-endian (false by default)
    • bdw_adjustment - Specifies how the value of a BDW is different from the block payload. For example, if the side in BDW headers includes BDW record itself, use .option("bdw_adjustment", "-4").
    • Options is_record_sequence and is_xcom are deprecated. Use .option("record_format", "V") instead.
  • #417 Multisegment ASCII text files have now direct support using record_format = D.

Feature Release

03 Aug 08:27
528f9ec
Compare
Choose a tag to compare
  • #405 Fix extracting records that contain redefines of the top level GROUPs.
  • #406 Use 'collapse_root' retention policy by default. This is the breaking,
    change, to restore the original behavior add .option("schema_retention_policy", "keep_original").
  • #407 The layout positions summary generated by the parser now contains level
    numbers for root level GROUPs. This is a breaking change if you have unit tests that depend on the formatting of the layout
    positions output.

Minor feature release

15 Jul 11:48
5161b66
Compare
Choose a tag to compare
  • #397 Fix skipping of empty lines when reading ASCII files with is_record_sequence = true
  • #394 Added an ability to specify multiple paths to read data from (Use .option("paths", inputPaths.mkString(","))). This is a workaround implementation since adding support for multiple paths in load() would require a big rewrite for spark-cobol from data source to data format.
  • #372 Added an option to better handle null values in DISPLAY formatted data: .option("improved_null_detection", "false")