CSV Interface v4.3.1

Improvements

• Stream reader: smarter work with text files containing a mixture of line ending characters.
• Filter: faster execution.

Bug fixes

• Delimiter guesser: can't guess delimiters when forcing streams to return vbLf as EOL char.
• Dedupe: broken function when calling from CSVInterface class module.

CSV Interface v4.3.0

This version includes an important code refactoring of several members of the library, as well as some improvements that allow to outline the tool as a solid solution for data management.

Improvements

• Key related records: users can now use keys to segregate records as they are stored when the keyTree property is set to True. For example, under a key "fruits" you can store the records "lemon", "pear" and for a key "tools" you can store "hammer" and "chainsaw"; the user can request the segregated records at any time with the GetIndexedItem method by specifying a key.
• Faster deduplication: the execution time of the "Dedupe" method is 40% less.
• Use headers titles for computations: Filter, Reduce and InsertField methods now supports headers titles (also with white spaces) as variables.
• Calculated fields: the new InsertField method unleashes the possibility to use complex expressions to calculate values and insert them as a new field/column in the data tables. For example the following instruction adds a new field called "Taxes" in the ninth position which is defined as 18% of the total profit in each record, giving the VBA "Currency" format to the computation made: : .InsertField(8, "Taxes", True, "FORMAT(Total Revenue * PERCENT(18);'Currency')").
• Easy data storage: copy data from arrays of arraylist with the flexible and robust Add2 method and the items property. Define and share data from scratch with the FromString and ToString methods.

Members changes

CSVinterface

• InsertField now has a Formula parameter to allow insertion of calculated fields.

CSVarrayList

• Added: Dedupe, FromString, InsertField, MergeFields, RearrangeFields, RemoveField, ShiftField, ShiftRecord, SortByField, SortKeys, SplitField, and ToString methods.
• Added: isSorted, keyTree, lastSortedIndex properties.
• Optimized: Add2 and Dedupe methods, items property.
• Bug fixed: parser unable to properly get the order of the fields given by the user. Affected methods: Dedupe, LeftJoin, RightJoin, InnerJoin.
• Bug fixed: unexpected behavior. Affected method: SortByField

CSVexpression

• Improvement: more than 60 string, mathematical, logical, date-time and financial built-in functions
• Bug fixed: boolean negation prevents recognizing variables. Affected method: Eval.
• Bug fixed: function parsing was broken. Affected method: Create.

CSV Interface v4.2.1

Members changes

CSVarrayList

• Bug fixed: Joins did not retrieve headers.

CSV Interface v4.2.0

This new version includes many useful features that had not been covered and suggested by some enthusiasts (see this and this other subreddit post). Now VBA CSV Interface is shaping up to be a much more powerful tool when working with CSV files.

Improvements

• Like SQL joins: left, Right and Inner joins for tables, flavored for VBA users. CSV Interface becomes the only VBA library that supports this feature.
• Group method: this will allow users to accumulate with COUNT, MIN, MAX, SUM and AVG operations.
• Reduce method: to reduce records by executing functions over fields through records.
• Sorting large files: useful to allow sorting a CSV file on disk without overloading memory, currently this feature is very required but supported by few libraries.

Members changes

CSVinterface

• CSVsubsetSplit now accepts a list of fields to split on and output to a path ending in "-WorkDir".
• Added: SortOnDisk method.
• Added: BlockAutoFormat parameter to DumpToSheet method in order to prevent CSV injection.
• Added: AfterExportRecord, AfterImportRecord, AfterLoadStream,AfterSort and AfterWriteStream events.
• Bug fixed: Unable to detect opened workbooks.
• Bug fixed: Export procedure adding extra lines to output CSV files

CSVarrayList

• Added: error handling to Filter method.
• Added: LeftJoin, RightJoin, InnerJoin, Group, Reduce, AddIndexedItem, GetIndexedItem, RemoveIndexedItem, KeyExist and KeyIndex methods.
• Added: Indexing property.
• Optimized: RemoveRange, RemoveAt and Insert method.
• Bug fixed: Sort method.
• Deprecated: IntroSort sorting method.

CSVexpression

• Optimized: performance of evaluations on numerical expressions.
• Bug fixed: truncation of strings preventing assigment to string variables.

CSV Interface v4.1.3

Improvements:

• Added error handling to Filter function

CSV Interface v4.1.2

Bugs fixed:

• Infinite loop when intercepting operators in Filter method.

CSV Interface v4.1.1

Improvements:

• More advanced fitering capabilities. Expressions such as f1='Asia' & f9>20 & f9<=50 & f8 $ '10/*/2014' will use VBA's versatile LIKE operator ($) to perform powerful querie over CSV data.

Deprecated members

• Filter2 method: this work can be done with the Filter method.

CSV Interface v4.1.0

Improvements:

• Integration with VBAexpressions core modules.

Deprecated members

• GetCSVsubset method: this work can be done with the Filter method.

CSV Interface v4.0.0

Bugs fixed:

• The GetRecord method was not able to discard unwanted fields.
• DumpToSheet method cannot rename the attached sheet if it does not exist before.

Improvements:

• Insert and remove fields and records.
• Data filtering.
• Rearrange, merge, split fields.
• Shift fields and records.
• Dedupe CSV records.
• The user can now sort data based on multiple columns with the IntroSort, QuickSort, HeapSort and Merge algorithms via the Sort method in a intuitive way: use -1 to indicate a descending sort on column 1. Also data can be sorted by fields, Microssoft calls it left to right sort, that means that all records will be rearranged together with the record, typically header, chosen as key.
• Optimized DumpToSheet and ExportToCSV methods.
• Parser and writer accept Unix DSV files, e.g., \, will escape the , when used as a field delimiter.
• I/O operations on UTF-8 CSV files, commonly found on web sites, are now supported via streams.
• Refactoring: Added CSVSniffer class module.
• Refactoring: Added CSVdialect class module (field delimiter, record delimiter and escape token are managed in it and used in CSVparserConfig module).
• Refactoring: Added EscapeStyle enumeration.
• Refactoring: Added escapeMode property.
• Refactoring: Added SortingAlgorithms enumeration.
• Refactoring: Added the utf8EncodedFile property.
• The delimiter sniffer has a powerful mix of simple scoring due to the field data type and robust statistical scoring in order to check the uniformity of data in fields and records. Only one row of data can instruct the dialect, however, if the CSV file has headers the disambiguation rate increases by 298%! The sniffer now returns a CSVdialect object with the guessed delimiters.

Member changes

• Renamed method: GuessDelimiters --> SniffDelimiters
• Renamed method: CSVdatasetSplit --> CSVsubsetSplit
• Renamed property: turnStreamRecDelimiterToLF --> multiEndOfLineCSV
• Renamed property: rectangularResults --> uniformLengthRecords
• Renamed enumeration: EscapeTokens --> QuoteTokens
• Renamed class module: parserConfig --> CSVparserConfig
• Renamed class module: ECPArrayList --> CSVArrayList
• Renamed class module: ECPTextStream --> CSVTextStream

Deprecated members

• unixEscapeMechanism property.

Documentation:

• Added an extensive set of details for each module of the CSV interface class.

CSV Interface v3.1.5

Improvements:

• The delimiter guesser is more smarter, being now able to disambiguate cases where there are fields with embedded tables and/or CSV files with persistent records that could not be disambiguated with the table scoring method. In this new version, subtables are also scored. For example, the delimiter guesser can detect that the semicolon (;) is the field delimiter in the following CSV file (check Rainbow CSV issue #92):

1;Orange VUHF;K6CF AnhmHlls|K6COV Orng|K6MWT LkFrstSntg|K6NBR NwprtBch|K6QEH FllrtnRyth|K6SOA LgnBch|K6SOA SnClmnt|K6SOA TrbcCnyn|K6SYU FllrtnSt.J|KA6EEK IrvnSgnlP|KE6FUZ AnhmDsnyl|N6ME Fllrtn|N6SLD LkFrstSntg|W6HBR OrngPlsnts|W6KRW SnClmnt|W6KRW TstnLmRdg|W6VLD HntngtnBch|WA6FV FntnVlly|WA6YNT Plcnt|WB6HRO CstMsCtyH;145.1400|145.1600|145.2200|145.2400|145.2600|145.4000|145.4200|146.0250|146.2650|146.7900|146.8950|146.8950|146.9400|146.9700|147.0600|147.4350|147.4650|147.6450|147.8550|147.9150;144.5400|144.5600|144.6200|144.6400|144.6600|144.8000|144.8200|146.6250|146.8650|146.1900|146.2950|146.2950|146.3400|146.3700|147.6600|146.4000|146.5050|147.0450|147.2550|147.3150;OFF;OFF;OFF;;;OFF;;;Selected;0.5;0.5;0.1;0.1

• More delimiter guessing tests have been added.