Skip to content

CsvParseStream: negative fieldsPerRecord doesn't work with skipFirstRow or columns #6434

Open
@mfulton26

Description

@mfulton26

Describe the bug

According to fieldsPerRecord documentation, I should be able to parse CSV with variable length records. I can successfully do so but not when I use skipFirstRow and/or columns.

working example
import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
import { assertEquals } from "jsr:@std/[email protected]/equals";
import { assertType, IsExact } from "jsr:@std/[email protected]/types";

const source = ReadableStream.from([
  "name,age\n",
  "Alice,34\n",
  "Bob\n", // incomplete record
]);
const stream = source.pipeThrough(new CsvParseStream({ fieldsPerRecord: -1 }));
const result = await Array.fromAsync(stream);

assertEquals(result, [
  ["name", "age"],
  ["Alice", "34"],
  ["Bob"],
]);
assertType<IsExact<typeof result, string[][]>>(true);

Steps to Reproduce

  1. Create a script with the contents of any of the following examples

    not working with skipFirstRow
    import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
    import { assertEquals } from "jsr:@std/[email protected]/equals";
    import { assertType, IsExact } from "jsr:@std/[email protected]/types";
    
    const source = ReadableStream.from([
      "name,age\n",
      "Alice,34\n",
      "Bob\n", // incomplete record
    ]);
    const stream = source.pipeThrough(
      new CsvParseStream({
        fieldsPerRecord: -1,
        skipFirstRow: true,
      }),
    );
    const result = await Array.fromAsync(stream);
    
    assertEquals(result, [
      { name: "Alice", age: "34" },
      { name: "Bob", age: undefined },
    ]);
    assertType<IsExact<typeof result, Record<string, string | undefined>[]>>(true);
    not working with columns
    import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
    import { assertEquals } from "jsr:@std/[email protected]/equals";
    import { assertType, IsExact } from "jsr:@std/[email protected]/types";
    
    const source = ReadableStream.from([
      "Alice,34\n",
      "Bob\n", // incomplete record
    ]);
    const stream = source.pipeThrough(
      new CsvParseStream({
        fieldsPerRecord: -1,
        columns: ["name", "age"],
      }),
    );
    const result = await Array.fromAsync(stream);
    
    assertEquals(result, [
      { name: "Alice", age: "34" },
      { name: "Bob", age: undefined },
    ]);
    assertType<
      IsExact<typeof result, Record<"name" | "age", string | undefined>[]>
    >(true);
    not working with skipFirstRow and columns
    import { CsvParseStream } from "jsr:@std/[email protected]/parse-stream";
    import { assertEquals } from "jsr:@std/[email protected]/equals";
    import { assertType, IsExact } from "jsr:@std/[email protected]/types";
    
    const source = ReadableStream.from([
      "name,age\n",
      "Alice,34\n",
      "Bob\n", // incomplete record
    ]);
    const stream = source.pipeThrough(
      new CsvParseStream({
        fieldsPerRecord: -1,
        skipFirstRow: true,
        columns: ["name", "age"],
      }),
    );
    const result = await Array.fromAsync(stream);
    
    assertEquals(result, [
      { name: "Alice", age: "34" },
      { name: "Bob", age: undefined },
    ]);
    assertType<
      IsExact<typeof result, Record<"name" | "age", string | undefined>[]>
    >(true);
  2. Execute the script with deno run

  3. See error:

    error: Uncaught (in promise) Error: Syntax error on line 3: The record has 1 fields, but the header has 2 fields
        throw new Error(
              ^
        at convertRowToObject (https://jsr.io/@std/csv/1.0.5/_io.ts:233:11)
        at CsvParseStream.#pull (https://jsr.io/@std/csv/1.0.5/parse_stream.ts:460:28)

Expected behavior

  1. The scripts should run successfully.
  2. TypeScript type checking should also pass.
  3. Record<T, string | undefined> or Partial<Record<T, string>> should be used instead of Record<T, string>

Environment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions