🐛 [firestore-bigquery-export] Documentation for Import existing documents (backfill) is misleading

[pedro-iatzky]

Describe your configuration

• Extension name: firestore-bigquery-export
• Extension version: 0.1.56

Describe the problem

The Readme docs say explicitly that exists an option to backfill the documents when installing the extension

But this option is actually not available. It seems that it has been firstly commented out in this commit and then finally removed in this other

Steps to reproduce:

Review the yaml config and you won't find any DO_BACKFILL option but check the readme and you will see the option is mentioned there

[0x80]

I was wondering about this as well.

I ended up installing @firebaseextensions/fs-bq-import-collection as a dev dependency, and wrapped the command line tool in a function so that I only have to pass the collection name.

Of course you will need to change the various flags to suit your own needs.

I trigger this function from the command line with my own CLI script runner.

I think it would be nice if the Firebase team exposed the run function directly from the package as well, and allow you to pass the arguments as an options object.

Also, currently it is a little funky that you have to specify all options, even the ones that have a default setting or fallback.

import { log } from "@repo/core/logging";
import { spawn } from "node:child_process";
import { projectId } from "~/config";

export default async function(
  collectionName: string
): Promise<void> {
  const args = [
    "@firebaseextensions/fs-bq-import-collection",
    "--non-interactive",
    "-P",
    projectId,
    "-B",
    projectId,
    "-d",
    "firestore_mirror",
    "-s",
    collectionName,
    "-t",
    collectionName,
    "-m",
    "true",
    "-u",
    "true",
    "-q",
    "false",
    "-l",
    "eu",
  ];

  return new Promise((resolve, reject) => {
    const process = spawn("npx", args);

    process.stdout.on("data", (data) => {
      log.info(data.toString());
    });

    process.stderr.on("data", (data) => {
      log.error(data.toString());
    });

    process.on("close", (code) => {
      if (code === 0) {
        resolve();
      } else {
        reject(new Error(`Process exited with code ${code}`));
      }
    });
  });
}