RFC: Qwik Community Plugins Support

[shairez]

Discussed in #20

^{Originally posted by shairez September 16, 2024}

What is it about?

A way to expand Qwik without increasing core size

What's the motivation for this proposal?

Problems you are trying to solve:

• Currently the Qwik monorepo contains a lot of adapters, plugins and integrations. We would like to make sure anything that is in core stays stable over time and it's hard to do it with an ever growing number of possible integrations.

Goals you are trying to achieve:

• Keep qwik core stable and trustworthy
• Give a way to community contributors to integrate their package into the Qwik CLI

Any other context or information you want to share:

---

Proposed Solution / Feature

A Qwik plugin system.

Scanability

Every package in node_modules that starts with a specific convention like qwik- should be scanned for its qwik-plugin-manifest.json file which describes specific instructions of which files to copy, modify, etc.

Extracting some of the existing plugins from core

Separating possible integrations as "maintained by core team" (anything under the QwikDev org) or "maintained by community" (anything under the QwikCommunity org) will allow the core team to take responsibility of fewer things but make sure they are always stable / maintained.

File management

We need to brainstorm how different types of plugins will look like (usually there is a difference between the types).

Plus, we would need to modify the Qwik CLI to be aware of external integrations so we could install or add them with ease (running their script).
We would need to provide an infrastructure for copying new files and modifying existing files that are read from the plugin's manifest.

Discoverability

We need a way to list core plugins and community plugins in a searchable area under the Qwik docs

---

Links / References

Here are some examples for existing plugin systems from several projects:

• https://docs.cypress.io/guides/component-testing/third-party-definitions

• https://nx.dev/extending-nx/recipes/create-preset

Example for a "discover plugins" page -

• https://nx.dev/plugin-registry

(via @ianlet):
Adding this one for inspiration:

• https://github.com/npm/cli/blob/latest/docs/lib/content/commands/npm-init.md

Which is the usual npm|pnpm|yarn create [...] command.

And this one for an other example of "plugin registry":

• https://github.com/asdf-vm/asdf-plugins

[ianlet]

Alright, me and my buddy Gemini (I tried others but ended up sticking with this guy) prepared something to progress on this RFC... It's just a draft, so be critical and let's make this as good as possible to write plugins/integrations for Qwik.

Based on what was discussed so far, here's a suggestion for the Qwik plugin system. Describing both a declarative approach and a programmatic approach:

Step 1: CLI Commands for End Users

The Qwik CLI provides commands to search for and install plugins.

Plugin Discovery

qwik plugin search <keyword>

• qwik plugin search: Base command to find plugins.
• <keyword>: (Optional) Filter plugins by keyword (e.g., auth, tailwind). If omitted, lists all plugins.

Search Result Output (Example):

Available Qwik Plugins:

Name: @qwikdev/qwik-city
Description: The official meta-framework for building full-stack Qwik apps.
Maintainer: QwikDev (Core)
Version: 0.0.1
Tags: meta-framework, routing, server

Name: @qwikcommunity/qwik-tailwind
Description: Integrates Tailwind CSS for styling Qwik applications.
Maintainer: QwikCommunity
Version: 1.0.0
Tags: styling, css

... and more ...

Plugin Installation

qwik plugin add <plugin-name>

• qwik plugin add: Installs a plugin.
• <plugin-name>: The plugin's name (e.g., @qwikcommunity/qwik-tailwind).

Installation Process:

1. Installs the plugin package using the project's package manager.
2. Locates the plugin's entry point.
3. Executes the plugin's logic (programmatic SDK) or processes the manifest (declarative SDK).
4. Provides user feedback.

Step 2: SDK for Plugin Developers

Plugin developers define plugin behavior using either a declarative manifest or a programmatic API.

2.1 Declarative Approach: qwik-plugin-manifest.json

This approach uses a JSON file to declare plugin actions, suitable for simpler plugins.

Example:

{
  "name": "@qwikcommunity/qwik-tailwind",
  "version": "1.0.0",
  "description": "Integrates Tailwind CSS for styling Qwik applications.",
  "maintainer": "QwikCommunity",
  "tags": ["styling", "css"],
  "qwikPlugin": {
    "type": "feature",
    "steps": [
      {
        "type": "prompt",
        "questions": [
          {
            "type": "text",
            "name": "projectName",
            "message": "What is the name of your project?",
            "initial": "MyAwesomeApp"
          }
        ]
      },
      {
        "type": "addFiles",
        "files": [
          {
            "source": "files/tailwind.config.js",
            "target": "tailwind.config.js"
          },
          {
            "source": "files/postcss.config.js",
            "target": "postcss.config.js"
          },
          {
            "source": "files/component.tsx.ejs",
            "target": "src/components/<%= projectName %>Component.tsx",
            "data": { "projectName": "MyProject" }
          }
        ]
      },
      {
        "type": "modifyFile",
        "file": "vite.config.ts",
        "operations": [
          {
            "type": "addImport",
            "import": "import tailwindcss from 'tailwindcss';",
            "where": "top"
          },
          {
            "type": "addToArray",
            "array": "plugins",
            "value": "tailwindcss()"
          }
        ]
      },
      {
        "type": "modifyFile",
        "file": "package.json",
        "operations": [
          {
            "type": "addDependency",
            "dependency": "tailwindcss",
            "version": "^3.0.0"
          },
          {
            "type": "addDevDependency",
            "dependency": "postcss",
            "version": "^8.0.0"
          },
          {
            "type": "addScript",
            "script": "build:css",
            "command": "tailwindcss -i ./src/global.css -o ./public/build/tailwind.css --watch"
          }
        ]
      },
      {
        "type": "deleteFiles",
        "files": ["./unwanted-file.txt"]
      },
      {
        "type": "moveFiles",
        "moves": [
          {
            "from": "./old-directory/file.txt",
            "to": "./new-directory/file.txt"
          }
        ]
      },
      {
        "type": "logMessage",
        "message": "Tailwind CSS integration complete!"
      }
    ]
  }
}

Manifest Structure:

• name, version, description, maintainer, tags: Plugin metadata.
• qwikPlugin:
  • type: Plugin category (e.g., "feature", "adapter", "integration", etc.). Mostly for filtering with the CLI.
  • steps: Array of actions.
    • addFiles: Copies files.
      • files: Array of { source: string; target: string; templateData?: object } objects.
        • source: File path in the plugin.
        • target: Destination path.
        • templateData: (Optional) Data for templating.
    • modifyFile: Modifies a file.
      • file: File path.
      • operations: Array of modification operations. The available operations depend on the file type.
    • deleteFiles: Deletes files.
      • files: Array of file paths.
    • moveFiles: Moves files.
      • moves: Array of { from: string; to: string } objects.
    • logMessage: Displays a message.
      • message: Message to display.
    • prompt: Prompts the user for input.
      • questions: Array of question objects (like prompts or inquirer).

2.2 Programmatic Approach

This approach uses a JavaScript/TypeScript module for more complex plugin logic.

Plugin Entry Point:

// qwik-plugin.ts

import { QwikPluginContext } from '@qwikdev/qwik-cli';

export default async function qwikPlugin(context: QwikPluginContext) {
  try {
    // Modify package.json
    await context.packageJson(async (packageJson) => {
      // Use package.json manipulation tools (e.g., @npmcli/package-json)
      packageJson.update({
        dependencies: {
          ...packageJson.content.dependencies,
          tailwindcss: '^3.0.0',
        },
        devDependencies: {
          ...packageJson.content.devDependencies,
          postcss: '^8.0.0',
        },
        scripts: {
          ...packageJson.content.scripts,
          'build:css': 'tailwindcss -i ./src/global.css -o ./public/build/tailwind.css --watch',
        },
      });
    });

    // Modifying vite.config.ts
    // For vite config files, it could actually be more interesting to have a higher level API to edit it.
    await context.modifyTsFile('vite.config.ts', (ast) => {
      // Use TypeScript AST manipulation tools (e.g., ts-morph)
      const importDeclaration = ast.createImportDeclaration({
        moduleSpecifier: 'tailwindcss',
        namedImports: [],
      });

      const pluginsArray = ast.getPropertyAssignment('plugins')?.getInitializerIfKind(ts.SyntaxKind.ArrayLiteralExpression);
      if (pluginsArray) {
        pluginsArray.addElement('tailwindcss()');
      }

      return ast;
    });

    // Modifying a CSS file
    await context.modifyCssFile('./src/global.css', (css) => {
      css.appendRule('.my-custom-style', {
        'color': 'blue',
        'font-weight': 'bold',
      });
      return css;
    });

    // Modifying a text file
    await context.modifyTextFile('README.md', (content) => {
      return `${content}\n\n## Plugin Integration\nThis project now includes the awesome Tailwind CSS plugin!`;
    });

    // Prompt user with a prompting tool like Prompts, Inquirer, etc.
    const answers = await context.prompt([
      {
        type: 'text',
        name: 'projectName',
        message: 'What is the name of your project?',
        initial: 'MyAwesomeApp',
      },
    ]);

    const { projectName } = answers;

    // Adding new files with templating
    await context.addFiles([
      {
        source: 'files/component.tsx.ejs',
        target: `src/components/${projectName}Component.tsx`,
        data: { projectName },
      },
      {
        source: 'files/styles.css',
        target: `src/components/${projectName}Component.css`,
      },
    ]);

    // Deleting a file
    await context.deleteFiles(['./unwanted-file.txt']);

    // Moving files
    await context.moveFiles([{ from: './old-directory/file.txt', to: './new-directory/file.txt' }]);

    // Log messages to the user
    context.log('Tailwind CSS integration complete!');
  } catch (error) {
    context.error(`Error: ${error}`);
    throw error;
  }
}

To make the plugin discoverable, the package.json of the plugin could have a specific entry point that the Qwik CLI looks for:

{
  "name": "@qwikcommunity/qwik-tailwind",
  "version": "1.0.0",
  "description": "Integrates Tailwind CSS for styling Qwik applications.",
  "maintainer": "QwikCommunity",
  "tags": ["styling", "css"],
  "qwikPlugin": "./qwik-plugin.ts" // Entry point for the plugin logic
}

QwikPluginContext API:

• context.projectRoot: string: (Hypothetical) Project root directory.
• context.packageJson(modifier: (packageJson: PackageJson) => Promise<void>): Modifies package.json.
  • modifier: Async function to update the packageJson object.
• context.modifyTsFile(file: string, modifier: (ast: SourceFile) => SourceFile | Promise<SourceFile>) and context.modifyJsFile: Modifies TypeScript/JavaScript files.
  • file: File path.
  • modifier: Async function to modify the AST.
• context.modifyCssFile(file: string, modifier: (stylesheet: CSSStyleSheet) => CSSStyleSheet | Promise<CSSStyleSheet>): Modifies CSS files.
  • file: File path.
  • modifier: Async function to modify the stylesheet object.
• context.modifyTextFile(file: string, modifier: (content: string) => string | Promise<string>): Modifies text files.
  • file: File path.
  • modifier: Async function to modify the content.
• context.addFiles(files: { source: string; target: string; templateData?: Record<string, any> }[]): Adds files.
  • files: Array of file objects.
    • source: File path in the plugin.
    • target: Destination path.
    • templateData: (Optional) Templating data.
• context.deleteFiles(files: string[]): Deletes files.
  • files: Array of file paths.
• context.moveFiles(moves: { from: string; to: string }[]): Moves files.
  • moves: Array of move objects.
• context.runScript(name: string): Runs a script in package.json.
  • name: Script name.
• context.hasScript(name: string): boolean: Checks if a script exists.
  • name: Script name.
• context.log(message: string): Logs a message.
  • message: Message to log.
• context.error(message: string): Logs an error.
  • message: Error message.
• context.prompt(questions: any[]): Promise<any>: Prompts the user for input.
  • questions: Array of question objects (like prompts).

[ianlet]

Another approach for the programmatic version is to offer "standalone" functions that provide the different features we would need instead of making them part of the context object. For example:

// qwik-plugin.ts

import {
  modifyPackageJson,
  modifyTsFile,
  modifyCssFile,
  modifyTextFile,
  addFiles,
  deleteFiles,
  moveFiles,
  runScript,
  hasScript,
  log,
  error,
  prompt,
  QwikPluginContext,
  QwikPlugin, // Hypothetical type for the main function
} from '@qwikdev/qwik-plugin-api'; // Hypothetical module

import { PackageJson } from '@npmcli/package-json';
import { SourceFile } from 'typescript';
import { CSSStyleSheet } from 'css';
import prompts from 'prompts';
import ejs from 'ejs';

const qwikPlugin: QwikPlugin = async (context: QwikPluginContext) => {
  try {
    const answers = await prompt(context, [
      {
        type: 'text',
        name: 'projectName',
        message: 'What is the name of your project?',
        initial: 'MyAwesomeApp',
      },
    ]);

    const { projectName } = answers;

    // Modify package.json
    await modifyPackageJson(context, async (packageJson) => {
      packageJson.update({
        dependencies: {
          ...packageJson.content.dependencies,
          tailwindcss: '^3.0.0',
        },
        devDependencies: {
          ...packageJson.content.devDependencies,
          postcss: '^8.0.0',
        },
        scripts: {
          ...packageJson.content.scripts,
          'build:css': 'tailwindcss -i ./src/global.css -o ./public/build/tailwind.css --watch',
        },
      });
    });

    // Modifying vite.config.ts
    await modifyTsFile(context, 'vite.config.ts', (ast) => {
      const importDeclaration = ast.createImportDeclaration({
        moduleSpecifier: 'tailwindcss',
        namedImports: [],
      });

      const pluginsArray = ast.getPropertyAssignment('plugins')?.getInitializerIfKind(ts.SyntaxKind.ArrayLiteralExpression);
      if (pluginsArray) {
        pluginsArray.addElement('tailwindcss()');
      }

      return ast;
    });

    // Modifying a CSS file
    await modifyCssFile(context, './src/global.css', (css) => {
      css.appendRule('.my-custom-style', {
        'color': 'blue',
        'font-weight': 'bold',
      });
      return css;
    });

    // Modifying a text file
    await modifyTextFile(context, 'README.md', (content) => {
      return `${content}\n\n## Plugin Integration\nThis project now includes the awesome Tailwind CSS plugin!`;
    });

    // Adding new files with templating
    await addFiles(context, [
      {
        source: 'files/component.tsx.ejs',
        target: `src/components/${projectName}Component.tsx`,
        templateData: { projectName },
      },
      {
        source: 'files/styles.css',
        target: `src/components/${projectName}Component.css`,
      },
    ]);

    // Deleting a file
    await deleteFiles(context, ['./unwanted-file.txt']);

    // Moving files
    await moveFiles(context, [{ from: './old-directory/file.txt', to: './new-directory/file.txt' }]);
  } catch (error) {
    error(context, `Error: ${error}`);
    throw error;
  }
};

export default qwikPlugin;

That way, it would be way more flexible and easier to introduce new higher-level API. And plugin developers will be able to develop their own modifiers.

[shairez]

Thanks @ianlet !

Here are my thoughts:

I like the programmatic approach (especially the second example with the utils), although the declarative could be useful for simple cases indeed.
I think using something from codemods or something similar to what @dmitry-stepanenko is doing could be better here, and not re-implementing from scratch.

Another thing to consider: we need to have a registry and we need to differentiate between official plugins, "approved" third party plugins (from the community) that will show up in our registry (could be submitted as a PR to the registry) and "unapproved" third party plugins (use at your own risk).

In terms of qwik add, I believe it should be able to scan that registry for official + "approved" plugins only, but if you want to install an "unapproved" third party plugin you can do it by pointing to the name of that package

Plugins should also have the ability to migrate to a newer version of themselves (like nx plugins), it'll make it easier for version updates and could use the same capabilites of the upcoming qwik migrate command (WDYT @dmitry-stepanenko ?)

And last thing - it'll be cool if we could consider an option to remove a plugin, but that's not a must for the first stage.