[Proposal] New Module: @golevelup/nestjs-google-cloud-pubsub

[chaikin-nikita]

Feature Request

This is a proposal to introduce a new module, @golevelup/nestjs-gcp-pubsub, to the golevelup/nestjs collection.

Problem

Integrating GCP Pub/Sub with NestJS using the standard libraries involves a significant amount of manual setup and boilerplate.

Developers are left to manually manage:

• The configuration, validation, and mapping of topics, subscriptions, schemas, and their revisions.
• Custom serialization/deserialization logic for different schema types (like Avro or ProtoBuf).
• Graceful shutdown of subscriptions to ensure messages are not lost when the application terminates.

This creates a high risk of runtime errors due to configuration drift and adds significant, repetitive complexity.

Proposed Solution

I propose a new module that solves this by integrating a core validation client with a new, type-safe API layer, all wrapped in a NestJS module.

The solution has two parts:

1. The Core Client (Already Implemented):

• A framework-agnostic PubsubClient that handles all critical runtime/compile-time logic: config drift detection, schema validation against remote revisions, and serialization.
• Exposes a .close() method to provide the logic for graceful shutdown.

2. The Type-Safe NestJS Layer (Work in Progress):

• This is the main part of the contribution, which I am currently developing (approx. 1 month of work remaining).
• It introduces the .initializeKit method, a type-inference engine that generates a fully-typed API (Publisher, Subscribe, Payloads) from a static config.
• It provides the NestjsGcpPubsubModule responsible for:
  • Calling the core client's validation logic on onModuleInit.
  • Hooking the core client's .close() method into onModuleDestroy.
  • Integrating the .initializeKit with NestJS's DI.

The initial implementation will focus on the pull model for subscriptions (the push model is out of scope for this initial version).

---

Status

To be clear, this is a work in progress that is already significantly underway.

• What is Done: The framework-agnostic core client (handling runtime/compile-time validation, serialization, shutdown) is complete and can be reviewed.
• What is In Progress: I am currently building the .initializeKit type-safety engine and the NestJS module to integrate it (40% already ready).

I am creating this issue to get early feedback and, most importantly, to understand if this is a feature you are interested in accepting before I complete the final integration.

The completed core client (which serves as the foundation) can be reviewed here:
➡️ https://github.com/chaikin-nikita/google-cloud-pubsub

---

Proposed API

The proposed API (which will be implemented by the new module) is designed to be highly type-safe, leveraging the .initializeKit.

pubsub.config.ts

import { Encodings, SchemaTypes } from '@google-cloud/pubsub';
import { pubsubClient } from '@golevelup/nestjs-gcp-pubsub';

const { Publisher, Subscribe, topicConfigurations, Payloads } = pubsubClient.initializeKit([
  {
    name: 'order.created',
    schema: {
      definition: {
        fields: [
          { name: 'field1', type: 'string' },
          { name: 'field2', type: 'int' },
          { name: 'field3', type: 'boolean' },
        ],
        name: 'Level3',
        type: 'record',
      },
      encoding: Encodings.Json,
      name: 'order.created.schema',
      type: SchemaTypes.Avro,
    },
    subscriptions: [{ name: 'order.created.subscription.analytic-service' }],
  },
  // ... other topics
]);

export { Publisher, Subscribe, topicConfigurations, Payloads };

app.module.ts

import { Module } from '@nestjs/common';
import { NestjsGcpPubsubModule, GcpPubsubMessage } from '@golevelup/nestjs-gcp-pubsub';
import { Publisher, topicConfigurations } from './pubsub.config';
import { AppService } from './app.service';

export const PUBLISHER_TOKEN = Symbol('TYPED_PUBLISHER');

@Module({
  imports: [
    NestjsGcpPubsubModule.registerAsync({
      topicConfigurations,
      Publisher: { injectionToken: PUBLISHER_TOKEN, ref: Publisher },
    }),
  ],
  providers: [AppService],
})
export class AppModule {}

app.service.ts

import { Inject, Injectable } from '@nestjs/common';
import { GcpPubsubMessage } from '@golevelup/nestjs-gcp-pubsub';
import { Subscribe, Publisher, Payloads } from './pubsub.config';
import { PUBLISHER_TOKEN } from './app.module';

@Injectable()
export class AppService {
  constructor(
    @Inject(PUBLISHER_TOKEN) private readonly publisher: InstanceType<typeof Publisher>
  ) {}

  public async publishMethod() {
    // Autocomplete for topic name: 'order.created'
    // Autocomplete and type-checking for the payload object!
    await this.publisher.publish('order.created', {
      field1: 'hello',
      field2: 123,
      field3: true,
      // field4: 'abc' // <-- TypeScript Error!
    });
  }

  // Autocomplete for topic and subscription names
  @Subscribe('order.created', 'order.created.subscription.analytic-service')
  public async subscribeMethod(
    // Typescript will throw error because of wrong type
    payload: GcpPubsubMessage<boolean> 
  ) {}

  @Subscribe('order.created', 'order.created.subscription.analytic-service')
  public async validSubscribeMethod(
    payload: GcpPubsubMessage<Payloads['order.created']> 
  ) {}
}

[WonderPanda]

This is cool! We are definitely open for contributions if you want to add this

[chaikin-nikita]

This is cool! We are definitely open for contributions if you want to add this

Cool. I'll get started on the integration and will open a Draft PR soon.