Error: Producer does not exist

[thatchej]

Have been debugging for a number of hours and am out of ideas. I am not sure if this is a bug or if something is being missed, but if it's the latter hopefully writing it out here will help.

Running into an unexpected Producer does not exist error when attempting to put a message onto an SQS queue.

With some parts omitted, here is the producer in question

import { QueueNames } from "../common/sqs/types.js";
import { Injectable } from "@nestjs/common";
import { SqsService } from "@ssut/nestjs-sqs";
import { v4 } from "uuid";
//...

@Injectable()
export class ClassThatSendsToSQS {
  // ...

  processMessage: MessageHandler = async (params: MessageParams) => {
    const value = {...};

    try {
        await this.sqsService.send(QueueNames.MyQueue, {
          id: v4(),
          body: value,
        });   
    } catch (error) {
      this.logger.warn(
        `Couldn't send message to SQS queue`,
      );
      this.logger.error(error);

      return;
    }
  };
}

and here is the module where nestjs-sqs is configured:

import { QueueNames } from "./types.js";
import { Module } from "@nestjs/common";
import { SqsModule } from "@ssut/nestjs-sqs";
import config from "config";

const awsSqsPrefix = config.get<string>("aws.sqs.prefix");
const prefix = `${awsSqsPrefix}`;
const region = "us-west-2";

@Module({
  imports: [
    // ...
    SqsModule.registerAsync({
      useFactory: () => {
        return {
          consumers: [
            {
              name: QueueNames.MyQueue,
              queueUrl: `${prefix}-my-queue`,
              region,
            },
          ],
          producers: [
            {
              name: QueueNames.MyQueue,
              queueUrl: `${prefix}-my-queue`,
              region,
            },
          ],
        };
      },
    }),
  ],
  controllers: [],
  providers: [
    MyQueueProcessor,
  ],
  exports: [
    MyQueueProcessor,
  ],
})
export class SqsQueuesModule {}

This SqsQueuesModule is part of the imports: [] array within the base module for a Nest app.

Note that this same app both produces to and consumes from the queue, hence having a config object for both.

The only slightly odd part of our setup is that we have a separate SqsModule.registerAsync call within a different Nest app that does not include this producer. I wouldn't think that should have any effect on the SqsModule within this separate app though.

Is something off here with my configuration? I've looked over it a silly number of times at this point but haven't seen anything out of the ordinary

Thanks for your help

[ssut]

Can you please create a repository which can reproduce this issue?

[thatchej]

Hey @ssut thanks for the reply. Unfortunately I think it would take a few hours to get a mock repo setup pointing at a local SQS, and part of me thinks that time would be better spent troubleshooting within the real application's setup (as I think that is where the problem lies somehow)

I will keep this up to date with any findings. I have an inclination that it may be related to having the SqsModule configured differently between two separate Nest apps so will go test that now.

At a high level however, does my producer and module setup look reasonable and expected to you?

[ssut]

Your setup looks just fine to me. However, I'm curios about the mention of "two separate Nest apps" part though - do you mean you're running 2 nest instances in the same context(runtime)?

If so, can you please inject ModulesContainer from @nestjs/core into your ClassThatSendsToSQS class and log this.modulesContainer.applicationId in the function like the following:

import { ModulesContainer } from '@nestjs/core';
// ...
constructor(private modulesContainer: ModulesContainer) {}
// ...
processMessage: MessageHandler = async (params: MessageParams) => {
  console.log('applicationId =', this.modulesContainer.applicationId);
}

if the IDs are the same in both apps, they're probably conflicting by some reason.

[thatchej]

From my understanding, these would be running in totally different runtimes. As an example we have our main application that has app.module.ts, main.ts, etc. This contains our GraphQL APIs and main business logic

Separately for our native Kafka consumers we have something like kafka-consumers.module.ts, main.kafka-consumers.ts, etc.. This is packaged and deployed via Helm as a separate Docker container/application so that we can scale the applications separately, and have some resiliency on different parts of the app if the kafka consumers get overloaded for example.

That being said, we have use cases for nestjs-sqs in both and so I am separately configuring producers and consumers within each. I see no reason this shouldn't work and I don't imagine there's any weird cross-pollination but it's about my only running theory atm unless I find a random typo somewhere that I haven't seen yet.

I will mess around with this built-in applicationId stuff and see if it leads me down any interesting rabbit holes