Doesn't work with Next.js in APIs (events seem sent but aren't available on Amplitude dashboard)

[Vadorequest]

Expected Behavior

Event sent using @amplitude/node should be available on the Amplitude Dashboard

Current Behavior

Events aren't available in Amplitude, as if they were never received.

Logs:

info  - ready on http://localhost:8888
2021-06-22T18:09:53.157Z [modules/core/amplitude/amplitudeServerClient] Logging Amplitude event "api-invoked" with properties: { apiEndpoint: 'status' }
2021-06-22T18:09:53.157Z [modules/core/amplitude/amplitudeServerClient] NodeClient {
  _events: [],
  _responseListeners: [],
  _flushTimer: null,
  _apiKey: '5ea02d86a6840c165fcc01377131fa13',
  _options: {
    serverUrl: 'https://api2.amplitude.com/2/httpapi',
    debug: true,
    maxCachedEvents: 16000,
    logLevel: 3,
    optOut: false,
    retryTimeouts: [
       100,  100,  200,  200,
       400,  400,  800,  800,
      1600, 1600, 3200, 3200
    ],
    retryClass: null,
    transportClass: null,
    uploadIntervalInSec: 0,
    minIdLength: null
  },
  _transportWithRetry: RetryHandler {
    _apiKey: '5ea02d86a6840c165fcc01377131fa13',
    _options: {
      serverUrl: 'https://api2.amplitude.com/2/httpapi',
      debug: true,
      maxCachedEvents: 16000,
      logLevel: 3,
      optOut: false,
      retryTimeouts: [Array],
      retryClass: null,
      transportClass: null,
      uploadIntervalInSec: 0,
      minIdLength: null
    },
    _transport: HTTPTransport {
      options: [Object],
      _uploadInProgress: false,
      _requestQueue: [AsyncQueue],
      module: [Object]
    },
    _idToBuffer: Map(0) {},
    _eventsInRetry: 0
  }
} 5ea02d86a6840c165fcc01377131fa13

response {
  status: 'success',
  statusCode: 200,
  body: {
    eventsIngested: 0,
    payloadSizeBytes: 58,
    serverUploadTime: 1624385394010
  }
}

The API key is defined, the response is sent and awaited for, the result is a success, but there are no data in the Amplitude Dashboard. The @amplitude/react-amplitude package works fine on the browser, though.

It doesn't work locally, and it doesn't work on Vercel either.

I'm not sure what's wrong.

Possible Solution

Potentially related to flushing and https://vercel.com/docs/platform/limits#streaming-responses, but I've flushed and awaited for the flush to avoid any issue. Also, it doesn't even work locally.

Steps to Reproduce

1. git clone git@github.com:UnlyEd/next-right-now.git nrn-refactor-amplitude && cd nrn-refactor-amplitude && git checkout refactor-amplitude && cp .env.local.example .env.local && yarn && yarn start
2. Go to localhost:8888/api/status and look at the server console

Environment

• SDK Version: "@amplitude/node": "1.6.1",
• Node version: v14

[Vadorequest]

Also, the UX on the server isn't great, because you're making either device_id or user_id required, but most of the time I don't know what's the device (it's an API call, there is no device) and sometimes I have no way to know who's the user either.

I think none of those fields should be required for server-side usage. They should both be optional.

[Vadorequest]

Okay, so it looks like it was because the call worked (200 response) but eventsIngested was 0.

And this is because of the user/device issue. I had to specify a device_id value even though I'm on a server and don't know what value to use. Should I rather use a dynamically generated value for every call?

[Vadorequest]

I've fixed it at 815ff95 (#374)

But this doesn't seem great, Amplitude now believes all events are related to the same user, which isn't really the case. So I'll rather use a dynamic UUID value as default user_id to counter this behavior.

I wish this had been explained in the README, I've lost a whole day on this.

[Vadorequest]

I'd like to know if it's necessary to await the flush() on Next.js API endpoints, I believe it is but that'd be great to have a confirmation. 😄

[yuhao900914]

Hi @Vadorequest, great to hear you solved the problem on your own.

1. The Amplitude-Node will call our Amplitude HTTP v2 API, the device_id and the user_id are semi-required there. So it cannot be all optional. If a device_id is not sent with the event, it will be set to a hashed version of the user_id. So you can set a dynamic UUID as default user_id when you don't know what's the user_id is and don't pass the device_id during logEvent.
2. In Amplitude-Node, the flush() function is async and will return a promise. So, yes, if you need to do something with the response you need to await it.

[Vadorequest]

In Amplitude-Node, the flush() function is async and will return a promise. So, yes, you need to await it.

I was asking in relation to https://vercel.com/docs/platform/limits#streaming-responses. I understand I need to await it if I want to do anything with the Response, but do I need to await it to make sure the event is sent in the context of AWS Lambda? If I don't await logEvent() but simply logEvent(), do I take the risk of the event not being processed? That's what a bit uncertain at the moment.

Ideally, I'd rather not await when calling logEvent, because I don't want to make any of my own API endpoints slower.

[yuhao900914]

@Vadorequest
To clarify your question,
You are using Vercel as the framework. And you are curious if ​Vercel does not support streaming responses will affect how you use Amplitude-Node and if you must add await when calling flush or logEvent?
cc: @ajhorst

[Vadorequest]

Yes, exactly.

…

On Tue, Jun 22, 2021, 22:30 Alyssa Yu ***@***.***> wrote: @Vadorequest <https://github.com/Vadorequest> To clarify your question, Do you mean your platform is deployed by Vercel. And you are curious if ​Vercel does not support streaming responses will affect how you use Amplitude-Node and if you must add await when calling flush or logEvent? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#123 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5BRYR4DECLUZEOZJRDX6TTUDXGVANCNFSM47EHW46A> .

[Vadorequest]

I've opened a related issue on Next.js to try to figure out the pros/cons and best practices regarding all of this. vercel/next.js#26523