Fix resourse cleaning

[Delagen]

request stream by demand, fix fs.unlink to call when response send to client or stream open

[Delagen]

For #568

[LinusU]

Would you mind explaining more what this change does and why you think it's the right approach?

I fail to see how this could fix an EPERM error 🤔

[Delagen]

@LinusU It does not fix EPERM error, it's my codebase error.
But it solve unneeded stream opening when code doesn't need it.
And remove files after handler ends and response finished even if files doesn't read in it.

Without this PR files are left opened till process exit if user handler do nothing with them

[LinusU]

Hmm, I'm afraid that relying on this could have unintended consequences. For example, I'm not sure that the following code would still work:

app.post('/upload', upload.single('file'), async (req, res) => {
  res.send(200, 'OK')

  const fileName = await allocateFileName()
  const target = fs.createWriteStream()

  req.file.stream.pipe(target)
})

Basically, I'm not sure (I don't have a definitive decision, just want to bring up discussion) that not using the stream before responding to the http request means that you don't want to use the file. 🤔

[Delagen]

@LinusU this code should work, but it wrong because it can crash, but user does not get any info about it. I recommend at least get stream before finishing request.
I only moved create stream to singleton with getter, and fix removing files at response finish.
But without my PR

app.post('/upload', upload.single('file'), async (req, res) => {
  res.send(200, 'OK')
})

will create tmp file and DOES NOT remove it at any time and takes handle, so app may reach open_files limit and crash.

[LinusU]

this code should work

I don't think that it does, since the file will have been removed by the time it hits the req.file.stream getter.

but it wrong because it can crash, but user does not get any info about it

That doesn't necessarily make it wrong. There could be plenty of reasons for not wanting to notify the user of errors.

will create tmp file and DOES NOT remove it at any time and takes handle, so app may reach open_files limit and crash.

The files should be removed, but I just saw that this does in fact leak file descriptors. Basically it boils down to the underlying FD not beeing freed when the ReadStream is garbage collected.

This is potentially something that should be fixed in Node.js, I'll open an issue about that.

We could probably use e.g. node-weak to work around that manually for now.

[noonii]

+1

I noticed in multer@next that OP is right it does not get deleted from temp file upon failed or successful request.

When I re-run the app, only then it is removed.