Rotation with zip not save order of files

[yakirzana]

Versions:

 "winston": "^3.2.1",
 "winston-daily-rotate-file": "^4.5.0"

Code:

new (transports.DailyRotateFile)({
                filename: "%DATE%.log",
                datePattern: 'YYYY-MM-DD',
                zippedArchive: true,
                maxSize: 1m,
                maxFiles: 50,
})

After reached the size limit on the same day, it continues to the next logs (2020-06-29.log.1) and zipping the old file (2020-06-29.log.gz).
But, after starting the program again, It will create a new log (2020-06-29.log) again and not continue from the previous counting (2020-06-29.log.1).
Also, after we reach the max size again, It will write to an existing file (2020-06-29.log.1), and it is not zipping the new log (2020-06-29.log) because we already have a zip file on this name.

[mattberther]

@yakirzana Can you verify the number of .json audit files in your log folder? If you find multiple files in there, please consider using the auditFile property to reuse the file, which should address your issue. If not, please re-open this issue.

[yakirzana]

@mattberther One file.
By the way, this file will not be updated with the zipped files.

I cant re-open this issue again; please update the status.

[yakirzana]

@mattberther Can you re-open the issue again?

[mattberther]

Please provide the smallest code snippet that reliably reproduces this problem. Thank you.

[yakirzana]

It includes saving the logs to a directory.
Run this script multiple times, and you will see the issue.

const fs = require('fs');
const {createLogger, format, transports} = require('winston');
require('winston-daily-rotate-file');

const dir = "logs";
let name =  "log" + '-%DATE%.log';
if (!fs.existsSync(dir))
    fs.mkdirSync(dir);

const logger = createLogger({
    format: format.combine(
        format.timestamp({
            format: "YYYY-MM-DDTHH:mm:ss.SSSZZ"
        }),
        format.printf(info => `${info.timestamp} ${info.level}: ${info.message}`)
    ),
    transports: [
        new transports.Console({
            level: 'info',
            localTime: true,
            format: format.combine(
                format.colorize(),
                format.printf(
                    info => `${info.timestamp} ${info.level}: ${info.message}`
                )
            )
        }),
        new (transports.DailyRotateFile)({
            filename: name,
            dirname: dir,
            datePattern: 'YYYY-MM-DD',
            zippedArchive: true,
            maxSize: "1k",
            maxFiles: 50
        })
    ]
});

for (let i = 0; i < 70; i++) {
    logger.info(i);
}

[mattberther]

Thank you @yakirzana. To confirm, is this only an issue when using the zippedArchive option? Do you see similar behaviour when you remove that option?

[yakirzana]

Thank you @yakirzana. To confirm, is this only an issue when using the zippedArchive option? Do you see similar behaviour when you remove that option?

Yes, also, it does not happen without the dirname option.

[mattberther]

Would you say that this is a duplicate of #125? If so, let's reopen that issue, since there's a lot of context there.

[yakirzana]

Would you say that this is a duplicate of #125? If so, let's reopen that issue, since there's a lot of context there.

No, I don't think it's related.

The issue is with the use of FileStreamRotataor, that don't know the zipped files.
On line 483 in FileStreamRotataor.js:
if(lastEntry.match(t_log)){
It return false, and without directory its return a value

"logs\\log-2020-07-12.log.2".match("logs\\log-2020-07-12.log") === null
"log-2020-07-12.log.2".match("log-2020-07-12.log") === not null

The platform-specific separator on Windows is \\ , and on POSIX / , so maybe the issue is only at Windows.

There is also a bug related to this:
rogerc/file-stream-rotator#71

So t_log = "logs\log-2020-07-12.log"

On line 497 in FileStreamRotataor.js:

while(f = fs.existsSync(t_log)){
            lastLogFile = t_log;
            fileCount++;
            t_log = logfile + "." + fileCount + options.extension;
        }

The file log-2020-07-12.log does not exist (Is existed on name log-2020-07-12.log.gz)
So the FileStreamRotataor create it again.

[yakirzana]

@mattberther Some temporally fix to this issue will be on FileStreamRotator.js
On line 481
Replace:

if(auditLog && auditLog.files && auditLog.files instanceof Array && auditLog.files.length > 0){
            var lastEntry = auditLog.files[auditLog.files.length - 1].name;
            if(lastEntry.match(t_log)){
                var lastCount = lastEntry.match(t_log + "\\.(\\d+)");
                // Thanks for the PR contribution from @andrefarzat - https://github.com/andrefarzat
                if(lastCount){                    
                    t_log = lastEntry;
                    fileCount = lastCount[1];
                }
            }
        }

With:

 if(auditLog && auditLog.files && auditLog.files instanceof Array && auditLog.files.length > 0){
            var lastEntry = auditLog.files[auditLog.files.length - 1].name;
            const  lastEntryWithoutPath = lastEntry.split("\\").pop();
            const  t_logWithoutPath = t_log.split("\\").pop();
            if(lastEntryWithoutPath.match(t_logWithoutPath)){
                var lastCount = lastEntryWithoutPath.match(t_logWithoutPath + "\\.(\\d+)");
                // Thanks for the PR contribution from @andrefarzat - https://github.com/andrefarzat
                if(lastCount){                    
                    t_log = lastEntry;
                    fileCount = lastCount[1];
                }
            }
        }

[mattberther]

Thank you, @yakirzana. I'm following rogerc/file-stream-rotator#72 and will update this transport once that change has been merged. I'll leave the ticket open until then.