Small copy-truncate fixes and release info update (#754)

* update the changelog

* update VERSION file.

* Add dateext and extension options handling.

Co-authored-by: arthur <[email protected]>

Author: ArthurKamalov

CHANGELOG.md

@@ -1,10 +1,10 @@
 Scalyr Agent 2 Changes By Release
 =================================
 
-## 2.1.21 "TBD" - April 30, 2021
+## 2.1.21 "Ultramarine" - Jun 1, 2021
 
 <!---
-Packaged by Arthur Kamalov <[email protected]> on Apr 30, 2021 21:00 -0800
+Packaged by Arthur Kamalov <[email protected]> on Jun 1, 2021 21:00 -0800
 --->
 
 Improvements:
@@ -15,6 +15,7 @@ Bug fixes:
 * Fix an issue where LogFileIterator during the copy-truncate process picks wrong pending file with a similar name causing loss of the log and showing negative bytes in agent status.
 
 Other:
+* The discovery logic of the log files, which have been rotated with copy truncate method, now can handle only default rogrotate configurations. 
 * Agent now emits a warning if running under Python 2.6 which we will stop supporting in the next release.
 
 ## 2.1.20 "Tabeisshi" - April 19, 2021

VERSION

@@ -1 +1 @@
-2.1.20
+2.1.21

scalyr_agent/log_processing.py

@@ -1237,7 +1237,7 @@ def __find_copy_truncate_file(self):
                         r"{0}\.\d+".format(re.escape(file_name)), os.path.basename(f)
                     )
                     or
-                    # the same but the number is located before the extension ("extension" option in logrotate)
+                    # the same but the number or date is located before the extension ("extension" option in logrotate)
                     # (e.g. foo.1.log)
                     re.match(
                         r"{0}\.\d+{1}".format(
@@ -1246,9 +1246,19 @@ def __find_copy_truncate_file(self):
                         os.path.basename(f),
                     )
                     or
+                    # (e.g.foo-20210527.log)
+                    re.match(
+                        r"{0}-\d{{8}}{1}".format(
+                            re.escape(file_prefix), re.escape(file_ext)
+                        ),
+                        os.path.basename(f),
+                    )
+                    or
                     # if rotated file name ends with the date without additional extension (e.g. foo.log-20210527),
                     # then the only thing that we can check is that the extension starts with original file's extension.
-                    re.match(r"{0}.+".format(re.escape(file_name)), os.path.basename(f))
+                    re.match(
+                        r"{0}-\d{{8}}".format(re.escape(file_name)), os.path.basename(f)
+                    )
                 ),
                 self.__file_system.list_files(dir_path),
             )

tests/unit/log_processing_test.py

@@ -563,6 +563,18 @@ def test_find_copy_truncate_with_ext_option_filename(self):
         )
         self.assertEqual(copied_file, located_copy_truncate_file)
 
+    def test_find_copy_truncate_with_ext_dateext_option_filename(self):
+        # Test find_copy_truncate_file with date and extension rotation scheme
+        filename = os.path.join(self.__tempdir, "app.log")
+        self.write_file(filename, b"")
+        copied_file = os.path.join(self.__tempdir, "app-20210101.log")
+        self.write_file(copied_file, b"")
+        app_log_file = self._create_iterator({"path": filename})
+        located_copy_truncate_file = (
+            app_log_file._LogFileIterator__find_copy_truncate_file()
+        )
+        self.assertEqual(copied_file, located_copy_truncate_file)
+
     def test_find_copy_truncate_with_ext_and_dateext_option_filename(self):
         # Test find_copy_truncate_file with date rotation scheme
         filename = os.path.join(self.__tempdir, "app.log")