2.1.36 "Corrntos" - September 15, 2022

Windows:

• The monitor field is set to windows_event_log_monitor in the windows_event_log monitor if the json config option is true.
• Proper handling of forwarded windows events (ie handling of EvtOpenPublisherMetadata exceptions) in the windows_event_log monitor.

2.1.35 "Xabboaturn" - September 8, 2022

Windows:

• More sophisticated handling of array-based (event field) names in windows_event_log monitor.

2.1.34 "Tadongolia" - August 30, 2022

Improvements:

• Add support for tracemalloc based memory profiler ("memory_profiler": "tracemalloc" config option). Keep in mind that tracemalloc is only available when running the agent under >= Python 3.5.
• Add new memory_profiler_max_items config option which sets maximum number of items by memory usage reported by the memory profiler.
• Add new enable_cpu_profiling and enable_memory_profiling config option with which user can enable either CPU or memory profiler, or both. Existing enable_profiling config behavior didn't change and setting it to true will enable both profilers (CPU and memory).
• Allow user to specify additional trace filters (path globs) for tracemalloc memory profiler using memory_profiler_ignore_path_globs config option. (e.g. memory_profiler_ignore_path_globs: ["**/scalyr_agent/util.py", "**/scalyr_agent/json_lib/**"]).
• Update syslog_monitor to ignore errors when decoding data from bytes into unicode if data falls outside of the utf-8 range.

Bug fixes:

• Update windows_event_log_monitor to handle embedded double quotes in fields.

2.1.33 "Chaavis" - August 17, 2022

Improvements:

• Add option stop_agent_on_failure for each monitor's configuration. If true, the agent will stop if the monitor fails. For Kubernetes deployments this is true by default for the Kubernetes monitor (scalyr_agent.builtin_monitors.kubernetes_monitor). The agent's pod will restart if the monitor fails.

Kubernetes Explorer:

• Update code to calculate per second rate for various metrics used by Kubernetes Explorer on the client (agent) side. This may result in slight CPU and memory usage increase when using Kubernetes Explorer functionality.

Windows:

• Add new json config option to the windows_event_log monitor. When this option is set to true, events are formatted as JSON.

Other:

• Changed log severity level from ERROR to WARNING for non-fatal and temporary network client error.
• Update agent packages to also bundle new LetsEncrypt CA root certificate (ISRG Root X2). Some of the environments use LetsEncrypt issued certificates.
• Update agent code base to log a warning with the server side SSL certificate in PEM format on SSL certificate validation failure for easier troubleshooting.
• Upgrade various bundled dependencies (orjson, docker).

Bug fixes:

• Set new stop_agent_on_failure monitor config option to true in the agent Docker image for Kubernetes deployments. Solves an issue, present in some rare edge cases, where the Kubernetes Monitor (scalyr_agent.builtin_monitors.kubernetes_monitor) exits, but the agent continues to run. The agent's pod will restart if the monitor fails.

2.1.32 "Occao" - July 27, 2022

Windows:

• Fix bug in Windows System Metrics and Windows Process Metrics monitor where user wasn't able to override / change default sampling rating of 30 seconds (sample_interval monitor config option).
• Update Windows Process Metrics monitor to log a message in case process with the specified pid / command line string is not found when retrieving process metrics.
• Update Windows Process Metrics monitor to throw an error in case invalid monitor configuration is specified (neither "pid" nor "commandline" config option is specified or both config options which are mutually exclusive are specified).

Bug fixes:

• Fix a bug with import_vars functionality which didn't work correctly when the same variable name prefix was used (e.g. SCALYR_FOO_TEST, SCALYR_FOO).
• Fix a bug with handling the log file of the Kubernetes Event Monitor twice, which led to duplication in the agent's status.
• Fix a bug in scalyr-agent-2-config --export-config, import-config options caused by Python 2 and 3 code incompatibility.
• Fix a bug with the wrong executable scalyr-agent-2-config in Docker and Kubernetes, due to which it could not be used.

Docker images:

• Upgrade various dependencies: orjson, requests, zstandard, lz4, docker.

Other:

• Support for Python 2.6 has been dropped.
• Support for ujson JSON library (json_library configuration option) has been removed in favor of orjson.
• Update agent log messages to include full name of the module which produced the message.

2.1.31 "Irati" - Jun 28, 2022

Windows:

• Update base Python interpreter for Windows MSI package from 3.8 to 3.10.
• Upgrade various bundled dependencies (pywin32, orjson, urllib3, six).

Bug fixes:

• Fix a regression introduced in v2.1.29 which would cause the agent to inadvertently skip connectivity check on startup.
• Default value for check_remote_if_no_tty config option is False. Previously the changelog entry incorrectly stated it defaults to True. This means that a connectivity check is not performed on startup if tty is not available.
• Fix a bug in syslog monitor on Windows under Python 3 which would prevent TCP handler from working.
• Fixed agent checkpoint selection bug that could cause old log files to be re-uploaded.
• Small bug with command line argument parsing for the Agent. Agent raised unhandled exception instead of normal argparse error message when agent main command wasn't specified.
• Fix a bug in the Agent's custom JSON parser, which did not raise error on unexpected ending of the JSON document which might be caused by a JSON syntax error.

Docker images:

• Upgrade orjson dependency

Other:

• Monitor emit_value() method now correctly sanitizes / escapes metric field names which are "reserved" (logfile, metric, value, serverHost, instance, severity). This is done to prevent possible collisions with special / reserved metric event attribute names which could cause issues with some queries. Metric field names which are escaped get added _ suffix (e.g. metric becomes metric_).
• Upgrade dependency requests library to 2.25.1.

2.1.30 "Heturn" - May 17, 2022

Kubernetes:

• Agent has been updated to periodically try to re-read Kubernetes authentication token value from /var/run/secrets/kubernetes.io/serviceaccount/token file on disk (every 5 minutes by default). This way agent also supports Kubernetes deployments where token files are periodically automatically refreshed / rotated (e.g. https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection).
• Fix an edge case with handling empty resourceVersion value in the Kubernetes Events Monitor.

Docker images:

• Upgrade Python used by Docker images from 3.8.12 to 3.8.13.

Bug fixes:

• Fix minimum Python version detection in the deb/rpm package pre/post install script and make sure agent packages also support Python >= 3.10 (e.g. Ubuntu 22.04). Contributed by Arkadiusz Skalski (@askalski85).

Other:

• Update the code to log connection related errors which are retried and are not fatal under WARNING instead of ERROR log level.
• Update agent start up log message to use format which is supported by the agent message parser.

Auter

2.1.29 "Auter" - Mar 15, 2022

Docker images:

• Kubernetes Docker image (scalyr-k8s-agent) has been updated to ignore checkpoint data for ephemeral log files (anything matching /var/log/scalyr-agent-2/*.log) on agent start up. Those log files are ephemeral (aka only available during container runtime) which means we don't want to re-use checkpoints for those log files across pod restarts (recreations). Previously, those checkpoints were preserved across restarts which meant that on subsequent pod restarts some of the early internal agent log messages produced by the agent during start up phase were not ingested.

Bug fixes:

• Kubernetes monitor now correctly dynamically detects pod metadata changes (e.g. annotations) when using containerd runtime. Previously metadata updates were not detected dynamically which meant agent needed to be restarted to pick up any metadata changes (such as Scalyr related annotations).

Dryria

2.1.28 "Dryria" - Feb 23, 2022

Improvements:

• Docker and Kubernetes monitors now support parsing date and time information from the container log lines with custom timezones (aka non UTC).

Docker images:

• Docker images now include udatetime time dependency which should speed up parsing date and time information from Docker container log lines.
• Upgrade zstandard and orjson dependency used by the Docker image.

Thonia

Improvements:

• Add new debug_level_logger_names config option. With this config option user can specify a list of logger names for which to set debug log level for. Previously user could only set debug level for all the loggers. This comes handy when we want to set debug level for a single of subset of loggers (e.g. for a single monitor).
• If profiling is enabled (enable_profiling configuration option), memory profiling data will be ingested by default (CPU profiling data was already being ingested when profiling was enabled, but not memory one).

Docker images:

• Upgrade Python used by Docker images from 3.8.10 to 3.8.12.
• Base distribution version for non slim Alpine Linux based images has been upgraded from Debian Buster (10) to Debian Bullseye (11).

Bug fixes:

• Fix a bug with the URL monitor not working correctly with POST methods when request_data was specified under Python 3.