Yavin

Improvements:

• Add collect_replica_metrics config option to the MySQL monitor. When this option is set to False (for backward compatibility reasons it defaults to True) we don't try to collect replica metrics and as such, user which is used to connect to the database only needs PROCESS permissions and nothing else.
• Add support for specifying new server_url config option for each session worker. This allows user to configure different session workers to use different server urls (e.g. some workers send data to agent.scalyr.com and other send data to eu.scalyr.com).

Other:

• Update agent to also log response message field value when we receive error/client/badParam status code from the API to make troubleshooting easier.

Docker Images:

• Docker images now utilize Python 3.8 (in the past they used Python 2.7).
• Docker images are now also produced and pushed to the registry for the linux/arm64 and linux/arm/v7 architecture.
• Docker image now includes pympler dependency by default. This means memory profiling can be enabled via the agent configuration option without the need to modify and re-build the Docker image.
• ujson dependency has been removed from the Docker image in favor of orjson which is more performant and now used by default.
• Alpine based Docker images which are 50% small than regular Debian buster-slim based ones are now available. Alpine based images contains -alpine tag name suffix.

v2.1.25

2.1.25 "Hamarus" - Nov 17, 2021

Other:

• Added a LetsEncrypt root certificate to the Agent's included certificate bundle.
• Update Kubernetes monitor to log a message under INFO log level that some container level metrics won't be available when detected runtime is containerd and not docker and container level metrics reporting is enabled.

Bug fixes:

• Fix plaintext mode in the Graphite monitor.
• Update Linux system metric monitor so it doesn't print an error and ignores /proc/net/netstat lines we don't support metrics for.

Xuntian

Improvements:

• Implemented optional line merging when running in Docker based Kubernetes to work around Docker's own 16KB line length limit. Use the merge_json_parsed_lines config option or SCALYR_MERGE_JSON_PARSED_LINES environment variable to enable this functionality.

Bug fixes:

• Update agent to throw more user-friendly error when we fail to parse fragment file due to invalid content.

Whipple

Bug fixes:

• Fix docker container builder scripts to only use buildx if it is available.
• Fix memory leak in the Syslog monitor which is caused by a bug in standard TCP/UDP servers in Python 3 (https://bugs.python.org/issue37193). The version of Python for Windows was updated to 3.8.10. For Linux users, who run agent on Python 3 and use Syslog monitor, it is also recommended to check if their Python 3 installation is up to date and has appropriate bug fixes.
• Fix bug in the copying manager which works with monitor (such as Docker or Syslog monitor). This bug might cause re-uploading of the old log messages in some edge cases.

Volans

Bug fixes:

• Don't log "skipping copying log lines" messages in case number of last produced bytes is 0.
• Fix Kubernetes Agent DaemonSet liveness probe timeout too short and unhealthy agent pod not restarting when a liveness probe timeout occurs.

Other:

• Update Windows Event Log monitor to emit a warning if maximum_records_per_source config option is set to a non-default value when using a new event API where that config option has no affect.

Ultramarine

Improvements:

• Allow journald monitor to be configured via globs on multiple fields via journald_logs.journald_globs config option.

Bug fixes:

• Fix an issue where log lines may be duplicated or lost in the Kubernetes monitor when running under CRI with an unstable connection to the K8s API.
• Fix an issue where LogFileIterator during the copy-truncate process picks wrong pending file with a similar name causing loss of the log and showing negative bytes in agent status.

Other:

• The discovery logic of the log files, which have been rotated with copy truncate method, now can handle only default rogrotate configurations.
• Agent now emits a warning if running under Python 2.6 which we will stop supporting in the next release.

Tabeisshi

Improvements:

• Add support for collecting some metrics Kubernetes when running in a CRI runtime.

Bug fixes:

• Ensure pod digest which we calculate and use to determine if pod info in the Kubernetes monitor has changed is deterministic and doesn't depend on dictionary item ordering.
• Fix a race condition in the worker session checkpoint read/write logic, which was introduced with the multi-worker feature.

Other:

• Changed the logging level of "not found" errors while querying pods from the Kubernetes API from ERROR to DEBUG, as these errors are always transient and result in no data loss.

StarTram

Improvements:

• Add Endpoint to the default event_object_filter values for the Kubernetes Event Monitor.
• Update scalyr-agent-2 status -v output to also include process id (pid) of the main agent process and children worker processes in scenarios where the agent is configured with multiple worker processes.
• Add a new experimental --systemd-managed flag to the scalyr-agent-2-config tool which converts existing scalyr agent installation to be managed by systemd instead of init.d.
• Add support for capturing k8s container names to the kubernetes monitor via log line attributes. Container name can be captured using ${k8s_container_name} config template syntax.
• Kubernetes monitor has been updated to log 403 errors when connecting to the Kubernetes API before falling back to the old fallback URL under warning log level. Previously those errors were logged under debug log level which would make it hard to troubleshoot some issues related to missing permissions, etc. since those log messages would only end up in debug log file which is disabled by default.

Bug fixes:

• Fix scalyr-agent-status -v to not emit / print warnings under some edge cases.
• Fix regression with syslog monitor which caused the ingestion of the file with a wrong parser. The root cause was in the too broad path glob that was used in the agent log's LogMatcher, so it also processed the agent_syslog.log as an agent log file.

Ravis

Improvements:

• Add new tcp_request_parser and tcp_message_delimiter config option to the syslog_monitor. Valid values for tcp_request_parser include default and batch. New TCP recv batch oriented request parser is much more efficient than the default one and should be a preferred choice in most situations. For backward compatibility reasons, the default parser hasn't been changed yet.
• Update agent to emit a warning if k8s_logs config option is defined, but Kubernetes monitor is not enabled / configured.
• Update Kubernetes and Docker monitor to not propagate and show some non-fatal errors.
• Field values in log lines for monitor metrics which contain extra fields are now sorted in alphabetic order. This should have no impact on the end user since server side parsers already support arbitrary ordering, but it's done to ensure consistent ordering and output for for monitor log lines.
• Update agent to throw more user-friendly exceptions on Windows when the agent doesn't have access to the agent config file and Windows registry.
• Update code which periodically prints out useful configuration settings to also include actual value of the json library used in case the config option is set to "auto".

Bug fixes:

• Fix a race condition in docker_monitor which could cause the monitor to throw exception on start up.
• Fix a config deprecated options bug when they are set to false.
• Fix agent so it doesn't throw an exception on Windows when trying to escalate permissions on agent start.
• Make sure we only print the value of win32_max_open_fds config option on Windows if it has changed.
• Fix a bug which could result in docker, journald and windows event log checkpoint files to be deleted when restarting the agent. This would only affect docker monitor configurations which are setup to ingest logs via Docker API and not json log files on disk (aka docker_raw_logs: false docker monitor option is set).
• Fix a small bug which might skip first lines of the worker session log file.

Other:

• Pre-built frozen Windows binaries now bundle and utilize Python 3 (3.8.7). In the past releases, they utilized Python 2.7, but due to the Python 2.7 being EOL for a year now, new release will utilize Python 3. This change should be opaque to the end user and everything should continue to work as it did in the previous releases.

Xothichi

Bug fixes:

• Fix syslog monitor default TCP message parser bug which was inadvertently introduced in 2.1.16 which may sometimes cause for a delayed ingest of some syslog data. This would only affect installations utilizing syslog monitor in TCP mode using the default message parser. As part of this fix, we have removed the new message parsers added in 2.1.16. If you were using them, you will revert to the default parser.