Merge pull request #2426 from facebook/changelog147

Cyan4973 · web-flow · commit 645a2975c394 · 2020-12-16T15:00:18.000-08:00
Updated CHANGELOG for v1.4.7
diff --git a/CHANGELOG b/CHANGELOG
@@ -1,3 +1,26 @@
+v1.4.7
+perf: stronger --long mode at high compression levels, by @senhuang42
+perf: stronger --patch-from at high compression levels, thanks to --long improvements
+perf: faster dictionary compression at medium compression levels, by @felixhandte
+perf: small speed & memory usage improvements for ZSTD_compress2(), by @terrelln
+perf: improved fast compression speeds with Visual Studio, by @animalize
+cli : Set nb of threads with environment variable ZSTD_NBTHREADS, by @senhuang42
+cli : accept decompressing files with *.zstd suffix
+cli : provide a condensed summary by default when processing multiple files
+cli : fix : stdin input no longer confused as user prompt
+cli : improve accuracy of several error messages
+api : new sequence ingestion API, by @senhuang42
+api : shared thread pool: control total nb of threads used by multiple compression jobs, by @marxin
+api : new ZSTD_getDictID_fromCDict(), by @LuAPi
+api : zlibWrapper only uses public API, and is compatible with dynamic library, by @terrelln
+api : fix : multithreaded compression has predictable output even in special cases (see #2327) (issue not accessible from cli)
+api : fix : dictionary compression correctly respects dictionary compression level (see #2303) (issue not accessible from cli)
+build: fix cmake script when using path with spaces, by @terrelln
+build: improved compile-time detection of aarch64/neon platforms, by @bsdimp
+build: Fix building on AIX 5.1, by @likema
+build: compile paramgrill with cmake on Windows, requested by @mirh
+doc : clarify repcode updates in format specification, by @felixhandte
+
 v1.4.6
 fix : Always return dstSize_tooSmall when that is the case
 fix : Fix ZSTD_initCStream_advanced() with static allocation and no dictionary
diff --git a/doc/zstd_manual.html b/doc/zstd_manual.html
@@ -1,10 +1,10 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>zstd 1.4.6 Manual</title>
+<title>zstd 1.4.7 Manual</title>
 </head>
 <body>
-<h1>zstd 1.4.6 Manual</h1>
+<h1>zstd 1.4.7 Manual</h1>
 <hr>
 <a name="Contents"></a><h2>Contents</h2>
 <ol>
diff --git a/lib/zstd.h b/lib/zstd.h
@@ -72,7 +72,7 @@ extern "C" {
 /*------   Version   ------*/
 #define ZSTD_VERSION_MAJOR    1
 #define ZSTD_VERSION_MINOR    4
-#define ZSTD_VERSION_RELEASE  6
+#define ZSTD_VERSION_RELEASE  7
 #define ZSTD_VERSION_NUMBER  (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
 
 /*! ZSTD_versionNumber() :
diff --git a/programs/zstd.1 b/programs/zstd.1
@@ -1,5 +1,5 @@
 .
-.TH "ZSTD" "1" "July 2020" "zstd 1.4.5" "User Commands"
+.TH "ZSTD" "1" "December 2020" "zstd 1.4.7" "User Commands"
 .
 .SH "NAME"
 \fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files
@@ -165,7 +165,7 @@ This is also used during compression when using with \-\-patch\-from=\. In this
 \fB\-\-[no\-]sparse\fR: enable / disable sparse FS support, to make files with many zeroes smaller on disk\. Creating sparse files may save disk space and speed up decompression by reducing the amount of disk I/O\. default: enabled when output is into a file, and disabled when output is stdout\. This setting overrides default and can force sparse mode over stdout\.
 .
 .IP "\(bu" 4
-\fB\-\-rm\fR: remove source file(s) after successful compression or decompression
+\fB\-\-rm\fR: remove source file(s) after successful compression or decompression\. If used in combination with \-o, will trigger a confirmation prompt (which can be silenced with \-f), as this is a destructive operation\.
 .
 .IP "\(bu" 4
 \fB\-k\fR, \fB\-\-keep\fR: keep source file(s) after successful compression or decompression\. This is the default behavior\.
@@ -212,7 +212,16 @@ If input directory contains "\.\.", the files in this directory will be ignored\
 .IP "" 0
 .
 .SS "Restricted usage of Environment Variables"
-Using environment variables to set parameters has security implications\. Therefore, this avenue is intentionally restricted\. Only \fBZSTD_CLEVEL\fR is supported currently, for setting compression level\. \fBZSTD_CLEVEL\fR can be used to set the level between 1 and 19 (the "normal" range)\. If the value of \fBZSTD_CLEVEL\fR is not a valid integer, it will be ignored with a warning message\. \fBZSTD_CLEVEL\fR just replaces the default compression level (\fB3\fR)\. It can be overridden by corresponding command line arguments\.
+Using environment variables to set parameters has security implications\. Therefore, this avenue is intentionally restricted\. Only \fBZSTD_CLEVEL\fR and \fBZSTD_NBTHREADS\fR are currently supported\. They set the compression level and number of threads to use during compression, respectively\.
+.
+.P
+\fBZSTD_CLEVEL\fR can be used to set the level between 1 and 19 (the "normal" range)\. If the value of \fBZSTD_CLEVEL\fR is not a valid integer, it will be ignored with a warning message\. \fBZSTD_CLEVEL\fR just replaces the default compression level (\fB3\fR)\.
+.
+.P
+\fBZSTD_NBTHREADS\fR can be used to set the number of threads \fBzstd\fR will attempt to use during compression\. If the value of \fBZSTD_NBTHREADS\fR is not a valid unsigned integer, it will be ignored with a warning message\. \'ZSTD_NBTHREADS\fBhas a default value of (\fR1\fB), and is capped at ZSTDMT_NBWORKERS_MAX==200\.\fRzstd` must be compiled with multithread support for this to have any effect\.
+.
+.P
+They can both be overridden by corresponding command line arguments: \fB\-#\fR for compression level and \fB\-T#\fR for number of compression threads\.
 .
 .SH "DICTIONARY BUILDER"
 \fBzstd\fR offers \fIdictionary\fR compression, which greatly improves efficiency on small files and messages\. It\'s possible to train \fBzstd\fR with a set of samples, the result of which is saved into a file called a \fBdictionary\fR\. Then during compression and decompression, reference the same dictionary, using command \fB\-D dictionaryFileName\fR\. Compression of small files similar to the sample set will be greatly improved\.
diff --git a/programs/zstdgrep.1 b/programs/zstdgrep.1
@@ -1,5 +1,5 @@
 .
-.TH "ZSTDGREP" "1" "July 2020" "zstd 1.4.5" "User Commands"
+.TH "ZSTDGREP" "1" "December 2020" "zstd 1.4.7" "User Commands"
 .
 .SH "NAME"
 \fBzstdgrep\fR \- print lines matching a pattern in zstandard\-compressed files
diff --git a/programs/zstdless.1 b/programs/zstdless.1
@@ -1,5 +1,5 @@
 .
-.TH "ZSTDLESS" "1" "July 2020" "zstd 1.4.5" "User Commands"
+.TH "ZSTDLESS" "1" "December 2020" "zstd 1.4.7" "User Commands"
 .
 .SH "NAME"
 \fBzstdless\fR \- view zstandard\-compressed files

Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,5 @@`
`1`	`1`	`.`
`2`		`-.TH "ZSTD" "1" "July 2020" "zstd 1.4.5" "User Commands"`
	`2`	`+.TH "ZSTD" "1" "December 2020" "zstd 1.4.7" "User Commands"`
`3`	`3`	`.`
`4`	`4`	`.SH "NAME"`
`5`	`5`	`\fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files`
`@@ -165,7 +165,7 @@ This is also used during compression when using with \-\-patch\-from=\. In this`
`165`	`165`	`\fB\-\-[no\-]sparse\fR: enable / disable sparse FS support, to make files with many zeroes smaller on disk\. Creating sparse files may save disk space and speed up decompression by reducing the amount of disk I/O\. default: enabled when output is into a file, and disabled when output is stdout\. This setting overrides default and can force sparse mode over stdout\.`
`166`	`166`	`.`
`167`	`167`	`.IP "\(bu" 4`
`168`		`-\fB\-\-rm\fR: remove source file(s) after successful compression or decompression`
	`168`	`+\fB\-\-rm\fR: remove source file(s) after successful compression or decompression\. If used in combination with \-o, will trigger a confirmation prompt (which can be silenced with \-f), as this is a destructive operation\.`
`169`	`169`	`.`
`170`	`170`	`.IP "\(bu" 4`
`171`	`171`	`\fB\-k\fR, \fB\-\-keep\fR: keep source file(s) after successful compression or decompression\. This is the default behavior\.`
`@@ -212,7 +212,16 @@ If input directory contains "\.\.", the files in this directory will be ignored\`
`212`	`212`	`.IP "" 0`
`213`	`213`	`.`
`214`	`214`	`.SS "Restricted usage of Environment Variables"`
`215`		-Using environment variables to set parameters has security implications\. Therefore, this avenue is intentionally restricted\. Only \fBZSTD_CLEVEL\fR is supported currently, for setting compression level\. \fBZSTD_CLEVEL\fR can be used to set the level between 1 and 19 (the "normal" range)\. If the value of \fBZSTD_CLEVEL\fR is not a valid integer, it will be ignored with a warning message\. \fBZSTD_CLEVEL\fR just replaces the default compression level (\fB3\fR)\. It can be overridden by corresponding command line arguments\.
	`215`	`+Using environment variables to set parameters has security implications\. Therefore, this avenue is intentionally restricted\. Only \fBZSTD_CLEVEL\fR and \fBZSTD_NBTHREADS\fR are currently supported\. They set the compression level and number of threads to use during compression, respectively\.`
	`216`	`+.`
	`217`	`+.P`
	`218`	`+\fBZSTD_CLEVEL\fR can be used to set the level between 1 and 19 (the "normal" range)\. If the value of \fBZSTD_CLEVEL\fR is not a valid integer, it will be ignored with a warning message\. \fBZSTD_CLEVEL\fR just replaces the default compression level (\fB3\fR)\.`
	`219`	`+.`
	`220`	`+.P`
	`221`	+\fBZSTD_NBTHREADS\fR can be used to set the number of threads \fBzstd\fR will attempt to use during compression\. If the value of \fBZSTD_NBTHREADS\fR is not a valid unsigned integer, it will be ignored with a warning message\. \'ZSTD_NBTHREADS\fBhas a default value of (\fR1\fB), and is capped at ZSTDMT_NBWORKERS_MAX==200\.\fRzstd` must be compiled with multithread support for this to have any effect\.
	`222`	`+.`
	`223`	`+.P`
	`224`	`+They can both be overridden by corresponding command line arguments: \fB\-#\fR for compression level and \fB\-T#\fR for number of compression threads\.`
`216`	`225`	`.`
`217`	`226`	`.SH "DICTIONARY BUILDER"`
`218`	`227`	`\fBzstd\fR offers \fIdictionary\fR compression, which greatly improves efficiency on small files and messages\. It\'s possible to train \fBzstd\fR with a set of samples, the result of which is saved into a file called a \fBdictionary\fR\. Then during compression and decompression, reference the same dictionary, using command \fB\-D dictionaryFileName\fR\. Compression of small files similar to the sample set will be greatly improved\.`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,5 @@`
`1`	`1`	`.`
`2`		`-.TH "ZSTDGREP" "1" "July 2020" "zstd 1.4.5" "User Commands"`
	`2`	`+.TH "ZSTDGREP" "1" "December 2020" "zstd 1.4.7" "User Commands"`
`3`	`3`	`.`
`4`	`4`	`.SH "NAME"`
`5`	`5`	`\fBzstdgrep\fR \- print lines matching a pattern in zstandard\-compressed files`
Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,5 @@`
`1`	`1`	`.`
`2`		`-.TH "ZSTDLESS" "1" "July 2020" "zstd 1.4.5" "User Commands"`
	`2`	`+.TH "ZSTDLESS" "1" "December 2020" "zstd 1.4.7" "User Commands"`
`3`	`3`	`.`
`4`	`4`	`.SH "NAME"`
`5`	`5`	`\fBzstdless\fR \- view zstandard\-compressed files`