Skip to content

Commit 99ec79c

Browse files
author
Sumedh Wale
committed
Updated dependencies to address major CVEs
- updated LICENSE and NOTICE files accordingly - updated sub-module links - fixed formatting in release notes
1 parent be8d170 commit 99ec79c

File tree

10 files changed

+209
-764
lines changed

10 files changed

+209
-764
lines changed

LICENSE

Lines changed: 65 additions & 592 deletions
Large diffs are not rendered by default.

NOTICE

Lines changed: 65 additions & 104 deletions
Large diffs are not rendered by default.

build.gradle

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -30,15 +30,14 @@ buildscript {
3030
classpath 'de.undercouch:gradle-download-task:3.4.3'
3131
classpath 'net.rdrei.android.buildtimetracker:gradle-plugin:0.11.+'
3232
classpath 'com.netflix.nebula:gradle-ospackage-plugin:5.2.+'
33-
// classpath 'org.owasp:dependency-check-gradle:6.5.3'
33+
classpath 'org.owasp:dependency-check-gradle:7.1.0.1'
3434
}
3535
}
3636

3737
apply plugin: 'wrapper'
3838
apply plugin: 'distribution'
3939
apply plugin: 'nebula.ospackage-base'
4040
apply plugin: "nebula.ospackage"
41-
// apply plugin: 'org.owasp.dependencycheck'
4241

4342
// def isEnterpriseProduct = rootProject.hasProperty('snappydata.enterprise')
4443

@@ -70,6 +69,7 @@ allprojects {
7069
apply plugin: 'com.github.johnrengelman.shadow'
7170
apply plugin: 'idea'
7271
apply plugin: "build-time-tracker"
72+
apply plugin: 'org.owasp.dependencycheck'
7373

7474
group = 'io.snappydata'
7575
version = '1.3.1'
@@ -112,7 +112,7 @@ allprojects {
112112
snappySparkMetricsLibVersion = '2.0.0.1'
113113
log4j2Version = '2.17.2'
114114
slf4jVersion = '1.7.32'
115-
junitVersion = '4.12'
115+
junitVersion = '4.13.2'
116116
mockitoVersion = '1.10.19'
117117
hadoopVersion = '3.2.0'
118118
awsSdkVersion = '1.11.375'
@@ -126,8 +126,8 @@ allprojects {
126126
fastutilVersion = '8.5.6'
127127
kryoVersion = '4.0.1'
128128
thriftVersion = '0.9.3'
129-
jacksonVersion = '2.13.1'
130-
jacksonDatabindVersion = '2.13.1'
129+
jacksonVersion = '2.13.3'
130+
jacksonDatabindVersion = '2.13.3'
131131
hiveVersion = '1.21.2.7.0.3.2-3'
132132
metricsVersion = '4.0.3'
133133
metrics2Version = '2.2.0'
@@ -140,6 +140,7 @@ allprojects {
140140
objenesisVersion = '3.0.1'
141141
rabbitMqVersion = '4.9.1'
142142
akkaVersion = '2.3.16'
143+
nettyAkkaVersion = '3.10.6.Final'
143144
sprayVersion = '1.3.4'
144145
sprayJsonVersion = '1.3.5'
145146
sprayShapelessVersion = '1.3.3'
@@ -148,7 +149,7 @@ allprojects {
148149
jodaTimeVersion = '2.10.1'
149150
slickVersion = '2.1.0'
150151
h2Version = '1.3.176'
151-
commonsIoVersion = '2.6'
152+
commonsIoVersion = '2.11.0'
152153
commonsPoolVersion = '1.6'
153154
dbcpVersion = '1.4'
154155
shiroVersion = '1.2.6'
@@ -990,7 +991,7 @@ gradle.taskGraph.whenReady { graph ->
990991
Set<String> skipPublishFor = [
991992
'snappydata_' + scalaBinaryVersion, 'snappy-spark', 'snappy-store',
992993
'snappy-dtests_' + scalaBinaryVersion, 'snappy-compatibility-tests_' + scalaBinaryVersion,
993-
'snappydata-native', 'snappydata-store-prebuild',
994+
'gemfire-junit', 'snappydata-native', 'snappydata-store-prebuild',
994995
'snappy-spark-assembly_' + scalaBinaryVersion
995996
]
996997

cluster/build.gradle

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,11 @@ dependencies {
171171
//exclude(group: 'org.scala-lang', module: 'scala-library')
172172
//exclude(group: 'org.apache.avro', module: 'avro')
173173
}
174-
compile(group:'com.google.cloud.bigdataoss', name:'gcs-connector', version: gcsHadoop3ConnectorVersion, classifier: 'shaded')
174+
compile(group:'com.google.cloud.bigdataoss', name:'gcs-connector', version: gcsHadoop3ConnectorVersion, classifier: 'shaded') {
175+
// shaded jar includes all dependencies but the pom still lists all dependencies as per
176+
// the non-shaded jar, so skip all dependencies explicitly
177+
transitive = false
178+
}
175179

176180
testCompile project(':dunit')
177181
testCompile "org.eclipse.collections:eclipse-collections:${eclipseCollectionsVersion}"

core/src/dunit/scala/io/snappydata/cluster/SnappyJobTestSupport.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ trait SnappyJobTestSupport extends Logging {
5353
* exceeds specified time, the method will throw exception.
5454
*/
5555
def submitAndWaitForCompletion(classFullName: String, jobCmdAffix: String = "",
56-
waitTimeMillis: Int = 60000): Unit = {
56+
waitTimeMillis: Int = 120000): Unit = {
5757
val consoleLog: String = submitJob(classFullName, jobCmdAffix)
5858
logInfo("Job submission response:" + consoleLog)
5959
val jobId = getJobId(consoleLog)

docs/release_notes/release_notes.md

Lines changed: 61 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ SnappyData 1.3.1 release includes the following new features over the previous 1
7676
to the latest Log4J 2.x (2.17.2) from the previous Log4J 1.x. Patches were ported for the Spark components
7777
(where support for Log4J 2.x will land only with the 3.3.0 release), while other components were updated
7878
to use Log4J/SLF4J. The Spark connector component supports both Log4J 2.x and Log4J 1.x to allow compatibility
79-
with upstream Spark releases which the SnappyData's Spark distribution only uses Log4J 2.x.
79+
with upstream Spark releases while the SnappyData's Spark distribution only uses Log4J 2.x.
8080

8181

8282
## Stability and Security Improvements
@@ -88,68 +88,74 @@ SnappyData 1.3.1 includes the following changes to improve stability and securit
8888
* Fixed a race condition in old entries cleaner thread deleting in-use snapshot entries.
8989

9090
* Allow retry in startup failure even for data nodes. In some rare cases region initialization may fail
91-
due to colocated region still being initialized, so retry region initialization.
91+
due to colocated region still being initialized, so retry region initialization for such cases.
9292

93-
* Fixed UDF names lookups to do exact regex match in the CSV list in the meta-data region.
93+
* Fixed UDF name lookups to do exact regex match in the CSV list in the meta-data region.
9494

9595
* Apart from Log4J, following dependencies were updated to address known security issues:
96-
- Jetty upgraded to 9.4.44.v20210927
97-
- jackson-mapper-asl and jackson-core-asl upgraded to 1.9.14-atlassian-6
98-
- jackson and jackson-databind upgraded to 2.13.1
99-
- Kafka upgraded to 2.2.2
100-
- [SPARK-34110](https://issues.apache.org/jira/browse/SPARK-34110): Upgrade Zookeeper to 3.6.2
101-
- [SPARK-37901](https://issues.apache.org/jira/browse/SPARK-37901): Upgrade Netty to 4.1.73
102-
- gcs-hadoop-connector upgraded to hadoop3-2.1.2
96+
- Jetty upgraded to 9.4.44.v20210927
97+
- jackson-mapper-asl and jackson-core-asl upgraded to 1.9.14-atlassian-6
98+
- jackson and jackson-databind upgraded to 2.13.1
99+
- Kafka upgraded to 2.2.2
100+
- [SPARK-34110](https://issues.apache.org/jira/browse/SPARK-34110): Upgrade Zookeeper to 3.6.2
101+
- [SPARK-37901](https://issues.apache.org/jira/browse/SPARK-37901): Upgrade Netty to 4.1.73
102+
- gcs-hadoop-connector upgraded to hadoop3-2.1.2
103103

104104
* Ported patches for the following issues from Apache Geode:
105-
- [GEODE-1252](https://issues.apache.org/jira/browse/GEODE-1252): Modify bits field atomically
106-
- [GEODE-2802](https://issues.apache.org/jira/browse/GEODE-2802): Tombstone version vector to contain only
105+
- [GEODE-1252](https://issues.apache.org/jira/browse/GEODE-1252): Modify bits field atomically
106+
- [GEODE-2802](https://issues.apache.org/jira/browse/GEODE-2802): Tombstone version vector to contain only
107107
the members that generate the tombstone
108-
- [GEODE-5278](https://issues.apache.org/jira/browse/GEODE-5278): Unexpected CommitConflictException caused by
108+
- [GEODE-5278](https://issues.apache.org/jira/browse/GEODE-5278): Unexpected CommitConflictException caused by
109109
faulty region synchronization
110-
- [GEODE-4083](https://issues.apache.org/jira/browse/GEODE-4083): Fix infinite loop caused by thread race
111-
changing version
112-
- [GEODE-3796](https://issues.apache.org/jira/browse/GEODE-3796): Changes are made to validate region version
113-
after the region is initialized
114-
- [GEODE-6058](https://issues.apache.org/jira/browse/GEODE-6058): recordVersion should allow update higher local
115-
version if for non-persistent region
116-
- [GEODE-6013](https://issues.apache.org/jira/browse/GEODE-6013): Use expected initial image requester's
117-
rvv information
118-
- [GEODE-2159](https://issues.apache.org/jira/browse/GEODE-2159): Add serialVersionUIDs to exception classes
119-
not having them
120-
- [GEODE-5559](https://issues.apache.org/jira/browse/GEODE-5559): Improve runtime of
121-
RegionVersionHolder.canonicalExceptions
122-
- [GEODE-5612](https://issues.apache.org/jira/browse/GEODE-5612): Fix RVVExceptionB.writeReceived()
123-
- [GEODE-7085](https://issues.apache.org/jira/browse/GEODE-7085): Ensure that the bitset stays within
124-
BIT_SET_WIDTH and is flushed in all code paths
125-
- GFE-50415: Wait for membership change in persistence advisor can hang if the event was missed
126-
- [GEODE-5111](https://issues.apache.org/jira/browse/GEODE-5111): Set offline members to null only when done
127-
waiting for them
110+
- [GEODE-4083](https://issues.apache.org/jira/browse/GEODE-4083): Fix infinite loop caused
111+
by thread race changing version
112+
- [GEODE-3796](https://issues.apache.org/jira/browse/GEODE-3796): Changes are made to
113+
validate region version after the region is initialized
114+
- [GEODE-6058](https://issues.apache.org/jira/browse/GEODE-6058): recordVersion should
115+
allow update higher local version if for non-persistent region
116+
- [GEODE-6013](https://issues.apache.org/jira/browse/GEODE-6013): Use expected initial
117+
image requester's rvv information
118+
- [GEODE-2159](https://issues.apache.org/jira/browse/GEODE-2159): Add serialVersionUIDs to
119+
exception classes not having them
120+
- [GEODE-5559](https://issues.apache.org/jira/browse/GEODE-5559): Improve runtime of
121+
RegionVersionHolder.canonicalExceptions
122+
- [GEODE-5612](https://issues.apache.org/jira/browse/GEODE-5612):
123+
Fix RVVExceptionB.writeReceived()
124+
- [GEODE-7085](https://issues.apache.org/jira/browse/GEODE-7085): Ensure that the bitset
125+
stays within BIT_SET_WIDTH and is flushed in all code paths
126+
- GFE-50415: Wait for membership change in persistence advisor can hang if the member
127+
join event was missed
128+
- [GEODE-5111](https://issues.apache.org/jira/browse/GEODE-5111): Set offline members to
129+
null only when done waiting for them
128130

129131
* Merged patches for the following Spark issues:
130-
- [SPARK-6305](https://issues.apache.org/jira/browse/SPARK-6305): Migrate from log4j1 to log4j2
131-
- Followups SPARK-37684, SPARK-37774 to upgrade log4j to 2.17.x
132-
- [SPARK-37791](https://issues.apache.org/jira/browse/SPARK-37791): Use log4j2 in examples
133-
- [SPARK-37794](https://issues.apache.org/jira/browse/SPARK-37794): Remove internal log4j bridge api usage
134-
- [SPARK-37746](https://issues.apache.org/jira/browse/SPARK-37746): log4j2-defaults.properties is not working
135-
since log4j 2 is always initialized by default
136-
- [SPARK-37792](https://issues.apache.org/jira/browse/SPARK-37792): Fix the check of custom configuration in
137-
SparkShellLoggingFilter
138-
- [SPARK-37795](https://issues.apache.org/jira/browse/SPARK-37795): Add a scalastyle rule to ban `org.apache.log4j`
139-
imports
140-
- [SPARK-37805](https://issues.apache.org/jira/browse/SPARK-37805): Refactor `TestUtils#configTestLog4j` method
141-
to use log4j2 api
142-
- [SPARK-37889](https://issues.apache.org/jira/browse/SPARK-37889): Replace Log4j2 MarkerFilter with RegexFilter
143-
- [SPARK-26267](https://issues.apache.org/jira/browse/SPARK-26267): Retry when detecting incorrect offsets
144-
from Kafka
145-
- [SPARK-37729](https://issues.apache.org/jira/browse/SPARK-37729): Fix SparkSession.setLogLevel that is not
146-
working in Spark Shell
147-
- [SPARK-37887](https://issues.apache.org/jira/browse/SPARK-37887): Fix the check of repl log level
148-
- [SPARK-37790](https://issues.apache.org/jira/browse/SPARK-37790): Upgrade SLF4J to 1.7.32
149-
- [SPARK-22324](https://issues.apache.org/jira/browse/SPARK-22324): Upgrade Arrow to 0.8.0
150-
- [SPARK-25598](https://issues.apache.org/jira/browse/SPARK-25598): Remove flume connector in Spark
151-
- [SPARK-37693](https://issues.apache.org/jira/browse/SPARK-37693): Fix ChildProcAppHandleSuite failed in Jenkins
152-
maven test
132+
- [SPARK-6305](https://issues.apache.org/jira/browse/SPARK-6305): Migrate from log4j1 to log4j2
133+
- Followups SPARK-37684, SPARK-37774 to upgrade log4j to 2.17.x
134+
- [SPARK-37791](https://issues.apache.org/jira/browse/SPARK-37791): Use log4j2 in examples
135+
- [SPARK-37794](https://issues.apache.org/jira/browse/SPARK-37794): Remove internal log4j
136+
bridge api usage
137+
- [SPARK-37746](https://issues.apache.org/jira/browse/SPARK-37746):
138+
log4j2-defaults.properties is not working since log4j 2 is always initialized by default
139+
- [SPARK-37792](https://issues.apache.org/jira/browse/SPARK-37792): Fix the check of
140+
custom configuration in SparkShellLoggingFilter
141+
- [SPARK-37795](https://issues.apache.org/jira/browse/SPARK-37795): Add a scalastyle rule
142+
to ban `org.apache.log4j` imports
143+
- [SPARK-37805](https://issues.apache.org/jira/browse/SPARK-37805):
144+
Refactor `TestUtils#configTestLog4j` method to use log4j2 api
145+
- [SPARK-37889](https://issues.apache.org/jira/browse/SPARK-37889): Replace Log4j2
146+
MarkerFilter with RegexFilter
147+
- [SPARK-26267](https://issues.apache.org/jira/browse/SPARK-26267): Retry when detecting
148+
incorrect offsets from Kafka
149+
- [SPARK-37729](https://issues.apache.org/jira/browse/SPARK-37729):
150+
Fix SparkSession.setLogLevel that is not working in Spark Shell
151+
- [SPARK-37887](https://issues.apache.org/jira/browse/SPARK-37887): Fix the check of REPL
152+
log level
153+
- [SPARK-37790](https://issues.apache.org/jira/browse/SPARK-37790): Upgrade SLF4J to 1.7.32
154+
- [SPARK-22324](https://issues.apache.org/jira/browse/SPARK-22324): Upgrade Arrow to 0.8.0
155+
- [SPARK-25598](https://issues.apache.org/jira/browse/SPARK-25598): Remove flume connector
156+
in Spark
157+
- [SPARK-37693](https://issues.apache.org/jira/browse/SPARK-37693):
158+
Fix ChildProcAppHandleSuite failed in Jenkins maven test
153159

154160

155161
## Resolved Issues
@@ -162,7 +168,7 @@ SnappyData 1.3.1 resolves the following major issues apart from the patches note
162168
## Known Issues
163169

164170
The known issues noted in [1.3.0 release notes](https://tibcosoftware.github.io/snappydata/1.3.0/release_notes/release_notes/#known-issues)
165-
still apply in 1.3.1 release.
171+
still apply in 1.3.1 release. These have been reproduced below for reference:
166172

167173
| Key | Item | Description | Workaround |
168174
| --- | ---- | ----------- | ---------- |

spark

Submodule spark updated from cba053b to 0169c3d

spark-jobserver

0 commit comments

Comments
 (0)