Skip to content

Commit b4ead5e

Browse files
authored
fix: bump netty to 4.1.118 and drop duplicate pyspark in mmlspark/release demo image (#2557)
Addresses MSRC case 110886 / incident 31000000570827. The mmlspark/release image (built from tools/docker/demo/Dockerfile) ships Spark 3.5.4, which pins netty 4.1.96.Final. That version is flagged for multiple CVEs (CVE-2023-44487, CVE-2024-29025, CVE-2025-24970, ...). Spark has not bumped netty in any 3.5.x release. netty 4.1.x is binary-compatible, so we replace all netty-*-4.1.96.Final*.jar files in /opt/spark/jars/ with 4.1.118.Final right after the Spark extract. This includes netty-codec-http2 (the specific artifact named by the finder). Also removes 'pyspark' from the conda install line. It was pulling a complete second Spark install (PySpark 4.0.1) into /usr/local/lib/python*/site-packages/pyspark/ that nothing in the demo image actually used (SPARK_HOME points at /opt/spark) and that doubled the surface area scanners report on. Validated locally: - /opt/spark/jars/netty-*-4.1.96.Final*.jar: 0 matches after build - /opt/spark/jars/netty-*-4.1.118.Final*.jar: full set present - /usr/local/lib/.../pyspark: no longer exists - spark-submit --version: works - spark.range(5).count(): returns 5 Jetty (shaded inside hadoop-client-runtime-3.3.4.jar at 9.4.43) is OUT OF SCOPE for this PR; that requires a Spark/Hadoop swap and will be tracked separately.
1 parent d0bbeae commit b4ead5e

1 file changed

Lines changed: 16 additions & 1 deletion

File tree

tools/docker/demo/Dockerfile

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ RUN curl -sSL https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64
3030
&& conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main \
3131
&& conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r \
3232
&& conda update -y conda \
33-
&& conda install -y python=3 jupyter pyspark \
33+
&& conda install -y python=3 jupyter \
3434
&& pip install --upgrade "PyJWT>=2.12.0" \
3535
&& conda clean --all --yes
3636

@@ -42,6 +42,21 @@ RUN wget https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/spark-${SP
4242
&& mv spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION} /opt/spark \
4343
&& rm spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz
4444

45+
# Patch netty 4.1.96.Final (CVE-2023-44487, CVE-2024-29025, CVE-2025-24970, ...) to 4.1.118.Final.
46+
# Spark 3.5.x pins netty 4.1.96 upstream; we override in-place since 4.1.x is binary-compatible.
47+
ENV NETTY_VERSION=4.1.118.Final
48+
RUN cd /opt/spark/jars \
49+
&& rm -f netty-*-4.1.96.Final*.jar \
50+
&& for c in all buffer codec codec-http codec-http2 codec-socks common handler handler-proxy resolver transport transport-classes-epoll transport-classes-kqueue transport-native-unix-common; do \
51+
curl -fsSLO "https://repo1.maven.org/maven2/io/netty/netty-${c}/${NETTY_VERSION}/netty-${c}-${NETTY_VERSION}.jar"; \
52+
done \
53+
&& for cls in linux-x86_64 linux-aarch_64; do \
54+
curl -fsSLO "https://repo1.maven.org/maven2/io/netty/netty-transport-native-epoll/${NETTY_VERSION}/netty-transport-native-epoll-${NETTY_VERSION}-${cls}.jar"; \
55+
done \
56+
&& for cls in osx-x86_64 osx-aarch_64; do \
57+
curl -fsSLO "https://repo1.maven.org/maven2/io/netty/netty-transport-native-kqueue/${NETTY_VERSION}/netty-transport-native-kqueue-${NETTY_VERSION}-${cls}.jar"; \
58+
done
59+
4560
ENV SPARK_HOME /opt/spark
4661
ENV PYTHONPATH $SPARK_HOME/python/:$SPARK_HOME/python/lib/py4j*:$PYTHON_PATH
4762
ENV PATH $SPARK_HOME/bin/:$SPARK_HOME/python/:$PATH

0 commit comments

Comments
 (0)