Skip to content

Commit e8b7b4e

Browse files
amoebakou
andauthored
GH-49686: [C++][FlightRPC][ODBC][Release] Create signing script for Windows FlightSQL ODBC build (#49788)
### Rationale for this change We need a script for the release manager to run during the release to locally sign the Windows artifacts for the FlightSQL ODBC driver. Ref: #49404 ### What changes are included in this PR? - New script, `07-flightsql-odbc-upload.sh` - Update to release guide - Renamed other release scripts to make space in the numbering scheme: - 07-publish-gh-release -> 08 - 08-binary-verify -> 09 - 08-vote-emai -> 10 ### Are these changes tested? Not 100% but I've tested each step separately. I tested on my fork using fake tags and releases. ### Are there any user-facing changes? No. * GitHub Issue: #49686 Lead-authored-by: Bryce Mecum <petridish@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Bryce Mecum <petridish@gmail.com>
1 parent acd8e44 commit e8b7b4e

8 files changed

Lines changed: 207 additions & 5 deletions

File tree

.pre-commit-config.yaml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -349,7 +349,8 @@ repos:
349349
?^cpp/examples/tutorial_examples/run\.sh$|
350350
?^cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc\.sh$|
351351
?^dev/release/05-binary-upload\.sh$|
352-
?^dev/release/08-binary-verify\.sh$|
352+
?^dev/release/07-flightsqlodbc-upload\.sh$|
353+
?^dev/release/09-binary-verify\.sh$|
353354
?^dev/release/binary-recover\.sh$|
354355
?^dev/release/post-03-binary\.sh$|
355356
?^dev/release/post-08-docs\.sh$|
@@ -376,6 +377,7 @@ repos:
376377
?^ci/scripts/python_test_type_annotations\.sh$|
377378
?^cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc\.sh$|
378379
?^dev/release/05-binary-upload\.sh$|
380+
?^dev/release/07-flightsqlodbc-upload\.sh$|
379381
?^dev/release/binary-recover\.sh$|
380382
?^dev/release/post-03-binary\.sh$|
381383
?^dev/release/post-08-docs\.sh$|

dev/release/.env.example

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,10 @@
3434
#
3535
# You must set this.
3636
#GH_TOKEN=secret
37+
38+
# For 07-flightsqlodbc-upload.sh. See that script for more details.
39+
#
40+
# ssl.com credentials in "username|password" format
41+
#ESIGNER_STOREPASS=username|password
42+
# ssl.com eSigner secret code (not the PIN)
43+
#ESIGNER_KEYPASS=otp_secret_code
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
#
20+
# FlightSQL ODBC Release Signing Script
21+
#
22+
# This script handles the signing of FlightSQL ODBC Windows binaries and MSI
23+
# installer. It requires jsign to be configured with ASF code signing
24+
# credentials. Keep reading below:
25+
#
26+
# Required environment variables:
27+
#
28+
# ESIGNER_STOREPASS - The ssl.com credentials in "username|password" format
29+
# ESIGNER_KEYPASS - The ssl.com eSigner secret code (not the PIN)
30+
#
31+
# Set these in .env.
32+
#
33+
# How to get ESIGNER_KEYPASS:
34+
#
35+
# 1. Log into ssl.com
36+
# 2. In your Dashboard, under "invitations", click the link under the order. Or
37+
# go to Orders, find the order, expand the order, and click "certificate
38+
# details"
39+
# 3. Enter your PIN to get your OTP. This is ESIGNER_KEYPASS.
40+
#
41+
# If you don't have access, see https://infra.apache.org/code-signing-use.html.
42+
43+
set -e
44+
set -u
45+
set -o pipefail
46+
47+
SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
48+
49+
if [ "$#" -ne 2 ]; then
50+
echo "Usage: $0 <version> <rc-num>"
51+
exit 1
52+
fi
53+
54+
. "${SOURCE_DIR}/utils-env.sh"
55+
56+
if [ -z "${ESIGNER_STOREPASS:-}" ]; then
57+
echo "ERROR: ESIGNER_STOREPASS is not set" >&2
58+
exit 1
59+
fi
60+
if [ -z "${ESIGNER_KEYPASS:-}" ]; then
61+
echo "ERROR: ESIGNER_KEYPASS is not set" >&2
62+
exit 1
63+
fi
64+
65+
version=$1
66+
rc=$2
67+
68+
version_with_rc="${version}-rc${rc}"
69+
tag="apache-arrow-${version_with_rc}"
70+
71+
dll_unsigned="arrow_flight_sql_odbc_unsigned.dll"
72+
dll_signed="arrow_flight_sql_odbc.dll"
73+
74+
: "${GITHUB_REPOSITORY:=apache/arrow}"
75+
76+
: "${PHASE_DEFAULT:=1}"
77+
: "${PHASE_SIGN_DLL:=${PHASE_DEFAULT}}"
78+
: "${PHASE_BUILD_MSI:=${PHASE_DEFAULT}}"
79+
: "${PHASE_SIGN_MSI:=${PHASE_DEFAULT}}"
80+
81+
if [ "${PHASE_SIGN_DLL}" -eq 0 ] && [ "${PHASE_BUILD_MSI}" -eq 0 ] && [ "${PHASE_SIGN_MSI}" -eq 0 ]; then
82+
echo "No phases specified. Exiting."
83+
exit 1
84+
fi
85+
86+
# Utility function to use jsign to check if a file is signed or not
87+
is_signed() {
88+
local file="$1"
89+
local exit_code=0
90+
jsign extract --format PEM "${file}" >/dev/null 2>&1 || exit_code=$?
91+
# jsign writes a PEM file even though it also prints to stdout. Clean up after
92+
# it. Use -f since so it still runs on unsigned files without error.
93+
rm -f "${file}.sig.pem"
94+
return ${exit_code}
95+
}
96+
97+
# Use dev/release/tmp for temporary files
98+
tmp_dir="${SOURCE_DIR}/tmp"
99+
if [ -e "${tmp_dir}" ]; then
100+
echo "ERROR: temp dir already exists: ${tmp_dir}. Remove it manually and run again." >&2
101+
exit 1
102+
fi
103+
104+
if [ "${PHASE_SIGN_DLL}" -gt 0 ]; then
105+
echo "[1/8 Downloading ${dll_unsigned} from release..."
106+
gh release download "${tag}" \
107+
--repo "${GITHUB_REPOSITORY}" \
108+
--pattern "${dll_unsigned}" \
109+
--dir "${tmp_dir}"
110+
if is_signed "${tmp_dir}/${dll_unsigned}"; then
111+
echo "ERROR: ${dll_unsigned} is already signed" >&2
112+
exit 1
113+
fi
114+
115+
echo "[2/8 Signing ${dll_unsigned}..."
116+
echo "NOTE: Running jsign. You may be prompted for your OTP PIN..."
117+
jsign --storetype ESIGNER \
118+
--alias d97c5110-c66a-4c0c-ac0c-1cd6af812ee6 \
119+
--storepass "${ESIGNER_STOREPASS}" \
120+
--keypass "${ESIGNER_KEYPASS}" \
121+
--tsaurl="http://ts.ssl.com" \
122+
--tsmode RFC3161 \
123+
--alg SHA256 \
124+
"${tmp_dir}/${dll_unsigned}"
125+
mv "${tmp_dir}/${dll_unsigned}" "${tmp_dir}/${dll_signed}"
126+
if ! is_signed "${tmp_dir}/${dll_signed}"; then
127+
echo "ERROR: ${dll_signed} is not signed" >&2
128+
exit 1
129+
fi
130+
131+
echo "[3/8 Uploading signed DLL to GitHub Release..."
132+
gh release upload "${tag}" \
133+
--repo "${GITHUB_REPOSITORY}" \
134+
--clobber \
135+
"${tmp_dir}/${dll_signed}"
136+
fi
137+
138+
if [ "${PHASE_BUILD_MSI}" -gt 0 ]; then
139+
echo "[4/8 Triggering odbc_release_step in cpp_extra.yml workflow..."
140+
gh workflow run cpp_extra.yml \
141+
--repo "${GITHUB_REPOSITORY}" \
142+
--ref "${tag}" \
143+
--field odbc_release_step=true
144+
145+
echo "[5/8 Waiting for workflow to complete. This can take a very long time..."
146+
REPOSITORY="${GITHUB_REPOSITORY}" \
147+
"${SOURCE_DIR}/utils-watch-gh-workflow.sh" "${tag}" cpp_extra.yml
148+
fi
149+
150+
if [ "${PHASE_SIGN_MSI}" -gt 0 ]; then
151+
echo "[6/8 Downloading unsigned MSI..."
152+
gh release download "${tag}" \
153+
--repo "${GITHUB_REPOSITORY}" \
154+
--pattern "Apache-Arrow-Flight-SQL-ODBC-${version}-win64.msi" \
155+
--dir "${tmp_dir}"
156+
msi="${tmp_dir}/Apache-Arrow-Flight-SQL-ODBC-${version}-win64.msi"
157+
if is_signed "${msi}"; then
158+
echo "ERROR: MSI is already signed" >&2
159+
exit 1
160+
fi
161+
162+
echo "[7/8 Signing MSI..."
163+
echo "NOTE: Running jsign. You may be prompted for your OTP PIN..."
164+
jsign --storetype ESIGNER \
165+
--alias d97c5110-c66a-4c0c-ac0c-1cd6af812ee6 \
166+
--storepass "${ESIGNER_STOREPASS}" \
167+
--keypass "${ESIGNER_KEYPASS}" \
168+
--tsaurl="http://ts.ssl.com" \
169+
--tsmode RFC3161 \
170+
--alg SHA256 \
171+
"${msi}"
172+
if ! is_signed "${msi}"; then
173+
echo "ERROR: MSI is not signed" >&2
174+
exit 1
175+
fi
176+
177+
echo "[8/8 Uploading signed MSI to GitHub Release..."
178+
gh release upload "${tag}" \
179+
--repo "${GITHUB_REPOSITORY}" \
180+
--clobber \
181+
"${msi}"
182+
fi
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ def setup
2424
detect_versions
2525
@tag_name_no_rc = "apache-arrow-#{@release_version}"
2626
@archive_name = "apache-arrow-#{@release_version}.tar.gz"
27-
@script = File.expand_path("dev/release/09-vote-email.sh")
27+
@script = File.expand_path("dev/release/10-vote-email.sh")
2828
@tarball_script = File.expand_path("dev/release/utils-create-release-tarball.sh")
2929
@env = File.expand_path("dev/release/.env")
3030

docs/source/developers/release.rst

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -264,13 +264,24 @@ Build source and binaries and submit them
264264
# NOTE: You need to have GitHub CLI installed to run this script.
265265
dev/release/06-matlab-upload.sh <version> <rc-number>
266266
267+
# Sign, build the installer for, and sign the installer for the FlightSQL
268+
# ODBC Windows driver
269+
#
270+
# NOTE: This must be run by a PMC member
271+
# Note: You need to have jsign installed and an available credential from
272+
# ASF to sign artifacts. Not all PMC members will have access to code
273+
# signing.
274+
# Note: The script requires setup of ssl.com environment variables.
275+
# Note: Invoking this script costs money.
276+
dev/release/07-flightsqlodbc-upload.sh <version> <rc-number>
277+
267278
# Move the Release Candidate GitHub Release from draft to published state
268279
# This will update the artifacts download URL which will be available for the
269280
# verification step.
270-
dev/release/07-publish-gh-release.sh <version> <rc-number>
281+
dev/release/08-publish-gh-release.sh <version> <rc-number>
271282
272283
# Start verifications for binaries and wheels
273-
dev/release/08-binary-verify.sh <version> <rc-number>
284+
dev/release/09-binary-verify.sh <version> <rc-number>
274285
275286
276287
Verify the Release
@@ -280,7 +291,7 @@ Verify the Release
280291
281292
# Once the automatic verification has passed start the vote thread
282293
# on dev@arrow.apache.org. To regenerate the email template use
283-
dev/release/09-vote-email.sh <version> <rc-number>
294+
dev/release/10-vote-email.sh <version> <rc-number>
284295
285296
See :ref:`release_verification` for details.
286297

0 commit comments

Comments
 (0)