Skip to content

Commit bab8a7d

Browse files
arotenberg-googlearotenberg-google
committed
Add Docker-based tests of client installation package.
1 parent 2511c40 commit bab8a7d

File tree

4 files changed

+88
-9
lines changed

4 files changed

+88
-9
lines changed

‎client/README.md‎

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -47,14 +47,29 @@ the distribution-specific commands to install these are:
4747
* Red Hat-based distros: `sudo yum install python38 python38-pip` (for e.g. Python
4848
3.8)
4949

50+
### GCP
51+
52+
You need the Google Cloud CLI client `gcloud` installed. [Read installation
53+
steps for the `gcloud` CLI package.](https://cloud.google.com/sdk/docs/install)
54+
55+
You need to have credentials configured locally for a GCP user with suitable
56+
permissions to perform batch translation. [Read how `gcloud` searches for
57+
application default credentials.](https://cloud.google.com/docs/authentication/application-default-credentials)
58+
59+
You need a GCP project and a Google Cloud Storage bucket to use for uploading
60+
your input SQL files and downloading the translated output. [Learn how to
61+
create a GCS bucket manually][creating buckets], or see the [instructions for
62+
using `provision.sh`](#running-using-runsh) to automatically provision a
63+
bucket for translation.
64+
5065
### Support for Encodings other than UTF-8
5166

5267
If all of the files you wish to translate are UTF-8 encoded
5368
(this is commonly the case), you can skip this section.
5469
Otherwise, you will need to install additional system dependencies:
5570

5671
* Debian-based distros: `sudo apt install pkg-config libicu-dev`
57-
* RedHat-based distros: `sudo yum install gcc gcc-c++ libicu-devel
72+
* Red Hat-based distros: `sudo yum install gcc gcc-c++ libicu-devel
5873
python38-devel`
5974

6075
**You must also remember**, upon reaching the step to `pip install` further down
@@ -64,14 +79,6 @@ in the Quickstart section below, to use this command instead:
6479
pip install ../dwh-migration-tools/client[icu]
6580
```
6681

67-
### GCP
68-
69-
You need a GCP project and a Google Cloud Storage bucket to use for uploading
70-
your input SQL files and downloading the translated output. [Learn how to
71-
create a GCS bucket manually][creating buckets], or see the [instructions for
72-
using `provision.sh`](#running-using-runsh) to automatically provision a
73-
bucket for translation.
74-
7582
## Quickstart
7683

7784
1. Download the repo from [google/dwh-migration-tools] in your choice of

‎client/tests/packaging/Dockerfile-RedHat‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
ARG ICU_ENABLED=false
2+
3+
FROM redhat/ubi9 AS dwh-migration-tools-base
4+
RUN dnf install -y \
5+
git \
6+
python3 \
7+
python3-pip
8+
# See https://cloud.google.com/sdk/docs/install
9+
COPY redhat-google-cloud-sdk.repo /etc/yum.repos.d/google-cloud-sdk.repo
10+
RUN dnf install -y google-cloud-cli
11+
WORKDIR /workspace
12+
RUN git clone -b main https://github.com/google/dwh-migration-tools \
13+
&& cp -R dwh-migration-tools/client/examples/teradata/sql project
14+
15+
FROM dwh-migration-tools-base AS dwh-migration-tools-icu-false
16+
RUN python3 -m pip install dwh-migration-tools/client
17+
18+
FROM dwh-migration-tools-base AS dwh-migration-tools-icu-true
19+
RUN dnf install -y \
20+
gcc \
21+
gcc-c++ \
22+
libicu-devel \
23+
python3-devel
24+
RUN python3 -m pip install dwh-migration-tools/client[icu]
25+
26+
FROM dwh-migration-tools-icu-${ICU_ENABLED}
27+
RUN dnf clean all -y && rm -rf /var/cache
28+
WORKDIR /workspace/project
29+
ENV BQMS_VERBOSE="True"
30+
ENV BQMS_MULTITHREADED="True"
31+
32+
ENTRYPOINT [ "/workspace/project/run.sh" ]

‎client/tests/packaging/Dockerfile-Ubuntu‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
ARG ICU_ENABLED=false
2+
3+
FROM ubuntu:22.04 AS dwh-migration-tools-base
4+
RUN apt-get update -y && apt-get install -y \
5+
curl \
6+
git \
7+
python3-pip
8+
# See https://cloud.google.com/sdk/docs/install
9+
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" \
10+
| tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg \
11+
| apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - \
12+
&& apt-get update -y \
13+
&& apt-get install -y google-cloud-cli
14+
WORKDIR /workspace
15+
RUN git clone -b main https://github.com/google/dwh-migration-tools \
16+
&& cp -R dwh-migration-tools/client/examples/teradata/sql project
17+
18+
FROM dwh-migration-tools-base AS dwh-migration-tools-icu-false
19+
RUN python3 -m pip install dwh-migration-tools/client
20+
21+
FROM dwh-migration-tools-base AS dwh-migration-tools-icu-true
22+
RUN apt-get install -y \
23+
libicu-dev \
24+
pkg-config
25+
RUN python3 -m pip install dwh-migration-tools/client[icu]
26+
27+
FROM dwh-migration-tools-icu-${ICU_ENABLED}
28+
RUN rm -rf /var/lib/apt/lists/*
29+
WORKDIR /workspace/project
30+
ENV BQMS_VERBOSE="True"
31+
ENV BQMS_MULTITHREADED="True"
32+
33+
ENTRYPOINT [ "/workspace/project/run.sh" ]

‎client/tests/packaging/redhat-google-cloud-sdk.repo‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
[google-cloud-cli]
2+
name=Google Cloud CLI
3+
baseurl=https://packages.cloud.google.com/yum/repos/cloud-sdk-el9-x86_64
4+
enabled=1
5+
gpgcheck=1
6+
repo_gpgcheck=0
7+
gpgkey=https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

0 commit comments

Comments
 (0)