Skip to content

Commit 3e1533a

Browse files
authored
Merge pull request #50 from jhanna-kx/master
Docker ux changes
2 parents 9dedb9c + 2cf3194 commit 3e1533a

File tree

7 files changed

+332
-89
lines changed

7 files changed

+332
-89
lines changed

.travis.yml

+21-4
Original file line numberDiff line numberDiff line change
@@ -2,23 +2,40 @@ language: c
22
os:
33
- linux
44
- osx
5+
git:
6+
depth: 1
57
script:
68
- make p.so
79
- echo "Preparing version $TRAVIS_BRANCH-$TRAVIS_COMMIT"
8-
- zip -r embedPy_$TRAVIS_OS_NAME-$TRAVIS_BRANCH.zip p.q p.k test.q tests m64/p.so
9-
l64/p.so LICENSE README.md || true
10+
- tar czf embedPy_$TRAVIS_OS_NAME-$TRAVIS_BRANCH.tgz p.q p.k test.q tests [ml]64/p.so LICENSE README.md
1011
- echo "Packaged as embedPy_$TRAVIS_OS_NAME-$TRAVIS_BRANCH.zip"
1112
compiler:
1213
- clang
13-
- gcc
1414
deploy:
1515
provider: releases
1616
api_key: "$GITHUB_APIKEY"
1717
file_glob: true
18-
file: embedPy_*.zip
18+
file: embedPy_$TRAVIS_OS_NAME-$TRAVIS_BRANCH.tgz
1919
skip_cleanup: true
2020
on:
2121
tags: true
22+
jobs:
23+
include:
24+
- stage: docker
25+
sudo: required
26+
services:
27+
- docker
28+
script:
29+
- docker build -t kxsys/embedpy:latest -f docker/Dockerfile .
30+
- docker tag kxsys/embedpy:latest kxsys/embedpy:$TRAVIS_BRANCH
31+
deploy:
32+
provider: script
33+
script:
34+
- echo $DOCKER_PASSWORD|docker login --username $DOCKER_USERNAME --password-stdin && docker push kxsys/embedpy:latest && docker push kxsys/embedpy:$TRAVIS_BRANCH
35+
skip_cleanup: true
36+
on:
37+
tags: true
38+
condition: $TRAVIS_BRANCH =~ ^[0-9]+\.
2239
notifications:
2340
slack:
2441
secure: "MR5icOywJJSFy7gnxeNXACN2nKiHniVgX1myiF+H9U4wsW6+2BHzPMcmy3FIUjL3Tb+PcAfq7+8RHerAx/XEB7nsVJsu9TyCjYWgaqx5Z9WNIDcjQh4wnFurhlzqrwF4gjelY+wxdffpsKUVaP+fermWbqHvv1aPPk4l3xEonXZ5J5uNkPTPufILzpx2cTxWg08o6PhvgjTrombs+9rGxahU8cfUSwTVuPCcPtOy70kcaowFfrU2kWzQk88NM5d/TVZdkOmGhLqVWSvRgaLXFkGYwRWatT7divlXX2evYwdYib++RUUhxDGZwJjdFCbGDNjrMyBbJOrCkndxLSATd6gN76dWuWsVV49K4YB0Mw5uFJFt6J0TnbkJC5Zsl9i6vik7m3anuIPOeAI54/wtlCvMeIpt2eQCPMy7xXu2hXNUR4ofM4yaVjKz69b5JIZ1o3vBxPQxAqIYggqPsTaLJM4vLpZlncnViBkfO/La5V9sJNQy3IyLSlEKry3giYfo8wbHT3PLn1xG7HW2DJ7hRn19yzTkCCNfaSzvq8YlSQvjNlPCO566ykYYG6SzG/NnbDqJAqydhpFuNC7ZB5dPuAOoDANCGs4gtfYKia+ZVxKYn25Y52vdi34L5TO83BUrsQWgytb0J6a4jDe3PWLVFSP7XLiy4h8Tyv76+Hm4lHk="

README.md

+14-20
Original file line numberDiff line numberDiff line change
@@ -50,30 +50,24 @@ To install, place `p.q` and `p.k` in `$QHOME` and `p.so` in `$QHOME/{l64,m64}`.
5050

5151
### Docker
5252

53-
If you have [Docker installed](https://www.docker.com/community-edition) you can alternatively create a directory called `q` and place your `kc.lic` (or `k4.lic`) file into a `q` directory and run:
53+
If you have [Docker installed](https://www.docker.com/community-edition) you can alternatively run:
5454

55-
$ docker run --rm -it -v `pwd`/q:/tmp/q kxsys/embedpy
56-
kx@1ba5d6c29709:~$ q
57-
KDB+ 3.5 2017.11.08 Copyright (C) 1993-2017 Kx Systems
58-
l64/ 4(16)core 7905MB kx 0123456789ab 172.17.0.2 EXPIRE 2018.12.04 [email protected] KOD #0000000
59-
60-
q)
61-
62-
You can drop straight into `bash` with:
63-
64-
$ docker run --rm -it -v `pwd`/q:/tmp/q kxsys/embedpy bash
65-
kx@8ac226623908:~$ conda info
55+
$ docker run -it --name myembedpy kxsys/embedpy
56+
kdb+ on demand - Personal Edition
6657

67-
active environment : kx
68-
active env location : /home/kx/.conda/envs/kx
6958
[snipped]
59+
60+
I agree to the terms of the license agreement for kdb+ on demand Personal Edition (N/y): y
61+
62+
If applicable please provide your company name (press enter for none): ACME Limited
63+
Please provide your name: Bob Smith
64+
Please provide your email (requires validation): [email protected]
65+
KDB+ 3.5 2018.04.25 Copyright (C) 1993-2018 Kx Systems
66+
l64/ 4()core 7905MB kx 0123456789ab 172.17.0.2 EXPIRE 2018.12.04 [email protected] KOD #0000000
67+
68+
q)
7069

71-
Lastly you can also pipe your program in:
72-
73-
$ echo 'p)print(1+2)' | docker run --rm -i -v `pwd`/q:/tmp/q kxsys/embedpy q -q
74-
3
75-
76-
**N.B.** [build instructions for the image are available](docker/README.md)
70+
**N.B.** Further options for running and build instructions for the image are available [here](docker/README.md)
7771

7872
## Usage
7973

docker/Dockerfile

+3-1
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,10 @@ RUN mkdir -p $QHOME
6565
RUN curl -o $QHOME/l64.zip -L http://dls.kx.com/253h1j3s2941/download/l64.zip \
6666
&& [ "$L64_SHA256" = "$(sha256sum $QHOME/l64.zip | cut -b1-64)" ] \
6767
&& unzip -d $QHOME $QHOME/l64.zip \
68-
&& rm $QHOME/l64.zip
68+
&& rm $QHOME/l64.zip \
69+
&& apt-get -yy --option=Dpkg::options::=--force-unsafe-io purge unzip
6970
COPY docker/q.wrapper /usr/local/bin/q
71+
COPY docker/kc.lic.py /opt/kx/
7072

7173
COPY --from=embedpy /opt/kx/embedPy /opt/kx/embedPy
7274
RUN ln -s -t $QHOME/l64 /opt/kx/embedPy/l64/p.so \

docker/README.md

+79-39
Original file line numberDiff line numberDiff line change
@@ -1,64 +1,104 @@
1-
This project builds the Docker image.
1+
# Run
22

3-
## Related Links
3+
If you have [Docker installed](https://www.docker.com/community-edition) there is a prebuilt docker image you can run with:
44

5-
* [Docker](https://docker.com)
6-
* [`Dockerfile`](https://docs.docker.com/engine/reference/builder/)
7-
* [Automated Builds](https://docs.docker.com/docker-cloud/builds/automated-build/)
8-
* [Advanced options for Autobuild and Autotest](https://docs.docker.com/docker-cloud/builds/advanced/)
5+
$ docker run -it --name myembedpy kxsys/embedpy
6+
kdb+ on demand - Personal Edition
7+
8+
[snipped]
9+
10+
I agree to the terms of the license agreement for kdb+ on demand Personal Edition (N/y): y
11+
12+
If applicable please provide your company name (press enter for none): ACME Limited
13+
Please provide your name: Bob Smith
14+
Please provide your email (requires validation): [email protected]
15+
KDB+ 3.5 2018.04.25 Copyright (C) 1993-2018 Kx Systems
16+
l64/ 4()core 7905MB kx 0123456789ab 172.17.0.2 EXPIRE 2018.12.04 [email protected] KOD #0000000
17+
18+
q)
919

10-
# Preflight
20+
For subsequent runs, you will not be prompted to redo the license setup when calling:
1121

12-
You will need [Docker installed](https://www.docker.com/community-edition) on your workstation; make sure it is a recent version as they are always breaking backwards compatibility.
22+
$ docker start -ai myembedpy
23+
KDB+ 3.5 2018.04.25 Copyright (C) 1993-2018 Kx Systems
24+
l64/ 4()core 7905MB kx 0123456789ab 172.17.0.2 EXPIRE 2018.12.04 [email protected] KOD #0000000
25+
26+
q)
1327

14-
Check out a copy of the project with:
28+
If you prefer to drop straight into `bash` you can with:
1529

16-
git clone https://github.com/KxSystems/embedPy.git
30+
$ docker run -it kxsys/embedpy bash
31+
[snipped license setup]
32+
kx@8ac226623908:~$ conda info
33+
34+
active environment : kx
35+
active env location : /home/kx/.conda/envs/kx
36+
[snipped]
37+
kx@8ac226623908:~$ q
38+
KDB+ 3.5 2018.04.25 Copyright (C) 1993-2018 Kx Systems
39+
l64/ 4()core 7905MB kx 0123456789ab 172.17.0.2 EXPIRE 2018.12.04 [email protected] KOD #0000000
40+
41+
q)
1742

18-
# Deploy
43+
**N.B.** [build instructions for the image are available](README.md#build)
1944

20-
[Docker Cloud](https://cloud.docker.com/) is configured to monitor when tags of the format `/^[0-9.]+/` are added to the [GitHub hosted project](https://github.com/KxSystems/embedPy), a corresponding Docker image file is generated and made available.
45+
## Headless/Presets
2146

22-
This is all done server side as the resulting image is north of 350MB and uploading that sort of thing is likely to prompt the network team to Release the Kraken!
47+
If you are running this in a headless configuration, or wish to provide out-of-bound the answers to the license process, then you can call upon the following environment variables. This is handled either by [`--env` (or `--env-file`) as detailed on the docker website](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file).
2348

24-
To do a deploy, you simply tag and push your releases as usual:
49+
* **`KDB_LICENSE_AGREE`:** pass in `yes` (case-insensitive) to agree to the license agreement
50+
* **`COMPANY` (optional):** provide company name
51+
* **`NAME`:** provide your name
52+
* **`EMAIL`:** provide your email address
2553

26-
git push
27-
git tag 0.7
28-
git push --tag
54+
If any of these are missing, then you will be prompted for the value as before.
2955

30-
## First Time Configuration
56+
**N.B.** alternatively if you already have a licence you can use `QLIC_KC` (also supported is `QLIC_K4`) which is the base64 encoded contents of your `kc.lic` file
3157

32-
1. Log into [Docker Cloud](https://cloud.docker.com/)
33-
1. Create the new repository called `embedpy`
34-
1. Go to the 'Builds' tab
35-
1. Set the 'Source Repository' to the GitHub account 'KxSystems' and project `embedPy`
36-
1. Set the 'Build Rules' to:
37-
* **Source Type:** Tag
38-
* **Source:** `/^[0-9.]+/`
39-
* **Docker Tag:** `{sourceref}`
40-
* **Dockerfile location:** `docker/Dockerfile`
41-
* **Build Context:** `/`
42-
* **Autobuild:** enabled
43-
* **Build Caching:** disabled (Docker Cloud is *really* buggy, `nocache=1` as a build env may help)
44-
1. Click on the 'Save' button
58+
This allows for the following usage:
4559

46-
If you prefer to not have Docker Cloud build on every tag push, you can alternatively:
4760

48-
1. Under 'Build Rules' disable 'Autobuild'
49-
1. Click on 'Save'
50-
1. At the bottom of the configuration page under 'Build triggers', create a URL that you can use to fire off a build
61+
$ echo \\\\ | docker run -i -e QLIC_KC=$(cat $QHOME/kc.lic | base64 -w0) --name myembedpywithlic kxsys/embedpy q -q
62+
$ echo 'p)print(1+2)' | docker start -i myembedpywithlic
63+
3
5164

52-
You should now be able to call `curl` on the supplied URL to trigger the build, sparing you from having to log in to click a button.
5365

5466
# Build
5567

68+
The instructions below are for building your own Docker image. A prebuilt Docker image is available on Docker Cloud, if you only want to run the embedPy image then install Docker and [read the instructions above](README.md#run) on how to do this.
69+
70+
## Preflight
71+
72+
You will need [Docker installed](https://www.docker.com/community-edition) on your workstation; make sure it is a recent version.
73+
74+
Check out a copy of the project with:
75+
76+
git clone https://github.com/KxSystems/embedPy.git
77+
78+
## Building
79+
5680
To build locally the project you run:
5781

5882
docker build -t embedpy -f docker/Dockerfile .
5983

60-
Other build arguments are supported and you should browse the `Dockerfile` to see what they are, but note for Docker Cloud you will need to make sure they are also explicitly exported in [`docker/hooks/build`](hooks/build) too.
84+
Once built, you should have a local `embedpy` image, you can run the following to use it:
85+
86+
docker run -it embedpy
87+
88+
# Deploy
89+
90+
[travisCI](https://travis-ci.org/) is configured to monitor when tags of the format `/^[0-9]+\./` are added to the [GitHub hosted project](https://github.com/KxSystems/embedPy), a corresponding Docker image is generated and made available on [Docker Cloud](https://cloud.docker.com/)
91+
92+
This is all done server side as the resulting image is north of 350MB.
93+
94+
To do a deploy, you simply tag and push your releases as usual:
95+
96+
git push
97+
git tag 0.7
98+
git push --tag
99+
61100

62-
Once built, you should have a local `embedpy` image you can run the following to use it:
101+
## Related Links
63102

64-
docker run -it -v `pwd`/q:/tmp/q embedpy
103+
* [Docker](https://docker.com)
104+
* [`Dockerfile`](https://docs.docker.com/engine/reference/builder/)

docker/hooks/build

-9
This file was deleted.

docker/init

+4-16
Original file line numberDiff line numberDiff line change
@@ -2,22 +2,6 @@
22

33
set -eu
44

5-
[ -d /tmp/q ] \
6-
|| { echo q non-functional, did you include the q directory? >&2; exit 1; }
7-
8-
[ -f /tmp/q/kc.lic ] && cat /tmp/q/kc.lic > "$QHOME/kc.lic" || true
9-
[ -f /tmp/q/k4.lic ] && cat /tmp/q/k4.lic > "$QHOME/k4.lic" || true
10-
[ -f "$QHOME/kc.lic" -o -f "$QHOME/k4.lic" ] \
11-
|| { echo missing kc.lic/k4.lic >&2; exit 1; }
12-
13-
[ "$(echo '-1 getenv`QHOME;' | q -q)" = "$QHOME" ] \
14-
|| { echo q non-functional >&2; exit 1; }
15-
16-
RC=0
17-
CHECK=$(echo '.z.K>=3.5' | q -q) || RC=$?
18-
[ $RC -eq 0 ] || { echo q non-functional >&2; exit 1; }
19-
[ "$CHECK" = "1b" ] || { echo "kdb+ version must be >= 3.5" >&2; exit 1; }
20-
215
if ! test -x "$1"; then
226
SH=$(which "$1")
237
shift
@@ -33,4 +17,8 @@ fi
3317

3418
HOME=/home/kx
3519
cd $HOME
20+
21+
export QLIC=$HOME
22+
chpst -u kx /bin/sh -l -c 'exec python /opt/kx/kc.lic.py'
23+
3624
exec chpst -u kx "$@"

0 commit comments

Comments
 (0)