Skip to content
This repository was archived by the owner on Apr 28, 2025. It is now read-only.

Commit 7d96faf

Browse files
stanleegoodspeedstanleegoodspeed
authored
Compose file cleanup and local synthetics (#218)
* add sythetics list * try synth in ci * update readme * update * try running compose * try adding sleep * remove sleep * test out ip * try again * test ping github ip * clean up compose files * remove synth workflow * update compose docs * update agent vers
1 parent 8a58698 commit 7d96faf

File tree

7 files changed

+218
-20
lines changed

7 files changed

+218
-20
lines changed

‎Makefile‎

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -118,8 +118,8 @@ local-attack-scenario-stop:
118118
local-attack-scenario-restart:
119119
docker-compose -f deploy/docker-compose/docker-compose-fixed-instrumented-attack.yml restart
120120

121-
.PHONY: local-baseline-start
122-
local-baseline-start:
121+
.PHONY: local-start
122+
local-start:
123123
POSTGRES_USER=postgres \
124124
POSTGRES_PASSWORD=postgres \
125125
ATTACK_HOST=nginx \
@@ -131,8 +131,42 @@ local-baseline-start:
131131
ATTACK_HYDRA_INTERVAL=120 \
132132
ATTACK_SSH=$(ENABLE_ATTACKS) \
133133
ATTACK_SSH_INTERVAL=90 \
134-
docker-compose -f deploy/docker-compose/docker-compose-local-baseline.yml up --build --force-recreate -d
134+
docker-compose -f deploy/docker-compose/docker-compose-local.yml up --build -d
135135

136-
.PHONY: local-baseline-stop
137-
local-baseline-stop:
138-
docker-compose -f deploy/docker-compose/docker-compose-local-baseline.yml down
136+
.PHONY: local-stop
137+
local-stop:
138+
docker-compose -f deploy/docker-compose/docker-compose-local.yml down
139+
140+
.PHONY: synthetics-start
141+
synthetics-start:
142+
datadog-ci synthetics run-tests --apiKey ${DD_API_KEY} --appKey ${DD_APP_KEY} --tunnel
143+
144+
.PHONY: latest-start
145+
latest-start:
146+
POSTGRES_USER=postgres \
147+
POSTGRES_PASSWORD=postgres \
148+
ATTACK_HOST=nginx \
149+
ATTACK_PORT=80 \
150+
DD_API_KEY=${DD_API_KEY} \
151+
ATTACK_GOBUSTER=$(ENABLE_ATTACKS) \
152+
ATTACK_GOBUSTER_INTERVAL=180 \
153+
ATTACK_HYDRA=$(ENABLE_ATTACKS) \
154+
ATTACK_HYDRA_INTERVAL=120 \
155+
ATTACK_SSH=$(ENABLE_ATTACKS) \
156+
ATTACK_SSH_INTERVAL=90 \
157+
docker-compose -f deploy/docker-compose/docker-compose-latest.yml up --build -d
158+
159+
.PHONY: latest-stop
160+
latest-stop:
161+
POSTGRES_USER=postgres \
162+
POSTGRES_PASSWORD=postgres \
163+
ATTACK_HOST=nginx \
164+
ATTACK_PORT=80 \
165+
DD_API_KEY=${DD_API_KEY} \
166+
ATTACK_GOBUSTER=$(ENABLE_ATTACKS) \
167+
ATTACK_GOBUSTER_INTERVAL=180 \
168+
ATTACK_HYDRA=$(ENABLE_ATTACKS) \
169+
ATTACK_HYDRA_INTERVAL=120 \
170+
ATTACK_SSH=$(ENABLE_ATTACKS) \
171+
ATTACK_SSH_INTERVAL=90 \
172+
docker-compose -f deploy/docker-compose/docker-compose-latest.yml down

‎README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,3 +162,24 @@ We eager load the `discount_type` relation on the `discount`, and can grab all i
162162
![N+1 Solved](https://github.com/DataDog/ecommerce-workshop/raw/master/images/solved-nplus.png)
163163

164164
The N+1 query example lives in `discounts-service/`, and the fixed version lives in `discounts-service-fixed/`.
165+
166+
## How to run synthetics locally
167+
168+
1. Install `@datadog/datadog-ci` via NPM or Yarn globally on your local machine:
169+
170+
```
171+
npm install -g @datadog/datadog-ci
172+
```
173+
```
174+
yarn global add @datadog/datadog-ci
175+
```
176+
177+
2. Obtain the API and APP Key from the DD corpsite account:
178+
3. From the project root, run the following:
179+
`DD_API_KEY="<API_KEY>" DD_APP_KEY="<APP_KEY>" make synthetics-start`
180+
181+
182+
*To add a new test:*
183+
1. Generate a synthetics test via the DD app
184+
2. Grab the public ID of the test (found in the top left of the synthetic page or in the URL) and add it to `storedog.synthetics.json`.
185+

‎deploy/docker-compose/.env‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
ADS_PORT=9292
2-
DISCOUNTS_PORT=8282
1+
ADS_PORT=7676
2+
DISCOUNTS_PORT=2814
33
ADS_ROUTE="http://advertisements"
44
DISCOUNTS_ROUTE="http://discounts"

‎deploy/docker-compose/README.md‎

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,24 +12,20 @@ They currently exist in three different versions:
1212

1313
`docker-compose-fixed-instrumented`: View a fixed application and compare it to the previously broken deployment.
1414

15+
`docker-compose-fixed-instrumented-attack`: The same as `docker-compose-fixed-instrumented` with the addition of a `attack` container that simulates an adversary attempting to hack Storedog
16+
1517
`docker-compose-traffic-replay`: Simulate traffic to the application.
1618

19+
`docker-compose-latest`: All services point to the `latest` tagged images and the compose file contains the most up-to-date env vars and config options
20+
21+
`docker-compose-local`: All services (except `frontend` and `agent`) are created using the local build context. This file is useful for local development, specifically when testing out new changes to Dockerfiles
22+
1723
The application itself runs on `docker-compose`. First, install Docker along with docker-compose. Then sign up with a trial [Datadog account](https://www.datadoghq.com/), and grab your API key from the Integrations->API tab.
1824

1925
To run any of the scenarios, be in the root directory:
2026

2127
```bash
22-
git clone https://github.com/DataDog/ecommerce-workshop.git
23-
cd ecommerce-workshop
24-
cd store-frontend/src/
25-
cp -R store-frontend-initial-state store-frontend-broken-instrumented
26-
cd store-frontend-broken-instrumented
27-
patch -t -p1 < ../broken-instrumented.patch
28-
cd ..
29-
cp -R store-frontend-initial-state store-frontend-instrumented-fixed
30-
cd store-frontend-instrumented-fixed
31-
patch -t -p1 < ../instrumented-fixed.patch
32-
cd ../../..
28+
make recreate-frontend-code
3329
POSTGRES_USER=postgres POSTGRES_PASSWORD=postgres DD_API_KEY=<YOUR_API_KEY> docker-compose -f deploy/docker-compose/<docker_compose_with_your_selected_scenario> up
3430
```
3531

‎deploy/docker-compose/docker-compose-latest.yml‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# Docker compose file which uses build context plus mounted volumes for easier local development
2+
# Note: storefront-fixed cannot use the build context because the .patch files are located in a parent directory of the Dockerfile
3+
version: '3'
4+
services:
5+
agent:
6+
image: "datadog/agent:7.35.0"
7+
environment:
8+
- DD_API_KEY
9+
- DD_APM_ENABLED=true
10+
- DD_LOGS_ENABLED=true
11+
- DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true
12+
- DD_PROCESS_AGENT_ENABLED=true
13+
- DD_DOCKER_LABELS_AS_TAGS={"my.custom.label.team":"team"}
14+
- DD_TAGS='env:development'
15+
ports:
16+
- "8126:8126"
17+
volumes:
18+
- /var/run/docker.sock:/var/run/docker.sock:ro
19+
- /proc/:/host/proc/:ro
20+
- /sys/fs/cgroup/:/host/sys/fs/cgroup:ro
21+
labels:
22+
com.datadoghq.ad.logs: '[{"source": "agent", "service": "agent"}]'
23+
discounts:
24+
environment:
25+
- FLASK_APP=discounts.py
26+
- FLASK_DEBUG=1
27+
- POSTGRES_PASSWORD
28+
- POSTGRES_USER
29+
- POSTGRES_HOST=db
30+
- DD_SERVICE=discounts-service
31+
- DD_AGENT_HOST=agent
32+
- DD_LOGS_INJECTION=true
33+
- DD_TRACE_ANALYTICS_ENABLED=true
34+
- DD_PROFILING_ENABLED=true
35+
- DD_VERSION=1.1
36+
image: public.ecr.aws/x2b9z2t7/ddtraining/discounts-fixed:latest
37+
command: flask run --port=${DISCOUNTS_PORT} --host=0.0.0.0 # If using any other port besides the default 8282, overriding the CMD is required
38+
volumes:
39+
- "../../discounts-service-fixed:/app"
40+
ports:
41+
- "${DISCOUNTS_PORT}:${DISCOUNTS_PORT}"
42+
- "22"
43+
depends_on:
44+
- agent
45+
- db
46+
labels:
47+
com.datadoghq.ad.logs: '[{"source": "python", "service": "discounts-service"}]'
48+
my.custom.label.team: "discount"
49+
frontend:
50+
environment:
51+
- DD_AGENT_HOST=agent
52+
- DD_LOGS_INJECTION=true
53+
- DD_TRACE_ANALYTICS_ENABLED=true
54+
- DD_SERVICE=store-frontend
55+
- DB_USERNAME
56+
- DB_PASSWORD
57+
- DD_VERSION=1.1
58+
- DD_CLIENT_TOKEN
59+
- DD_APPLICATION_ID
60+
- DD_ENV=development
61+
- DD_SITE=datadoghq.com
62+
- RAILS_HIDE_STACKTRACE=true
63+
- ADS_PORT=${ADS_PORT}
64+
- DISCOUNTS_PORT=${DISCOUNTS_PORT}
65+
- ADS_ROUTE=${ADS_ROUTE}
66+
- DISCOUNTS_ROUTE=${DISCOUNTS_ROUTE}
67+
image: public.ecr.aws/x2b9z2t7/ddtraining/storefront-fixed:latest
68+
volumes:
69+
- "../../store-frontend/src/store-frontend-instrumented-fixed:/app"
70+
ports:
71+
- "3000:3000"
72+
depends_on:
73+
- agent
74+
- db
75+
- discounts
76+
- advertisements
77+
labels:
78+
com.datadoghq.ad.logs: '[{"source": "ruby", "service": "store-frontend"}]'
79+
my.custom.label.team: "frontend"
80+
advertisements:
81+
environment:
82+
- FLASK_APP=ads.py
83+
- FLASK_DEBUG=1
84+
- POSTGRES_PASSWORD
85+
- POSTGRES_USER
86+
- POSTGRES_HOST=db
87+
- DD_SERVICE=advertisements-service
88+
- DD_AGENT_HOST=agent
89+
- DD_LOGS_INJECTION=true
90+
- DD_TRACE_ANALYTICS_ENABLED=true
91+
- DD_PROFILING_ENABLED=true
92+
- DD_VERSION=1.0
93+
image: public.ecr.aws/x2b9z2t7/ddtraining/advertisements-fixed:latest
94+
command: flask run --port=${ADS_PORT} --host=0.0.0.0 # If using any other port besides the default 9292, overriding the CMD is required
95+
volumes:
96+
- "../../ads-service-fixed:/app"
97+
ports:
98+
- "${ADS_PORT}:${ADS_PORT}"
99+
depends_on:
100+
- agent
101+
- db
102+
labels:
103+
com.datadoghq.ad.logs: '[{"source": "python", "service": "ads-service"}]'
104+
my.custom.label.team: "advertisements"
105+
db:
106+
image: postgres:11-alpine
107+
restart: always
108+
environment:
109+
- POSTGRES_PASSWORD
110+
- POSTGRES_USER
111+
labels:
112+
com.datadoghq.ad.logs: '[{"source": "postgresql", "service": "postgres"}]'
113+
attackbox:
114+
image: public.ecr.aws/x2b9z2t7/ddtraining/attackbox:latest
115+
logging:
116+
driver: none # this only works when using docker-compose (vs docker compose)
117+
environment:
118+
- ATTACK_GOBUSTER
119+
- ATTACK_HYDRA
120+
- ATTACK_GOBUSTER_INTERVAL
121+
- ATTACK_HYDRA_INTERVAL
122+
- ATTACK_SSH
123+
- ATTACK_SSH_INTERVAL
124+
- ATTACK_HOST
125+
- ATTACK_PORT
126+
depends_on:
127+
- discounts
128+
- frontend
129+
nginx:
130+
restart: always
131+
image: public.ecr.aws/x2b9z2t7/ddtraining/nginx:latest
132+
ports:
133+
- "80:80"
134+
depends_on:
135+
- frontend
136+
labels:
137+
com.datadoghq.ad.logs: '[{"source": "nginx", "service": "nginx"}]'

deploy/docker-compose/docker-compose-local-baseline.yml renamed to deploy/docker-compose/docker-compose-local.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
version: '3'
44
services:
55
agent:
6-
image: "datadog/agent:7.29.0"
6+
image: "datadog/agent:7.35.0"
77
environment:
88
- DD_API_KEY
99
- DD_APM_ENABLED=true

‎storedog.synthetics.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"tests": [
3+
{
4+
"id": "w7z-73y-z5s"
5+
},
6+
{
7+
"id": "gq2-uwi-m23"
8+
}
9+
]
10+
}

0 commit comments

Comments
 (0)