Skip to content

Commit 7149e59

Browse files
authored
Merge pull request #2025 from ChildMindInstitute/develop
2026.02.1 Release
2 parents 896ace4 + f787ce3 commit 7149e59

54 files changed

Lines changed: 1685 additions & 465 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.default

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -62,14 +62,14 @@ MAILING__VALIDATE_CERTS=False
6262
NOTIFICATION__API_KEY=
6363

6464
# CDN configs (container by default)
65-
CDN__ENDPOINT_URL=http://localhost:9000
65+
CDN__DOMAIN=http://localhost:9000
6666
CDN__SECRET_KEY=miniosecret
6767
CDN__ACCESS_KEY=minioaccess
68-
CDN__REGION=
68+
CDN__REGION=us-east-1
6969
CDN__BUCKET_ANSWER=answer
7070
CDN__BUCKET=media
7171
CDN__BUCKET_OPERATIONS=operations
72-
CDN__STORAGE_ADDRESS=http://localhost:9000
72+
CDN__STORAGE_ADDRESS=
7373
CDN__TTL_SIGNED_URLS=3600
7474

7575
# jsonld converter

.github/workflows/_deploy.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ jobs:
8787
VERSION=${{ inputs.image-tag }}
8888
DD_VERSION=${{ inputs.image-tag }}
8989
SENTRY_RELEASE=${{ inputs.image-tag }}
90+
COMMIT_ID=${{ github.sha }}
9091
docker-labels: |
9192
com.datadoghq.tags.version=${{ inputs.image-tag }}
9293
- name: Update Task Definition

.github/workflows/tests.yaml

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,6 @@ jobs:
1515
with:
1616
version: '2.23.3'
1717

18-
# - name: Set up Python
19-
# uses: actions/setup-python@v4
20-
# with:
21-
# python-version: '3.13' # or your preferred version
2218
- name: Install uv
2319
uses: astral-sh/setup-uv@v6
2420

@@ -32,7 +28,29 @@ jobs:
3228
cp .env.default .env
3329
echo -e "\nRABBITMQ__USE_SSL=False" >> .env
3430
docker compose up -d rabbitmq postgres redis mailhog
35-
make test
31+
PYTHONPATH=src uv run pytest -s -vv --junit-xml=test-results.xml
32+
33+
- name: Surface failing tests
34+
if: always()
35+
uses: pmeier/pytest-results-action@main
36+
with:
37+
# A list of JUnit XML files, directories containing the former, and wildcard
38+
# patterns to process.
39+
# See @actions/glob for supported patterns.
40+
path: test-results.xml
41+
42+
# (Optional) Add a summary of the results at the top of the report
43+
summary: true
44+
45+
# (Optional) Select which results should be included in the report.
46+
# Follows the same syntax as `pytest -r`
47+
display-options: fEX
48+
49+
# (Optional) Fail the workflow if no JUnit XML was found.
50+
fail-on-empty: true
51+
52+
# (Optional) Title of the test results section in the workflow summary
53+
title: Test results
3654

3755
- name: Ensure CLI Runs
3856
run: |

README.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,14 @@ cp .env.default .env
8282
| DATABASE\_\_POOL\_SIZE | 5 | Database connection pool size |
8383
| DATABASE\_\_POOL\_OVERFLOW\_SIZE | 10 | Allowed overflow size of the connection pool |
8484
| DATABASE\_\_POOL\_TIMEOUT | 30 | The number of seconds to wait for a connection from the pool to become available |
85+
| CDN\_\_BUCKET | - | Bucket name to store applet media files |
86+
| CDN\_\_BUCKET\_ANSWER | - | Bucket name to store applet answer files |
87+
| CDN\_\_BUCKET\_OPERATIONS | - | Bucket to store intermediate files |
88+
| CDN\_\_BUCKET\_OVERRIDE | - | Bucket name to store applet media files in DR environment |
89+
| CDN\_\_BUCKET\_ANSWER\_OVERRIDE | - | Bucket name to store applet answer files in DR environment |
90+
| CDN\_\_BUCKET\_OPERATIONS\_OVERRIDE | - | Bucket to store intermediate files in DR environment |
91+
| CDN\_\_DOMAIN | - | Domain name that fronts the applet media bucket |
92+
| CDN\_\_REGION | - | Region that buckets exist in |
8593
| CORS\_\_ALLOW\_ORIGINS | `*` | Represents the list of allowed origins. Set the `Access-Control-Allow-Origin` header. Example: `https://dev.com,http://localhost:8000` |
8694
| CORS\_\_ALLOW\_ORIGINS\_REGEX | - | Regex pattern of allowed origins. |
8795
| CORS\_\_ALLOW\_CREDENTIALS | true | Set the `Access-Control-Allow-Credentials` header |
@@ -101,7 +109,7 @@ cp .env.default .env
101109
| REDIS\_\_MFA\_GLOBAL\_LOCKOUT\_TTL | 900 | Global MFA lockout duration in seconds (default: 15 minutes) |
102110
| MFA\_\_TOTP\_ENCRYPTION\_KEY | - | Base64-encoded Fernet encryption key for encrypting TOTP secrets. Generate using: `python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"` |
103111
| MFA\_\_TOTP\_ISSUER\_NAME | MindLogger | Issuer name shown in authenticator apps (e.g., Google Authenticator) |
104-
| MFA\_\_TOTP\_VALID\_WINDOW | 1 | Number of time steps to check before/after current time for TOTP validation (default allows ±30 seconds) |
112+
| MFA\_\_TOTP\_VALID\_WINDOW | 1 | Number of time steps to check before/after current time for TOTP validation (default allows ±30 seconds) |
105113
| MFA\_\_PENDING\_MFA\_EXPIRATION\_SECONDS | 600 | Expiration time in seconds for pending MFA setup (default: 10 minutes) |
106114
| MFA\_\_RECOVERY\_CODE\_ENCRYPTION\_KEY | - | Base64-encoded Fernet encryption key for encrypting recovery codes. Generate using: `python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"` |
107115
| MFA\_\_RECOVERY\_CODE\_COUNT | 10 | Number of recovery codes to generate per user |
@@ -165,13 +173,6 @@ Run RabbitMQ
165173
```
166174

167175

168-
> ⚠️ When using MinIO more configuration is needed to configure boto3 to talk to the local endpoints
169-
> ```
170-
> AWS_ACCESS_KEY_ID=minioaccess
171-
> AWS_SECRET_ACCESS_KEY=miniosecret
172-
> AWS_ENDPOINT_URL=http://localhost:9000
173-
> AWS_DEFAULT_REGION=us-east-1
174-
> ```
175176

176177
> 🛑 **NOTE:** If the application can't find the `RabbitMQ` service even though it's running normally, change your
177178
`RABBITMQ__URL` to your local ip address instead of `localhost`
@@ -299,6 +300,7 @@ psql# alter user test with password 'test';
299300
- [Arbitrary Server](docs/arbitrary.md)
300301
- [Curious CLI](docs/cli.md)
301302
- [Database](docs/db.md)
303+
- [Local File Upload/Storage (S3, etc)](docs/storage.md)
302304
- [Git Helpers](docs/git.md)
303305
- [PyCharm Setup](docs/pycharm.md)
304306
- [Security tokens (for deployments)](docs/tokens.md)

compose/minio/create_bucket.sh

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,4 +27,23 @@
2727
/usr/bin/mc anonymous set upload local/${CDN__BUCKET_OPERATIONS};
2828
/usr/bin/mc anonymous set download local/${CDN__BUCKET_OPERATIONS};
2929

30+
# DR buckets
31+
/usr/bin/mc mb -p local/${CDN__BUCKET_OVERRIDE};
32+
/usr/bin/mc policy set download local/${CDN__BUCKET_OVERRIDE};
33+
/usr/bin/mc policy set public local/${CDN__BUCKET_OVERRIDE};
34+
/usr/bin/mc anonymous set upload local/${CDN__BUCKET_OVERRIDE};
35+
/usr/bin/mc anonymous set download local/${CDN__BUCKET_OVERRIDE};
36+
37+
/usr/bin/mc mb -p local/${CDN__BUCKET_ANSWER_OVERRIDE};
38+
/usr/bin/mc policy set download local/${CDN__BUCKET_ANSWER_OVERRIDE};
39+
/usr/bin/mc policy set public local/${CDN__BUCKET_ANSWER_OVERRIDE};
40+
/usr/bin/mc anonymous set upload local/${CDN__BUCKET_ANSWER_OVERRIDE};
41+
/usr/bin/mc anonymous set download local/${CDN__BUCKET_ANSWER_OVERRIDE};
42+
43+
/usr/bin/mc mb -p local/${CDN__BUCKET_OPERATIONS_OVERRIDE};
44+
/usr/bin/mc policy set download local/${CDN__BUCKET_OPERATIONS_OVERRIDE};
45+
/usr/bin/mc policy set public local/${CDN__BUCKET_OPERATIONS_OVERRIDE};
46+
/usr/bin/mc anonymous set upload local/${CDN__BUCKET_OPERATIONS_OVERRIDE};
47+
/usr/bin/mc anonymous set download local/${CDN__BUCKET_OPERATIONS_OVERRIDE};
48+
3049
exit 0;

conftest.py

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
import datetime
2+
import logging
23
import os
34
import uuid
45
from typing import Any, AsyncGenerator, Callable, Generator, cast
56

67
import nest_asyncio
78
import pytest
9+
import structlog
810
import taskiq_fastapi
911
from alembic import command
1012
from alembic.config import Config
@@ -46,6 +48,33 @@
4648
nest_asyncio.apply()
4749

4850

51+
@pytest.fixture(autouse=True, scope="session")
52+
def _redirect_structlog_to_stdlib_and_mute() -> Generator[None, None, None]:
53+
"""Setup logging for CI"""
54+
if os.getenv("CI", "false").lower() == "true":
55+
prev = structlog.get_config()
56+
57+
structlog.configure(
58+
processors=[
59+
structlog.stdlib.add_log_level,
60+
structlog.stdlib.PositionalArgumentsFormatter(),
61+
structlog.processors.format_exc_info,
62+
structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
63+
],
64+
logger_factory=structlog.stdlib.LoggerFactory(),
65+
wrapper_class=structlog.make_filtering_bound_logger(logging.WARNING),
66+
cache_logger_on_first_use=True,
67+
)
68+
69+
try:
70+
yield None
71+
finally:
72+
structlog.reset_defaults()
73+
structlog.configure(**prev)
74+
else:
75+
yield None
76+
77+
4978
@pytest.fixture(scope="session")
5079
async def global_engine():
5180
engine = build_engine(settings.database.url)
@@ -85,6 +114,72 @@ def pytest_addoption(parser: Parser) -> None:
85114
)
86115

87116

117+
def configure() -> None:
118+
import logging.config
119+
120+
import structlog
121+
122+
timestamper = structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M:%S")
123+
pre_chain = [
124+
# Add the log level and a timestamp to the event_dict if the log entry
125+
# is not from structlog.
126+
structlog.stdlib.add_log_level,
127+
timestamper,
128+
]
129+
130+
logging.config.dictConfig(
131+
{
132+
"version": 1,
133+
"disable_existing_loggers": False,
134+
"formatters": {
135+
"plain": {
136+
"()": structlog.stdlib.ProcessorFormatter,
137+
"processor": structlog.dev.ConsoleRenderer(colors=False),
138+
"foreign_pre_chain": pre_chain,
139+
},
140+
# "colored": {
141+
# "()": structlog.stdlib.ProcessorFormatter,
142+
# "processor": structlog.dev.ConsoleRenderer(colors=True),
143+
# "foreign_pre_chain": pre_chain,
144+
# },
145+
},
146+
"handlers": {
147+
"default": {
148+
"level": "ERROR",
149+
"class": "logging.StreamHandler",
150+
"formatter": "plain", # <---Change to "plain"
151+
},
152+
"file": {
153+
"level": "ERROR",
154+
"class": "logging.handlers.WatchedFileHandler",
155+
"filename": "test.log",
156+
"formatter": "plain",
157+
},
158+
},
159+
"loggers": {
160+
"": {
161+
"handlers": ["default", "file"],
162+
"level": "ERROR",
163+
"propagate": True,
164+
},
165+
},
166+
}
167+
)
168+
structlog.configure(
169+
processors=[
170+
structlog.stdlib.add_log_level,
171+
structlog.stdlib.PositionalArgumentsFormatter(),
172+
timestamper,
173+
structlog.processors.StackInfoRenderer(),
174+
structlog.processors.format_exc_info,
175+
structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
176+
],
177+
logger_factory=structlog.stdlib.LoggerFactory(),
178+
wrapper_class=structlog.stdlib.BoundLogger,
179+
cache_logger_on_first_use=True,
180+
)
181+
182+
88183
def before():
89184
os.environ["PYTEST_APP_TESTING"] = "1"
90185
for alembic_cfg in alembic_configs:

docker-compose.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,9 @@ services:
174174
CDN__BUCKET: "${CDN__BUCKET:-cmi-media-local}"
175175
CDN__BUCKET_ANSWER: "${CDN__BUCKET_ANSWER:-cmi-answer-local}"
176176
CDN__BUCKET_OPERATIONS: "${CDN__BUCKET_OPERATIONS:-cmi-ops-local}"
177+
CDN__BUCKET_OVERRIDE: "${CDN__BUCKET_OVERRIDE:-cmi-media-local-dr}"
178+
CDN__BUCKET_ANSWER_OVERRIDE: "${CDN__BUCKET_ANSWER_OVERRIDE:-cmi-answer-local-dr}"
179+
CDN__BUCKET_OPERATIONS_OVERRIDE: "${CDN__BUCKET_OPERATIONS_OVERRIDE:-cmi-ops-local-dr}"
177180
depends_on:
178181
- minio
179182
volumes:

docs/cli.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ python src/cli.py [COMMAND] [SUBCOMMAND] [OPTIONS]
99
```
1010

1111
## Available Top-Level Commands
12+
- `mfa` – MFA management (clear, status)
1213
- `arbitrary` – Manage arbitrary server settings and data transfer
1314
- `patch` – Execute or list database/data patches
1415
- `encryption` – Encrypt, decrypt, or re-encrypt data

docs/storage.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Local Object Storage
2+
3+
⚠️ When using MinIO more configuration is needed to configure boto3 to talk to the local endpoints
4+
5+
## Environment Variables
6+
7+
Do not set `CDN__DOMAIN`. It conflicts with local settings and settings validation will fail
8+
9+
Ensure the following are set to these values to work with how MinIO is setup in Docker compose file:
10+
11+
```
12+
CDN__ENDPOINT_URL=http://localhost:9000
13+
CDN__SECRET_KEY=miniosecret
14+
CDN__ACCESS_KEY=minioaccess
15+
CDN__STORAGE_ADDRESS=http://localhost:9000/cmi-media-local
16+
```
17+
18+
## Docker Startup
19+
20+
When starting supporting services via docker compose be sure to include the
21+
following services in the startup command:
22+
23+
* `minio`
24+
* `createbuckets`

pyproject.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ dependencies = [
2121
"httpx==0.28.*",
2222
"jinja2==3.1.*",
2323
"more-itertools==10.7.0",
24-
"nh3==0.2.*",
24+
"nh3==0.3.*",
2525
"pydantic[email]>=2.0",
2626
"pyjwt==2.10.1",
2727
"pymongo==4.13.0",
@@ -79,6 +79,7 @@ dev = [
7979
"gevent==25.5.1",
8080
"greenlet==3.2.2",
8181
"ipdb==0.13.13",
82+
"moto[s3]>=5.1.20",
8283
"mypy==1.16.0",
8384
"nest-asyncio==1.6.0",
8485
"polyfactory==2.21.0",

0 commit comments

Comments
 (0)