Skip to content

Commit b76e3b0

Browse files
authored
# [P1B] Add /ready endpoint and Alembic migration setup (#61)
* refactor: Complete routerization - Add system_router with 4 endpoints - Created system_router.py with root, health, status, validators/metrics endpoints - Updated main.py (reduced from 1899 to 1880 lines) - Updated routers/__init__.py to include system_router - Updated README.md with complete routerization info - Created smoke tests: test_router_smoke.py - Protected internal docs in .gitignore - Added push_branch_with_token.ps1 utility script Total: 42 endpoints organized into 6 routers main.py: 2817 → 1880 lines (reduced 937 lines, ~33%) Related to #58 * chore: Add PR_CREATION_GUIDE.md to .gitignore (internal doc) * fix: Remove rate limiter from /health endpoint for Railway health checks - Health check endpoints must not have rate limiting - Railway/Docker health probes can call /health frequently - Added error handling to always return 200 OK - Prevents 'service unavailable' deployment failures Fixes Railway deployment issue: 1/1 replicas never became healthy * feat: Add /ready endpoint (P1B) and Alembic migration setup - Implement /ready endpoint in system_router.py with comprehensive checks: * SQLite database connectivity (4 databases) * ChromaDB client availability * Embedding service with timeout (2s) * Feature flag: ENABLE_HEALTH_READY (default: true) * Returns 200 when all checks pass, 503 if any fails - Setup Alembic for database migration planning: * Add alembic==1.13.1 to requirements.txt * Create alembic.ini configuration * Create alembic/env.py with SQLite/PostgreSQL support * Create alembic/script.py.mako template * Add alembic/README.md documentation * Create docs/DATABASE_MIGRATION_PLANNING.md with migration strategy - Add test for /ready endpoint in test_router_smoke.py - Update OpenAPI docs test to verify /ready endpoint Closes: P1B Health & Readiness Endpoints Related: Database migration planning for PostgreSQL
1 parent 9e88481 commit b76e3b0

13 files changed

Lines changed: 756 additions & 435 deletions

README.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -262,11 +262,8 @@ graph TB
262262
- `rag_router.py` - RAG endpoints (4 endpoints)
263263
- `tiers_router.py` - Continuum Memory tier management (5 endpoints)
264264
- `spice_router.py` - SPICE framework endpoints (6 endpoints)
265-
refactor/routerization
266265
- `system_router.py` - System endpoints: root, health, status, validators/metrics (4 endpoints)
267266
- **Total**: 42 endpoints organized into 6 routers
268-
269-
main
270267
- **Benefits**: Better code organization, easier maintenance, OSS-friendly structure
271268
- **⏰ Automated Scheduler**: Auto-learning from RSS every 4 hours + Multi-timescale scheduler (hourly/daily/weekly/monthly)
272269
- **🔍 Self-Diagnosis**: Knowledge gap detection and learning focus suggestions

alembic.ini

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# A generic, single database configuration.
2+
3+
[alembic]
4+
# path to migration scripts
5+
script_location = alembic
6+
7+
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
8+
# Uncomment the line below if you want the files to be prepended with date and time
9+
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
10+
11+
# sys.path path, will be prepended to sys.path if present.
12+
# defaults to the current working directory.
13+
prepend_sys_path = .
14+
15+
# timezone to use when rendering the date within the migration file
16+
# as well as the filename.
17+
# If specified, requires the python-dateutil library that can be
18+
# installed by adding `alembic[tz]` to the pip requirements
19+
# string value is passed to dateutil.tz.gettz()
20+
# leave blank for localtime
21+
# timezone =
22+
23+
# max length of characters to apply to the
24+
# "slug" field
25+
# truncate_slug_length = 40
26+
27+
# set to 'true' to run the environment during
28+
# the 'revision' command, regardless of autogenerate
29+
# revision_environment = false
30+
31+
# set to 'true' to allow .pyc and .pyo files without
32+
# a source .py file to be detected as revisions in the
33+
# versions/ directory
34+
# sourceless = false
35+
36+
# version location specification; This defaults
37+
# to alembic/versions. When using multiple version
38+
# directories, initial revisions must be specified with --version-path.
39+
# The path separator used here should be the separator specified by "version_path_separator" below.
40+
# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions
41+
42+
# version path separator; As mentioned above, this is the character used to split
43+
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
44+
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
45+
# Valid values for version_path_separator are:
46+
#
47+
# version_path_separator = :
48+
# version_path_separator = ;
49+
# version_path_separator = space
50+
version_path_separator = os # Use os.pathsep. Default configuration used for new projects.
51+
52+
# set to 'true' to search source files recursively
53+
# in each "version_locations" directory
54+
# new in Alembic version 1.10
55+
# recursive_version_locations = false
56+
57+
# the output encoding used when revision files
58+
# are written from script.py.mako
59+
# output_encoding = utf-8
60+
61+
sqlalchemy.url = sqlite:///data/stillme_migration.db
62+
63+
64+
[post_write_hooks]
65+
# post_write_hooks defines scripts or Python functions that are run
66+
# on newly generated revision scripts. See the documentation for further
67+
# detail and examples
68+
69+
# format using "black" - use the console_scripts runner, against the "black" entrypoint
70+
# hooks = black
71+
# black.type = console_scripts
72+
# black.entrypoint = black
73+
# black.options = -l 79 REVISION_SCRIPT_FILENAME
74+
75+
# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
76+
# hooks = ruff
77+
# ruff.type = exec
78+
# ruff.executable = %(here)s/.venv/bin/ruff
79+
# ruff.options = --fix REVISION_SCRIPT_FILENAME
80+
81+
# Logging configuration
82+
[loggers]
83+
keys = root,sqlalchemy,alembic
84+
85+
[handlers]
86+
keys = console
87+
88+
[formatters]
89+
keys = generic
90+
91+
[logger_root]
92+
level = WARN
93+
handlers = console
94+
qualname =
95+
96+
[logger_sqlalchemy]
97+
level = WARN
98+
handlers =
99+
qualname = sqlalchemy.engine
100+
101+
[logger_alembic]
102+
level = INFO
103+
handlers =
104+
qualname = alembic
105+
106+
[handler_console]
107+
class = StreamHandler
108+
args = (sys.stderr,)
109+
level = NOTSET
110+
formatter = generic
111+
112+
[formatter_generic]
113+
format = %(levelname)-5.5s [%(name)s] %(message)s
114+
datefmt = %H:%M:%S
115+

alembic/README.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Alembic Database Migrations
2+
3+
This directory contains database migration scripts for StillMe.
4+
5+
## Current Status
6+
7+
**Current Database**: SQLite (multiple files)
8+
- `data/knowledge_retention.db`
9+
- `data/continuum_memory.db`
10+
- `data/rss_fetch_history.db`
11+
- `data/accuracy_scores.db`
12+
- `data/vector_db/chroma.sqlite3` (ChromaDB)
13+
14+
**Future Target**: PostgreSQL (single database with schemas)
15+
16+
## Migration Strategy
17+
18+
### Phase 1: Current State (SQLite)
19+
- Alembic is set up but not actively used
20+
- Current schema is managed via `CREATE TABLE IF NOT EXISTS` in code
21+
- Migration scripts will be created to capture current schema
22+
23+
### Phase 2: PostgreSQL Migration Planning
24+
- Design unified PostgreSQL schema
25+
- Create migration scripts to move from SQLite to PostgreSQL
26+
- Use Alembic for schema versioning going forward
27+
28+
## Usage
29+
30+
### Initialize Alembic (First Time)
31+
```bash
32+
# Alembic is already initialized
33+
# To create first migration:
34+
alembic revision --autogenerate -m "Initial schema"
35+
```
36+
37+
### Create New Migration
38+
```bash
39+
# Auto-generate migration from model changes
40+
alembic revision --autogenerate -m "Description of changes"
41+
42+
# Create empty migration
43+
alembic revision -m "Description of changes"
44+
```
45+
46+
### Apply Migrations
47+
```bash
48+
# Apply all pending migrations
49+
alembic upgrade head
50+
51+
# Apply specific revision
52+
alembic upgrade <revision_id>
53+
54+
# Rollback one migration
55+
alembic downgrade -1
56+
57+
# Rollback to specific revision
58+
alembic downgrade <revision_id>
59+
```
60+
61+
### Check Current Revision
62+
```bash
63+
# Show current revision
64+
alembic current
65+
66+
# Show migration history
67+
alembic history
68+
```
69+
70+
## Configuration
71+
72+
- **Config File**: `alembic.ini`
73+
- **Environment**: `alembic/env.py`
74+
- **Database URL**: Set via `DATABASE_URL` environment variable (PostgreSQL) or `SQLITE_DB_PATH` (SQLite)
75+
76+
## Notes
77+
78+
- Alembic is currently set up for future PostgreSQL migration
79+
- Current SQLite databases are managed directly in code
80+
- Migration scripts will be created when PostgreSQL migration begins
81+

alembic/env.py

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
"""
2+
Alembic environment configuration for StillMe database migrations
3+
Supports SQLite (current) and PostgreSQL (future migration target)
4+
"""
5+
6+
from logging.config import fileConfig
7+
from sqlalchemy import engine_from_config
8+
from sqlalchemy import pool
9+
from alembic import context
10+
import os
11+
import sys
12+
13+
# Add project root to path
14+
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
15+
16+
# Import base for autogenerate support
17+
# For now, we'll use declarative_base for future SQLAlchemy models
18+
from sqlalchemy.ext.declarative import declarative_base
19+
20+
# this is the Alembic Config object, which provides
21+
# access to the values within the .ini file in use.
22+
config = context.config
23+
24+
# Interpret the config file for Python logging.
25+
# This line sets up loggers basically.
26+
if config.config_file_name is not None:
27+
fileConfig(config.config_file_name)
28+
29+
# add your model's MetaData object here
30+
# for 'autogenerate' support
31+
# from myapp import mymodel
32+
# target_metadata = mymodel.Base.metadata
33+
# For now, we'll create a base for future use
34+
Base = declarative_base()
35+
target_metadata = Base.metadata
36+
37+
# other values from the config, defined by the needs of env.py,
38+
# can be acquired:
39+
# my_important_option = config.get_main_option("my_important_option")
40+
# ... etc.
41+
42+
43+
def get_url():
44+
"""Get database URL from environment or config"""
45+
# Check for DATABASE_URL environment variable (for PostgreSQL)
46+
database_url = os.getenv("DATABASE_URL")
47+
if database_url:
48+
return database_url
49+
50+
# Check for SQLite path in environment
51+
sqlite_path = os.getenv("SQLITE_DB_PATH", "data/stillme_migration.db")
52+
return f"sqlite:///{sqlite_path}"
53+
54+
55+
def run_migrations_offline() -> None:
56+
"""Run migrations in 'offline' mode.
57+
58+
This configures the context with just a URL
59+
and not an Engine, though an Engine is acceptable
60+
here as well. By skipping the Engine creation
61+
we don't even need a DBAPI to be available.
62+
63+
Calls to context.execute() here emit the given string to the
64+
script output.
65+
66+
"""
67+
url = get_url()
68+
context.configure(
69+
url=url,
70+
target_metadata=target_metadata,
71+
literal_binds=True,
72+
dialect_opts={"paramstyle": "named"},
73+
)
74+
75+
with context.begin_transaction():
76+
context.run_migrations()
77+
78+
79+
def run_migrations_online() -> None:
80+
"""Run migrations in 'online' mode.
81+
82+
In this scenario we need to create an Engine
83+
and associate a connection with the context.
84+
85+
"""
86+
# Override sqlalchemy.url in config with our dynamic URL
87+
configuration = config.get_section(config.config_ini_section)
88+
configuration["sqlalchemy.url"] = get_url()
89+
90+
connectable = engine_from_config(
91+
configuration,
92+
prefix="sqlalchemy.",
93+
poolclass=pool.NullPool,
94+
)
95+
96+
with connectable.connect() as connection:
97+
context.configure(
98+
connection=connection, target_metadata=target_metadata
99+
)
100+
101+
with context.begin_transaction():
102+
context.run_migrations()
103+
104+
105+
if context.is_offline_mode():
106+
run_migrations_offline()
107+
else:
108+
run_migrations_online()
109+

alembic/script.py.mako

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
"""${message}
2+
3+
Revision ID: ${up_revision}
4+
Revises: ${down_revision | comma,n}
5+
Create Date: ${create_date}
6+
7+
"""
8+
from typing import Sequence, Union
9+
10+
from alembic import op
11+
import sqlalchemy as sa
12+
${imports if imports else ""}
13+
14+
# revision identifiers, used by Alembic.
15+
revision: str = ${repr(up_revision)}
16+
down_revision: Union[str, None] = ${repr(down_revision)}
17+
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
18+
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
19+
20+
21+
def upgrade() -> None:
22+
${upgrades if upgrades else "pass"}
23+
24+
25+
def downgrade() -> None:
26+
${downgrades if downgrades else "pass"}
27+

0 commit comments

Comments
 (0)