Phoenix Search
FastAPI search microservice for patient and user lookup
Phoenix Search
Phoenix Search is a FastAPI search microservice for patient and user lookup. It serves authenticated search requests from LiveHealth surfaces, queries the user_details Elasticsearch index, and uses MySQL plus Redis for source data, session context, rate limiting, and operational checks.
CDC is part of Phoenix Search. The CDC pipeline keeps the Elasticsearch projection in sync from MySQL through Debezium, Redpanda, the Phoenix CDC consumer, the user_meta projection table, and the Elasticsearch ingest pipeline.
Start with Start Here for the reading order. For the full high-level and low-level system view, see Architecture. For frontend endpoint and ephemeral-token auth, see Auth and Endpoints.
Repository Information
| Property | Value |
|---|---|
| Repository | phoenix-search |
| Language | Python 3.13+ |
| Main Branch | main |
| Framework | FastAPI |
| Search Backend | Elasticsearch |
| Primary Index | user_details |
| CDC Runtime | Python consumer + Debezium + Redpanda |
Production Snapshot
Latest saved dashboard screenshots for production IN show Phoenix Search handling the observed traffic window with healthy service and Elasticsearch signals. The full migration evidence, old-vs-new index comparison, and OpenTelemetry notes are in Post-Migration Results.
| Signal | Observed Production Value |
|---|---|
| API request error rate | 0% in the HTTP service dashboard |
| Main endpoint | POST /api/v1/users/search, about 97% of endpoint time |
| Search endpoint latency | Median about 22.76 ms, p95 about 47.67 ms |
| ES query latency | p50 about 2.5 ms, p95 about 4.75 ms, p99 about 4.95 ms |
| CDC health | 1, with ES document freshness around 3.6s |
| ES cluster | GREEN, 3 active data nodes |
user_details index | About 157.44M docs, 67.61 GB primary, 135.75 GB total |
| Storage reduction | About 47% lower than the old userdetails index, roughly 50% in practical terms |
The detailed dashboard snapshot and migration results live in Post-Migration Results.
Clone Repository
git clone git@github.com:CrelioHealth/phoenix-search.gitTech Stack
| Category | Technology | Purpose |
|---|---|---|
| Runtime | Python 3.13+ | API and CDC consumer runtime |
| Web Framework | FastAPI | HTTP API server |
| Search | Elasticsearch | Patient and user search index |
| Cache / Sessions | Redis | Django session lookup, KV support, rate-limit config |
| Database | MySQL | Source-of-truth user data and user_meta projection |
| CDC Broker | Redpanda | Kafka-compatible event transport |
| CDC Source | Debezium MySQL Connector | MySQL binlog capture |
| CLI / Packaging | uv, Typer | Local commands and project execution |
| Observability | Prometheus, OpenTelemetry, HyperDX, Sentry | Metrics, traces, logs, and errors |
System Architecture Overview
Key Components
1. FastAPI Application (search/web/application.py)
The app factory registers monitoring routes at the root and API routes under /api. Production traffic can arrive behind the ALB path prefix /phoenix-search; StripPrefixMiddleware removes that prefix before FastAPI routing.
| Route Area | Registered Path | Notes |
|---|---|---|
| Home and health | /, /health, /health/live, /health/ready | Monitoring routes are public |
| Metrics | /metrics | Prometheus exposition endpoint |
| OpenAPI | /api/openapi.json | FastAPI OpenAPI document |
| Docs | /docs | FastAPI Swagger UI |
| User Search | /api/v1/users/search | Authenticated user search API |
2. User Search Domain (search/domains/user_search/)
The domain owns its router, schemas, service, repository, filters, and Elasticsearch query builders.
| File | Role |
|---|---|
router.py | FastAPI endpoints and dependencies |
schemas.py | Request and response models |
service.py | Session-aware search orchestration, hit bucketing, MySQL detail lookup |
repository.py | Elasticsearch query execution and circuit-breaker handling |
constant.py | SEARCH_MAPPING keys and searchable fields |
Search requests are scoped by the authenticated session. The service resolves lab, branch, organization, referral, and multi-center constraints before querying Elasticsearch, sends ES routing from the resolved lab_id values, then groups hits into registered, samples, orders, and other_labs.
3. Authentication and Session Context
SessionAuthMiddleware authenticates every non-guest route. It supports:
| Client Type | Auth Mechanism |
|---|---|
| Web app | sessionid cookie resolved from Redis-backed Django sessions |
| Mobile app | Bearer JWT when X-App-Name is present |
| Ephemeral web flow | Bearer JWT without X-App-Name |
The validated session is stored on request state and in a context variable so route dependencies can resolve lab_id, organization scope, and search restrictions.
4. Lifecycle and Observability (search/web/lifespan.py)
Startup initializes OpenTelemetry, Prometheus instrumentation, Elasticsearch, MySQL, Redis, and rate-limit Redis. A background CDC freshness probe periodically queries the newest last_updated_time in Elasticsearch and exposes freshness through /health plus metrics.
5. CDC Subsystem (cdc/)
CDC is documented in this section because it is the write-side sync path for Phoenix Search. It is not a separate service in the docs navigation. See Architecture for the system view, CDC for the pipeline, and Operations for runbooks.
Local Setup
Prerequisites
| Tool | Version / Requirement |
|---|---|
| Python | 3.13+ |
| uv | Latest |
| Docker | Latest |
| MySQL | 8.x source database; hosted externally for dev, Docker for tests |
Quick Start
make install
cp .env.example .env
cp docker/.env.example docker/.env
make up-dev
make seed
make runThe API runs on http://localhost:8000, and FastAPI docs are available at http://localhost:8000/docs.
Services and Ports
| Service | Dev | Test | Description |
|---|---|---|---|
| API | 8000 | 8010 | FastAPI server |
| Elasticsearch | 9210 | 9212 | Search backend |
| Kibana | 5601 | - | Elasticsearch dashboard |
| Redis | internal | internal | Cache, sessions, rate-limit state |
| MySQL | 3306 | 3308 | External dev DB, Docker test DB |
| HyperDX | 8080 | - | Observability UI |
Core Flows
Search Request
Detail Lookup
MySQL to Elasticsearch Sync
Make Targets
| Command | What it does |
|---|---|
make install | Install Python dependencies |
make run | Start the API server |
make up-dev | Start Redis, Elasticsearch, and Kibana |
make up-all | Start all local infra plus HyperDX |
make down | Stop all containers |
make test | Run unit tests without Docker |
make test-docker | Run the full suite in Docker |
make lint | Run Ruff and mypy |
make seed | Seed 50 users across 2 labs |
make seed-reset | Drop, recreate, and seed fresh local data |
make register-pipeline | Register the Elasticsearch ingest pipeline |
make run-cdc | Start the local CDC stack |
make connector-status | Show Debezium connector health |
make backfill-migrate | Populate the MySQL projection table |
make backfill-run | Push projected data into Elasticsearch |
Source References
| Concern | Source |
|---|---|
| App factory and ALB prefix handling | search/web/application.py |
| Route registration | search/web/api/router.py |
| User search endpoints | search/domains/user_search/router.py |
| User search request and response models | search/domains/user_search/schemas.py |
| Search orchestration | search/domains/user_search/service.py |
| Elasticsearch query execution | search/domains/user_search/repository.py |
| Query shape routing | search/domains/user_search/query.py |
| CDC runbook | cdc/RUNBOOK.md |
| CDC settings | cdc/consumers/config.py |