Architecture

Phoenix Search Architecture

Phoenix Search has two tightly coupled planes:

The API reads from Elasticsearch for search, but MySQL remains the source of truth for patient details and search scope resolution. CDC is the write-side projection system that keeps Elasticsearch usable for low-latency search.

High-Level Architecture

System Boundaries

Runtime Topology

The API and CDC consumer are separate runtimes. They share dependencies and the same target index, but they do not call each other directly. The API only observes CDC through Elasticsearch freshness metrics and health checks.

High-Level Data Flow

Read Path

Write / Sync Path

Low-Level API Architecture

Application Assembly

The FastAPI app is assembled in search/web/application.py.

Starlette middleware runs in reverse add order, so StripPrefixMiddleware is the first custom middleware to see production requests with the ALB prefix.

Startup and Shutdown

search/web/lifespan.py owns process lifecycle.

Route Registration

Development and test environments also register debug routes under /api/debug.

Search Request Internals

Search Context

SearchContext is the immutable bundle that moves through the search pipeline:

Scope Resolution

The caller cannot choose lab_id directly. Lab and org scope always comes from the authenticated session and MySQL lookup helpers.

Query Builder

search/domains/user_search/query.py is intentionally shape-routed:

Important query rules:

• SEARCH_MAPPING selects the allowed logical fields for each search_key.
• search_fields intersects with the selected mapping; no overlap returns an empty result.
• There are no wildcard queries and no fallback catch-all query.
• Search results sort by _score and then last_updated_time desc.
• matched_field comes from ES highlights or named queries.
• Buckets come from named query tags, then the service splits multi-center hits into other_labs.

Elasticsearch Repository

search/domains/user_search/repository.py owns the ES call:

User Detail Lookup Internals

User detail lookup is intentionally MySQL-backed, not ES-backed.

This path uses MySQL so detail views are source-of-truth even if Elasticsearch is temporarily stale.

Low-Level CDC Architecture

Connector Layer

Production connector properties that define the architecture:

Consumer Layer

The consumer is at-least-once. Offsets are stored only after successful handling or after DLQ publication. Replays are expected, so handlers use idempotent writes and ordering guards.

Phase Routing

CDC_PHASE_OVERRIDE can force a phase for recovery. The override must be migration or running.

Source Table Handler Paths

Field Resolver

cdc/consumers/field_resolver.py composes ES documents during running phase.

The resolver treats userDetails.labId_id = -1 as missing. That prevents merged-away patient rows from leaking sentinel data into Elasticsearch.

Retry and Ordering Model

Data Architecture

Source Tables and Projection

Field Ownership

Elasticsearch Document Shape

The ES document combines identity plus aggregate arrays:

{
  "id": 101,
  "lab_id": 1,
  "full_name": "John Doe",
  "lab_patient_id": "P-1001",
  "contact": "9999999999",
  "manual_sample_ids": ["S-1001"],
  "order_numbers": ["ORD-1001"],
  "lab_bill_ids": ["5001"],
  "org_ids": [20],
  "referral_ids": [77],
  "branch_ids": [10],
  "last_updated_time": "2024-01-02T10:00:00Z"
}

The API search path depends on ES routing by lab_id. If a patient moves labs, CDC deletes the old routed document and indexes the new routed document.

Elasticsearch Routing Model

Phoenix Search uses Elasticsearch custom routing on the user_details index. The index mapping declares _routing.required = true, so every write, delete, and point lookup must pass the same routing key that was used when the document was indexed.

Routing is not the only security boundary. The API also adds lab_id / org / branch / referral filters from the authenticated session. Routing targets the relevant ES shard or shards; filters enforce the allowed result scope.

Write paths must use the same rule:

Debugging must include routing. A document can exist under one routing key and appear missing under another:

curl -s -u elastic:<PASSWORD> \
  "https://<ES_HOST>:9200/user_details/_doc/<USER_DETAILS_ID>?routing=<LAB_ID>"

Consistency Model

The important rule is that Elasticsearch is a projection, not the source of truth. When ES and MySQL disagree, MySQL wins and CDC/backfill should repair ES.

Failure Domains

Source References