Backend Implementation

The Integration Dashboard's backend decouples business triggers from transport and logging. Core workflows (billing, registration, sample lifecycle) complete without waiting for third-party responses. Logging is a passive side effect, not part of the execution path.

Architecture

Core Building Blocks

1. Event Vocabulary (actionCategory)

All integrations are keyed to a two-level event vocabulary:

• actionCategory — broad business group (e.g., Billing, Sample Lifecycle, Registration).
• actionCategoryList — concrete triggerable event (e.g., "Bill Generation", "Sample Receive", "Report Signed").

2. Integration Root Record (developerAuthentication)

The master record for an integration account. It holds:

• Auth key and developer identity.
• Global enabled/disabled toggles.
• Lab/org ownership.

3. Delivery Configuration (labIntegration)

Maps an Action ID to a specific endpoint. Controls:

• Where to send the data — URL, or host/port for Mirth.
• How to send it — GET or POST, batch or single.
• Behavioral flags — is_auto_retry, is_logs_disabled, specialized transport (Mirth via TCP, Fax via vendor).

4. Observability Layer (DocumentDB)

Operational logs live in DocumentDB because integration payloads are schema-variable and retry history needs append-only writes:

Logging Decorators

Logging is implemented as passive decorator classes applied to specific functions. They observe the function they wrap, capture input/output, and persist a structured record. They do not route requests, queue tasks, or make any integration decisions.

IntegrationDirectoryLogger

Applied to: Inbound API view functions — third-party systems calling LiveHealth APIs.

How it works:

1. Extracts the requestToken from the incoming request to resolve lab_id, developer_id, and identity fields.
2. Calls the wrapped view function without any modification.
3. After the function returns, assembles a log payload from:
   • Request: body, headers, params, IP address, origin (mobile/web).
   • Response: status code, body, response time.
   • Business context: bill ID, patient ID, LRR ID (derived from the response if available).
4. Saves the assembled payload to IntegrationDirectory via DocumentDbClient.

Broken transaction handling: If the current DB transaction has needs_rollback=True (inside an atomic() block poisoned by an exception), the decorator cannot write to DocumentDB directly. It falls back to a Fusion webhook pointing to an internal log-creation endpoint, ensuring the record is persisted once the atomic block exits.

Short-circuit conditions (decorator skips logging entirely and just runs the function):

• DOCUMENT_DB_ENABLED is False.
• Request body cannot be parsed.
• is_api_retry flag is set in the body (retry calls skip re-logging).
• No valid requestToken found for the auth key.

@csrf_exempt
@IntegrationDirectoryLogger()
def appointmentConfirmAPI(request, authKey):
    ...

WebhookIntegrationDirectoryLogger

Applied to: Internal webhook handler functions that perform the actual outbound call to a third-party system — Mirth handoff, fax delivery, report email, outsource billing.

These handlers are invoked by Fusion workers. The incoming request body always carries an integration_payload dict, built earlier by get_integration_payload() and passed through the Fusion task.

How it works:

1. Reads integration_payload from the request body.
2. Calls the wrapped handler function without any modification.
3. After the handler returns, inspects the response to determine outcome.

Status resolution:

• SUCCESS — HTTP 2xx and (where applicable) sent_to_mirth=True.
• FAIL — Non-2xx or explicit error body.
• QUEUED — Special case for async fax vendors (e.g., HumbleFax) where delivery is deferred.
• "" (blank) — Response code 209 or suppression tag in body — used to suppress noisy/expected errors from the dashboard view.

# Outbound handler — updates the pre-created log
@WebhookIntegrationDirectoryLogger(is_outbound_webhook_endpoint=True)
def send_data_to_mirth(request):
    ...

# Fax/report handler — creates a new log after completion
@WebhookIntegrationDirectoryLogger(is_retry_enabled=True)
def send_report_email_webhook(request):
    ...

IntegrationDirectoryLogger (crelio-app / py3)

Repository: crelio-app (Python 3)

A unified decorator that consolidates the responsibilities of both py2 decorators into a single class. It is the primary logger for integrations triggered from the py3 service. Currently active for Action IDs 60 (Claim Submission Webhook) and 61 (Claim Approval / Remittance Webhook), and is intended to become the standard for all new integrations built in crelio-app.

Modes — controlled by constructor flags:

How it works:

1. Extracts the HttpRequest or DRF Request object from function arguments.
2. Depending on the mode, builds the integration_payload, resolves lab_id, and (for outbound) reads log_id from the request body.
3. Calls the wrapped function without any modification.
4. After the function returns (or raises), parses the response to determine status, error_message, response_time, and sent_to_mirth.
5. Persists the assembled log:
   • Outbound mode: updates the existing IntegrationDirectory doc by log_id.
   • Retry mode: delegates to IntegrationRetryLog.update_existing_record_and_push_update().
   • All other modes: saves a new IntegrationDirectory document.

Broken transaction handling: Same pattern as py2 — if connection.needs_rollback=True inside an atomic() block, the decorator falls back to a Fusion webhook (/api-v3/integration/logs/new or /api-v3/integration/logs/update) to persist the record out-of-band.

Short-circuit conditions (decorator skips logging and just runs the function):

• integration_payload or lab_id cannot be resolved.
• Outbound mode: log_id is missing.
• Third-party mode: always proceeds (no short-circuit — payload is built from metadata).

Status resolution:

• SUCCESS — HTTP 2xx (and sent_to_mirth=True for Mirth outbound).
• FAIL — Non-2xx or explicit error body.
• QUEUED — HumbleFax vendor (fax delivery is async).
• "" (blank) — Response code 209 or #NOTFORDASHBOARD in error message — suppresses record from dashboard view.

# Inbound API endpoint (default mode)
@IntegrationDirectoryLogger()
def get(self, request):
    ...

# Outbound webhook handler (updates pre-created log)
@IntegrationDirectoryLogger(is_outbound_webhook_endpoint=True)
def post(self, request):
    ...

# Retry-enabled handler
@IntegrationDirectoryLogger(is_retry_enabled=True)
def post(self, request):
    ...

get_integration_payload()

Location: crm/integrations/utils.py

This is the standard payload builder called by trigger functions before queuing a Fusion task. It is a regular function returning a dict, not a decorator.

What it builds:

• Business context: lab_id, lab_bill_id, patient_id, lrr_id, report_id, test_names.
• Integration metadata: action_category_id, action_category_list_id, developer_id, webhook_id.
• Dispatch config: request_type, is_batch, end_point, extra_data.
• Retry context: serialized retry_context for future intent reconstruction.

Gatekeeping — returns {} and skips logging if:

• labIntegration.is_logs_disabled is True.
• labIntegration.url is empty.
• The endpoint URL resolves to a path in INBOUND_API_PATH_LIST (prevents self-referential logging loops).

The returned payload is passed to save_log_instance() to pre-create a QUEUED-equivalent log before Fusion dispatch. The resulting log_id (the DocumentDB _id) is injected back into the payload so that WebhookIntegrationDirectoryLogger can locate and update the correct record when the handler completes.

Outbound Push — Step by Step

The trigger function does not make the HTTP call itself. It queues a Fusion task, and the Fusion worker (webhooks.py) handles the actual dispatch. Status is reported back via integration_callback_url.

Example: Billing API Trigger Flow

To understand how a particular endpoint triggers an integration and captures a log, consider the Bill Generation flow:

1. Endpoint Triggered: The client hits the billing API, routed via url(r'^billing/$', BillingView.as_view()).
2. Business Logic: The BillingView class processes the POST request and delegates to the bill_patient() core utility function to create the bill.
3. Trigger Evaluation: Once the bill is saved and finalized, bill_patient() calls commomFunctionForIntegrationBillCategory(). This function evaluates active integrations for the Bill Generation action category.
4. Payload Construction: Inside the trigger function, get_integration_payload() is executed to prepare the exact integration payload.
5. Log Capture: During payload construction, a log document is pre-created in DocumentDB (with a QUEUED status). The generated log_id and the integration_callback_url are embedded into the payload.
6. Task Dispatch: commonFunctionForIntegrationWebhookCall() is then called, which passes the payload to Fusion.
7. Callback Execution: The Fusion worker makes the outbound HTTP call to the third party. Upon completion, it POSTs the result back to the integration_callback_url, where @WebhookIntegrationDirectoryLogger updates the previously captured log with SUCCESS or FAIL.

Key points:

• integration_callback_url is baked into integration_payload by get_integration_payload() at trigger time. The Fusion worker doesn't need to know anything about the dashboard — it just POSTs to this URL after every call.
• The suppression check (code 209 / #NOTFORDASHBOARD) happens inside the Fusion worker, before the callback fires. Suppressed responses never reach WebhookIntegrationDirectoryLogger.
• The x-internal-token header on the callback POST identifies it as an internal system call.

Retry Mechanism

Failed integrations can be retried manually from the dashboard or automatically based on labIntegration config.

Manual Retry

Initiated via the dashboard UI, routed through RetryIntegrationView.

1. Fetches the parent IntegrationDirectory document by log_id.
2. Reads action_category_list_id to look up the corresponding trigger function via ACTION_CATEGORY_TO_META.
3. Reads retry_context from the original log to reconstruct the business arguments.
4. Calls the appropriate _prepare_kwargs_* function to rebuild fresh domain objects.
5. Calls the original trigger function with the rebuilt arguments.
6. WebhookIntegrationDirectoryLogger (with is_retry_enabled=True) captures the retry outcome and calls IntegrationRetryLog.update_existing_record_and_push_update().
7. The parent IntegrationDirectory document's retry_count is incremented and status updated.
8. A Pusher notification is dispatched via UpdateRetryIntegrationStatus to update the dashboard in real time.

Auto-Retry

Controlled per labIntegration row:

• is_auto_retry — enables the feature for that action.
• auto_retry_attempts — maximum automatic attempts before giving up.

Intent Reconstruction (_prepare_kwargs_*)

Retries do not replay the original serialized request body. The system maps the log back to its triggering function and rebuilds domain objects from the database, ensuring any state changes since the first failure (e.g., updated patient record, corrected bill) are included.

• Each retriable action maps to a _prepare_kwargs_* function in crm/integrations/utils.py.
• These functions read retry_context (saved in the original log) and use it to re-fetch live ORM objects.
• The rebuilt kwargs are passed directly to the original trigger function.

Mirth / HL7 Handoff

For Mirth integrations, the transport is TCP socket (not HTTP). The trigger function calls send_hl7_manual_sample_update() or equivalent, which:

1. Reads host, port, and transport settings from labIntegration.integrationExtraDetails.
2. Calls update_message_with_ids() to inject LABID and LOGID into the HL7 message body for downstream traceability, and stamps the log document with stage="LIVEHEALTHAPI".
3. Opens a TCP socket and sends the formatted HL7 message to Mirth.
4. WebhookIntegrationDirectoryLogger wraps the handler and records the outcome.

Debugging Mental Model

When investigating or extending an integration, trace these four steps:

1. Identify the event — What business action fired? (e.g., bill generated, sample received).
2. Check configuration — Does the lab have an active labIntegration row for that actionCategoryListId?
3. Trace the trigger — Which function in integration_functions.py calls get_integration_payload() and queues the Fusion task?
4. Read the log — Find the IntegrationDirectory document. Check status, error_message, response_body, and retry_count.