# Dentally: Just-in-time Syncing

Just-in-time syncing involves:

1. Sync **all** patients from the client's Dentally account
2. Syncing the patient's data on-demand at the point of linking

This should remove the need for excessive API calls to Dentally to fetch all the data at once and at least make the integration more usable.

The goals of this change are to:

1. Make financial data available for **all** Dentally patients
2. Do so without being rate limited by Dentally

Auto-linking and pulling more data than we need to improve the existing Dentally integration are explicit non-goals.

In summary, we need to:

1. Temporarily disable Dentally to eliminate backwards compatibility concerns
2. Track the link state beyond existence of linking (e.g linked, syncing, completed) so we only sync data
3. Implement a mechanism to sync all of a patient's data just-in-time upon linking
4. Update the UI to show that we are extracting data just-in-time using some form of 'syncing' state

## Design

### UI

#### Types

We will need to fine-tune linking state so that a link can be:

1. Unlinked - the default state
2. Linking - the user is waiting for a request to link to complete
3. Syncing - the patient is linked but we're waiting for the patient to sync
4. Linked - the patient is linked and up-to-date

We should also streamline the Dentally state types to merge disparate things together, which should reduce complexity enough that the additional complexity from just-in-time loading balances out.

interface PatientDetails {
	id: number;  
	date_of_birth: string;  
	first_name: string;  
	last_name: string;  
	middle_name: string | null;  
	email_address: string;  
	home_phone: string;  
	phone: string;  
	work_phone: string;
}

interface Invoice {  
    id: number;  
    date: string;  
    amount: string;  
}  
  
interface Treatment {  
    id: number;  
    name: string;  
    amount: string;  
}

interface PatientFinancials {
	dentally_id: number; // this is erroneously in camel case
	  
	invoices: {  
	    unpaid: Array<Invoice>;  
	    paid: Array<Invoice>;  
	};  
	  
	uninvoiced_treatments: Array<Treatment>;  
	  
	total: {  
	    paid: string;  
	    unpaid: string;  
	};
}

interface SyncingState {
	state: 'syncing';
	details: PatientDetails;
}

interface SyncedState {
	state: 'synced';
	details: PatientDetails;
	financials: PatientFinancials;
}

type LinkState =
	| { state: 'disabled' }
	| { state: 'loading' }
	| { state: 'unlinked' }
	| { state: 'linking' }
	| SyncingState
	| SyncedState;

interface LinkStore {
	[patientId: string]: LinkState;
}

_Can we encapsulate whether Dentally is disabled within the state? This wouldn't actually be stored but could be a valid state returned by a selector if it's disabled, encapsulating this complexity_

_Another alternative is to short-circuit the `getLink` action if dentally is some how disabled, which would store `disabled` in state_

_In either case, the intention is to pull in and encapsulate the complexity of checking whether Dentally is disabled, hiding how exactly Dentally is disabled._

#### State

We should simplify the Dentally state to merge link state, the patient record and the patient's financial data so that:

1. Patient details are available in syncing and linked state
2. Financial details are only available in linked state

We will need to respond to the `DentallyPatientSynced` websocket message to then refresh the link state, which will include the patient's details and financial data. This means we will need to introduce a `dentally/patient/synced` action.

The `getLink` action will now be responsible for fetching:

1. The link state (e.g unlinked, syncing, synced)
2. The patient details
3. The financial details
4. Short-circuiting if Dentally is disabled

We will also need to remodel the selectors around the streamlined state. We will need to know:

1. The patient's link state
2. Helpers to narrow the types of each state shape (and test state)
4. A helper to select the patient details from the object
5. A helper to select the patient financials from the object

#### Components

We will need to modify these components to respect the refined UI states:

1. `Dentally/Linking.tsx`
2. `Dentally/LinkBox.tsx`
3. `Patient/Patient.tsx`
4. `Patient/Attributes.tsx`
5. `Patient/NameBlock.tsx`

##### `Patient/Attributes.tsx`

We need to change the prop type to accept the complete Dentally record instead of just financials.

##### `Patient/NameBlock.tsx`

We need to:

- Replace financials in props with complete Dentally link state
- Replace testing of financial state with outlined helpers
- Refactor financial UI logic to separate, co-located component, isolating the complexity of deciding financial state

##### `Dentally/Linking.tsx`

This component renders a summary of the financial data using a bar with paid and potential values.

Currently, this uses `financial` state to test if it is loading and as the store for paid and unpaid totals. With the streamlined updates to state, this should be in the loading state when `syncing`.

##### `Dentally/LinkBox.tsx`

This component renders details about the Dentally patient, along with interactions to unlink or view the patient in Dentally.

We will need to:

- Remove testing if Dentally is ready
- Replace type to accept complete Dentally link state
- Remove selection of financial data
- Replace prop with link state select and use helpers to select patient and financial state
- Replace passing financial data to `NameBlock` with complete, current link state

##### `Patient/Patient.tsx`

This component interfaces with the state management layer, initiating actions to load the Dentally patient.

We will need to:

- Remove testing if Dentally is ready
- Remove loading of financial details.
- Replace selection of link and financial state with the new selectors.

##### `Patient/Integrations.tsx`

- Replace testing if Dentally is ready with state selector
- _Does this really need to query state?_

##### `Patient/IntegrationPanel.tsx`

- Replace testing link state with selectors and helpers

### API

#### Domain

We will need to expand the `Link` value with:

final class Link  
{  
    public function __construct(  
        public readonly int $id,  
        public readonly UuidInterface $patientId,
        public readonly DateTimeImmutable $createdAt,
        public readonly ?DateTimeImmutable $syncingAt,
        public readonly ?DateTimeImmutable $syncedAt
    ) {  
    }
}

#### Persistence

We will need to add the following fields to the `dentally_patient_links` table:

- `created_at` - this tells us when the link was created
- `syncing_at` - this tells us when the link started syncing at, which also tells us that it is in the syncing state if not null and `synced_at` is null
- `synced_at` - this tells us when the link completed syncing, which also tells us that it is in the syncing state if not null

This means we will need to update `LinkGateway` implementations to persist and load these new fields

#### Application

##### Syncing

We will need to change the `SyncReceiver` to pull **all** patients on first sync. This can be achieved by reducing the scope of `TimeBasedExtractor` to only query the `patients` endpoint.

If the `last_synced_at` is null, we will sync from `1970-01-01` instead of 6 months ago. This should reliably extract and load **all** a practice's patients on first sync, and then reverting to syncing last updated patients in response to webhook events.

If `last_synced_at` is not null, we should also sync dependent objects for the extracted patients (to prevent attempting to sync a large amount of data from the full sync). This will then use `JustInTimeExtractor` instead of `DependencyExtractor`.

`DependencyExtractor` can be removed.

##### Linking

In addition, we will need a new extractor focused on pulling objects based solely on patient ID:

interface JustInTimeExtractor {
	public function get(int $patientId): array;
}

This interface will extract:

- Treatment plans
- Treatment plan items
- Invoices
- Appointments
- Treatment appointments

This will be used in response on a `DentallyPatientLinked` event which must be emitted when a Leadflo patient is linked to a Dentally patient.

The `SyncDentallyPatient` listener will be responsible for:

1. Updating the state of the link to syncing
2. Extracting all data for the patient
3. Loading all data for the patient into the DB
4. Updating the state of the link to completed
5. Emitting the `DentallyPatientSynced` event

Once the extracted objects are loaded into the DB, we must emit a `DentallyPatientSynced` event, which is broadcastable and used by the UI to pull the Dentally financial data for the patient.

##### Queries

We will need to combine the `Patient` and `PatientFinancials` queries into a single read model.

This will then also need a state:

enum LinkState : string
{
	case Linked: 'linked';
	case Syncing: 'syncing';
	case Synced: 'synced';
}

We must also _remove_ reference to `dentally_accounts` as we are no longer syncing this object.

## Implementation Plan

Preliminary:

- [ ] Send out an email about disabling the Dentally integration pending improvements
- [ ] Disable the Dentally integration whilst we work on planned improvements

Testing:

- [ ] Cover `Patient/NameBlock.tsx` with a unit test
- [ ] Cover `Patient/Attributes.tsx` with a unit test
- [ ] Cover `Dentally/Linking.tsx` with a unit test
- [ ] Cover `Patient/Integrations.tsx` with a unit test
- [ ] Cover `Dentally/LinkBox.tsx` with an integration test
- [ ] Cover `Patient/Patient.tsx` with an integration test
- [ ] Cover Dentally linking with e2e test

UI:

- [ ] Consolidate the Dentally UI link state into a new, single slice
- [ ] Implement UI state selectors
- [ ] Implement UI state helpers
- [ ] Switch `Patient/Attributes.tsx` to new state slice
- [ ] Switch `Patient/NameBlock.tsx` to new state slice
- [ ] Switch `Dentally/Linking.tsx` to new state slice
- [ ] Switch `Dentally/LinkBox.tsx` to new state slice
- [ ] Switch `Patient/Patient.tsx` to new state slice
- [ ] Switch `Patient/Integrations.tsx` to new state slice
- [ ] Switch `Patient/IntegrationsPanel.tsx` to new state slice

API:

- [ ] Define new columns for `dentally_patient_links` table
- [ ] Expand `Link` value with additional fields
- [ ] Update `LinkGateway` implementations to support new fields
- [ ] Reduce `TimeBasedExtractor` implementations to only extract `patient` objects
- [ ] Define and implement `JustInTimeExtractor`
- [ ] Define `DentallyPatientLinked` event and dispatch on link
- [ ] Define `DentallyPatientSynced` event
- [ ] Implement `SyncDentallyPatient` listener
- [ ] Modify `SyncReceiver` to pull all and only patients on initial sync
- [ ] Modify `SyncReceiver` to use `JustInTimeExtractor` on partial syncs
- [ ] Consolidate `Patient` and `PatientFinancials` queries into a single read model

## Testing

- There exists a suite of unit and integration tests in the API
- We should expand this by covering the UI components with integration tests to maintain functionality
- We should consider expanding e2e tests to test Dentally functionality

## Future considerations