# Application Form Copies

New table `copies` lives in partner's database. Possible attributes:

- `id` (uuid)
- `application_form_id` (uuid)
- `title` (string)
- `owner_id` (uuid)
- `locked_at` (datetime)
- `soft_deleted_at` (datetime)
- `schema_version` (integer)
- `document` (jsonb/protobuf)
- protobuf cans be [~3.5x faster](https://hackernoon.com/protobuf-vs-json-in-the-ruby-world) in serializing/deserializing compared to Oj
- spike on performance here on writing answers
- spike on breaking this document up into multiple columns and performance benefits
- `last_touched_by_user_at` (datetime)
- in lieu of app form having active copy id
- `created_at` (datetime)
- `updated_at` (datetime)

Possible document...

:::info
This removes the need for the following tables:
- `answers`
- `answer_choices`
- `answer_pools`
- `application_forms_question_sets`
- `forced_exclusions`
:::

{
  afqs_list: [{
    id: '<UUID>',
    question_set_id: '...',
    remove_if_ineligible: true,
    completed_at: '...',
    added_by_type: '...',
    added_by_id: '...'
    created_at: '...',
    updated_at: '...'
  }, {
    id: '<UUID>',
    question_set_id: '...'
    remove_if_ineligible: true,
    completed_at: nil,
    added_by_type: '...',
    added_by_id: '...'
    created_at: '...',
    updated_at: '...'
  }],
  
  # Spiking on flat answers vs answers nested under pools
  # Spike on answers being a hash with { id => { ... } }
  answers: [{
    id: '<UUID>',
    question_id: '...', # mqs_first_name (text)
    value: 'John',
    active: true,
    required: true,
    is_readonly: false,
    display_type: 'public',
    answered_by_type: '...',
    answered_by_id: '...',
    answered_by_source: '...',
    created_at: '...',
    updated_at: '...'
  }, {
    id: '<UUID>',
    question_id: '...', # mqs_fein (text / sensitive)
    encrypted_value: '...',
    encrypted_value_iv: '...',
    # ...
  }],
  
  answer_pools: [{
    id: '<UUID>',
    question_pool_id: '...', # locations
    soft_deleted: false,
    created_at: '...',
    updated_at: '...',
    answers: [{
      id: '<UUID>',
      question_id: '...', # mqs_loc_state (dropdown)
      choice_id: '...',
      # ...
    }],
    answer_pools: [{
      id: '<UUID>',
      question_pool_id: '...', # buildings (nested under locations)
      soft_deleted: false,
      created_at: '...',
      updated_at: '...',
      answers: [{
        id: '<UUID>',
        question_id: '...', # multi-select question
        choice_ids: ['...', '...']
        # ...
    }],
    # ...
  }],
    
  forced_excluded_carrier_product_ids: ['...', '...'],
    
  # These are moved from app forms table. Please review the
  # app forms table and verify more don't need to be moved over...
  agent_bypassed_carrier_product_ids: ['...', '...'],
}

By nesting pools under each other we remove the need for `:parent_pool_id`. It also makes deleting pools simpler. It also better matches our quote start API.

Tables that need a `copy_id` column:

- `application_form_processes`
- These are tied to answers, so need to know the copy
- `quote_requests`
- This is the whole reason we are doing copies
- `eligibility_results`
- These are sometimes tied to an AFQS, so need to know the copy
- `messages` Types include: `:application_form`, `:answer`, `:real_time_eligibility`, `:quote_request`. But the `metadata` column can contain any identifying information for the given type.
- Add `copy_id` column
- `application_form_disposition` ???
- Adi will follow-up with product on this
- `events` ???
- Types include:
- `:quote_request_selected`
- `:application_form_retrieved`
- `:external_transfer`
- `:guest_flow_email_sent`
- `:agent_needs_bind_assistance_email_sent`
- `:bridged_to_off_platform_carrier`
- These can be created everytime a user clicks on something
- Do we allow more than one email to be sent to an agent (one for each copy? or one for each app form?)
- There is a discussion about removing `:application_form_id` and making a polymorphic column that points to what object it is applied on
- `naics_searches` ???
- Adi will follow-up with BI on this
- But sounds like we might be able to keep this on the app form
- `payment_methods` ???
- Keep on app form for now
- `tokens` ???
- Only ever used on guest flow to access their app form
- Jared thinks this should live on the copy now
- Jared thinks this might also be used in our websocket cable
- So maybe add `copy_id` column

Do we slowly deprecate the `:application_form_id` columns for the above?

### Notes:

- Must live behind a feature flag. Feature flag is controlled by owner.
- We will need to detect if an app form is a "copy"-type app form dynamically and progress forward with that regardless whether the feature flag is on or off.
- On app form creation when the feature flag is enabled we will create a default copy for the app form. Do we want the default copy id to match the app form id? This will make it more backwards compatible.
- No two instance can work on a copy at any given time. So we will want to lock on copy id.
- Or we let it raise an error if the nonce is not what is expected
- When no copy id can be inferred then we use the copy with the latest `last_touched_by_user_at` value.
- As above we would then obtain a lock on this copy.
- Front-end is serial but app form processes and quote request imports are async.
- Need to speed up quote request import
- Need to get smart about when we check out the copy lock for app form processes, e.g.,
1. Create context and checkout lock before submitting to third-party API
2. Release lock and submit to third-party API
3. Create new context and checkout lock again
- For long-lived processes (e.g., app form processes, quote requests, ...) we may need a more long-lived lock that blocks copying of a copy (~5 min)
- This could be a short-term solution and after we clean these up we could support them updating multiple copies at once
- Cloning...
- Within tenant: We clone all copies and re-run rules, but only re-quote last touched copy
- Across tenants: We clone ONLY the requested copy (if no copy id supplied then we assume last touched copy)
- What about locked app forms?
- Only mark quotes state on other copies for specific carriers (e.g., NW). Should this property live on the tenant in Authenticator?
- Maybe for day 0 we mark all carrier quotes as stale when quoting in another copy
- App form state (and copy state) are dynamically derived.
- Perform fuzzy lookup of app form id and check if it is really a copy id?
- First lookup id in copies table, if not found then look in app forms table
- Great for backwards compatibility.
- Doesn't stop us from creating new copy-only endpoints as we slowly migrate over.
- Explore how referrals work in the future.
- Explore putting the KMS key on the copy and copying that key to all copies. Even copying the KMS key during cloning.

### Endpoints

Create a new copy on an app form:

POST /application_forms/:application_form_id/copies

Do we want to make all other endpoints "shallow" (i.e., only `/copies/:id`)? It seems in cases like `answers`, `answer_pools`, AFQS's, ... we put all of these under the app form (i.e., `/application_forms/:app_form_id/*`).

So for example, if we wanted to soft delete a copy on an app form:

DELETE /application_forms/:application_form_id/copies/:id

or we could have...

DELETE /copies/:id

Maybe we should slowly replace the relevant `/application_forms/:app_form_id/*` endpoints with `/copies/:copy_id/*`, but do fuzzy matching on `:app_form_id` in the meantime until a cutover.

After discussion...
- Only keep it under the `/copies/*` namespace
- Utilize current endpoints as much as possible for backwards compatibility
- Spin off new endpoints as we notice a discrepancy
- Look into deprecating old endpoints

### User Stories Notes

-