---
# System prepended metadata

title: Home Office

---

# Home Office

## Bodies of Work

At a high-level, the work required for Home Office breaks down into four buckets:

* Data Parsing
    * E-mail Parsing
    * OCR
* Data Processing (Batch)
    * Apply algorithms and create summary JSON to power the UI
* Algorithms
    1) Extractive
    2) Classifiers
* UI

## Sprint Roadmap

1) Sprint 1 (Wrapping July 27th)
    * Batch processing/UI pipeline working 
    * Demonstrate the solution can identify key names, locations, dates and identify the name of a MP and their constituency
        * Should probably incorporate Robinson's improvements here (QALEX + People Rez + Affiliations)
        * Incorporate @Kiran's UI improvements
    * Identify the questions being asked by the MP and constituent
    * Prove the ability of linking the questions in outcome 2 to the 'application type'
3) Sprints 2-4 (???)
    * Data Science creates extractive/classification functions based on user input
    * Templated NLG
    * Integrate Robinson's People 
    * Better drill-down to document
    * Work with client to draft report/presentation deck


## Data Processing + Parsing

1) Start with a case, consisting of an e-mail and multiple attachments
2) Parse attachments (running OCR if need be)
3) Run classification (sentence- and document-level), extraction algorithms against each document, generating a doc-level JSON (see below)
4) Run code on all of the doc-level JSONs to aggregate and resolve the doc-level extractions/classifications into one case-level JSON file to power the UI

### Document-level JSON

Stored as `<doc_id>.json` (maybe hash the OG filenames, assuming they're unique?). This would tell the UI where to find the local file `/standard/path/to/files/case-id/<doc_id>.<extension>`

```
{
    'extractions': [
        {
            'type': 'mp_name'
            'span': [115, 150],
            'val': 'Dominic Raab',
            'pred_proba': .95            
        },
        {
            'type': 'person',
            'span': [1115, 1150],
            'val': 'Kiran Smith',
            'pred_proba': .91
        }
        ...
    ],
    'sentence_classifications': [
        {
            'type': 'compassionate_need',
            'span': [160, 250],
            'key_span': ['190', 210],
            pred_proba: .85
        },
        ...
    ],
    'document_classifications': [
        {
            'type': 'from_mp'
            'span': null,
            'pred_proba': .99
        },
        ...
    ],
    'text': "This is the full doc text to slice with spans.",
    'extension': 'pdf'
}
```

### Summary-level JSON

Developed by Kiran based on UI considerations -- see, e.g., https://github.com/PrimerAI/home-office/blob/4a6c2b7db814de830a9439c803d254bc5385eb37/ukvi-frontend/src/components/CaseView/_sample_case_3.json. 

## Algorithms

### Extractive

Could be achieved by Regex + logic or a QA model. Explicitly mentioned:
1) MP Name and Address
1) Constituent Name and Address
2) Constituent Nationality
2) Constituent DOB
3) Case Reference #
3) Future Dates
4) Questions from MP/Constituent

Potentially valuable as well:
* Key numbers (re-use primer-core)

### Classifiers

All explicitly mentioned. Sentence-level feels like the right level of granularity here for the following:

1) Compassionate Need
    * Funeral
    * Death/orphaned
    * Forced marriage
    * Critical/terminal illness
    * Emergency
    * Child exploitation
    * FGM
    * Rape/sexual assault
    * Torture
    * Persecution
    * Domestic violence
2) Process Urgency
    * Urgent travel
    * Deportation
    * Removal
    * Detention
    * Persecution
    * Asylum
1) Disqualifying Criteria
    * Criminal
6) Less urgent reasons (e.g., mental health conditions)
4) Complaint
5) Other British government departments
6) FCO or foreign governments
7) Asks/Request for Urgency
    8) Expedite
    9) Request
8) Sponsor Details (need more clarity here, though)


Document-level "classifiers"/filters:
* From MP
* From Consituent
* Angry/charged language (could also handle at sentence level -- not sure here?)
* Legal language (e.g., referencing sub-clauses of various documents, etc.)
* Irrelevant docs
    * Passport scans

## UI

Just use a good component library (Carbon FTW!).

## Appendix

### User Workflow

1) Is the UK Home Office the right group to triage?
2) If so, it is urgent?
    a) If not: form response
    b) If so: commission to difference group
