# Apex CSV Downloader :page_with_curl:

### Purpose of this doc:
Capture and mock up how CSV downloading will work in Apex.
#### Factors to determine:
1) Data Flows
2) API Structure
3) UI Components

## Data Flows

- Download CSV button built in `livepicker`
- When the user clicks download CSV button in UI,
- API Call is made with selection
- API will generate an s3 file and send URL back to front end
NOTE: URLs need to expire
- UI formats file name based on current standard
- file will automatically start downloading in the browser

## API Structure
- API Call will have the same interface as the observation call with a different endpoint

csvGenerator({
    measures,
    dimensionInstances,
  }: {
    measures: string[]
    dimensionInstances: string[]
  }): string // file name generated from backend
  

// Globally unique generated file name
let expectedCSVFileName = `somefilename.csv`


expect(
  csvGenerator({
    measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(CA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(MA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(WA))"
]
  }),
).toEqual(expectedCSVFileName)

## [Example 1D - Single Measure Chart](https://apps.livestories.com/charts/5758e2bf-8ddc-4b53-8c7d-1f7d366eeb9c)

### API Call

liveAPIClient.csvGenerator(
    measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(CA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(MA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(WA))"
]
)

### Result

#### RAW CSV:

Indicator,Measure,Time,State,People,Error
Women in the Labor Force,People,2013-2017,California,15665864,2029.302693
Women in the Labor Force,People,2013-2017,Massachusetts,2901798,662.6542647
Women in the Labor Force,People,2013-2017,United States,131092196,6801.021339
Women in the Labor Force,People,2013-2017,Washington,2887525,879.0807952

#### Pretty Printed

| Indicator |Measure| Time | Location | Value | Error |
| -------- | -------- | -------- | -------- | -------- | -------- |
| Women in the Labor Force | People | 2013-2017 | California | 15665864 | 2029.302693 |
| Women in the Labor Force| People | 2013-2017 |Massachusetts |2901798 |662.6542647|
| Women in the Labor Force | People | 2013-2017 | United States | 131092196 | 6801.021339 |
| Women in the Labor Force | People | 2013-2017 |Washington | 2887525 | 879.0807952 |

### Detailed Generation of CSV

| "Indicator" as string |"Measure" as string| dimension.name | "Value" as string | "Error" as string |
| -------- | -------- | -------- | -------- | -------- | -------- |
| inidcator.name | measure.scaledLabel | dimensionInstance.name | observation.value | observation.error |

- Number of dimension columns will be generated based on the unique # of Dimension2.dimension.id from the selection
- Column Headers will be the `unique(Dimension2Instance.dimension.name)`
- Column Values will be the `Dimension2Instance.name`
- There will be a "Measure" column with the header as a static string
- Column Values will be all the `measure.scaledLabel` of selected measures
- There will be a "Value" column with the header as a static string
- There will be an "Error" column with the header as a static string

## [Example 1D - Multi Measure Chart](https://apps.livestories.com/charts/50052d11-49df-4d43-a0b3-e0c4d5cf5780)

### API Call

liveAPIClient.csvGenerator(
    measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0)", "M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))", 
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):LOCATION(T:NATION@1.0:C(US))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2012,2016))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2011,2015))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2012,2016))"]
)

### Resulting CSV
#### RAW CSV:

Indicator,Measure,Time,Location,People,Error
Women in the Labor Force,People,2011-2015,United States,15665864,2029.302693
Women in the Labor Force,People,2012-2016,United States,15665864,2029.302693
Women in the Labor Force,People,2013-2017,United States,2901798,662.6542647
Labor Force Participation,People,2012-2016,United States,131093350,6801.021339
Labor Force Participation,People,2013-2017,United States,131092196,879.0807952

#### Pretty Printed
| Indicator |Measure| Time | Location | Value | Error |
| -------- | -------- | -------- | -------- | -------- | -------- |
|Women in the Labor Force|People|2011-2015|United States|15665864|2029.30269317283|
|Women in the Labor Force|People|2012-2016|United States|15665864|2029.30269317283|
|Women in the Labor Force|People|2013-2017|United States|2901798|662.654264696942|
|Labor Force Participation |People|2012-2016|United States|131093350|6801.0213386832|
|Labor Force Participation |People|2013-2017|United States|131092196|879.080795185117|

### Generation of CSV

| "Indicator" as string |"Measure" as string| dimension.name | "Value" as string | "Error" as string |
| -------- | -------- | -------- | -------- | -------- | -------- |
| inidcator.name | measure.scaledLabel | dimensionInstance.name | observation.value | observation.error |

- Number of dimension columns will be generated based on the unique # of Dimension2.dimension.id from the selection
- Column Headers will be the `dimension.name`
- Column Values will be the `dimensionInstance.name`

## Unanswered Questions
- What do we do about indicators? :crying_cat_face:
- Do we want to include a measure unit column? Currently not implemented in Apex
*Note to our future selves: we may need this in the future. The user will be able to specify what the user would like to add to the CSV*
- What do we want to do about the file name? santizied by backend? Backend will deal with this.
- What do we want the user facing filename to be? Indicator name - special case multimeasure case
NOTE: UI will decide (look at character limit). Mirror download PNG download
- NOTE: Union of measures in multi measure case
- How do we want to deal with async download?
- End point to see if `filename.csv` is populated
- Initially if you get the file, start download
- If it takes too long, put up a spinner

## UI Component Design:

- `<CSVDownloadMenu/>`
- Exported component in `livepicker`

## TO DO:
- [ ] UI component design

## Implementiaon Thoughts from Antonia (take it or leave it)

I just used the following example to help me get an idea of how this many work and to make sure I wasn't missing anything! No need to use any of it as you all probably have a much more efficient implementation, just me hot take when I was thinking about it.

- COLUMNS: I would expect the general logic for the column headers to be able to leverage the logic used to create the response for the `metaData` call for the given Input `{ measures, dimensionInstances }`.

For example:

Using the `metaData` response for

let selection = {
 measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(CA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(MA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(WA))"
]
}

let metaData = liveAPIClient.metaData(selection)
let observations = liveAPIClient.observations(selection)

// lodash _.unique https://lodash.com/docs/4.17.15#uniq
let dimensionColumns = Object.values(
  Object.keys(metaData)
    .filter(dimensionInstance => selection.dimensionInstances.includes(dimensionInstance))
    .reduce((map, obj) => ((map[obj] = metaData[obj]), map), {}),
).reduce(
  (map, obj) => ((map[obj.dimension.type.id] = obj.dimension.type.name), map),
  {},
)

// {T:LS@1.0:LOCATION: "Location", T:LS@1.0:TIME: "Time"}


let measureColumns = Object.values(
  Object.keys(metaData)
    .filter(measure => selection.measures.includes(measure))
    .reduce((map, obj) => ((map[obj] = metaData[obj]), map), {}),
).reduce((map, obj) => ((map[obj.id] = obj.name), map), {})

// {M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0): "Total (People)"}

measureColumns["value"] = "Value"
measureColumns["error"] = "Error"

// let indicatorColumn = TBD 
// NOTE: I am not sure how you will look up the Indicator for each measure as it is not included on the measure model :( sorry

let csvHeaders =  {...measureColumns, ...dimensionColumns}

// Using the result from observation call to map the csvRows
let csvRow =