---
# System prepended metadata

title: 'Apex CSV Downloader :page_with_curl:'

---

# Apex CSV Downloader :page_with_curl:

### Purpose of this doc:
Capture and mock up how CSV downloading will work in Apex. 
#### Factors to determine: 
1) Data Flows
2) API Structure
3) UI Components 



## Data Flows

- Download CSV button built in `livepicker`
- When the user clicks download CSV button in UI,
    - API Call is made with selection
    - API will generate an s3 file and send URL back to front end
NOTE: URLs need to expire
    - UI formats file name based on current standard
    - file will automatically start downloading in the browser

## API Structure
- API Call will have the same interface as the observation call with a different endpoint
```typescript=
csvGenerator({
    measures,
    dimensionInstances,
  }: {
    measures: string[]
    dimensionInstances: string[]
  }): string // file name generated from backend
  
```


```typescript=
// Globally unique generated file name
let expectedCSVFileName = `somefilename.csv`


expect(
  csvGenerator({
    measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(CA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(MA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(WA))"
]
  }),
).toEqual(expectedCSVFileName)


```

## [Example 1D - Single Measure Chart](https://apps.livestories.com/charts/5758e2bf-8ddc-4b53-8c7d-1f7d366eeb9c)
![](https://i.imgur.com/u7ZYN2z.png)


### API Call
```typescript=
liveAPIClient.csvGenerator(
    measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(CA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(MA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(WA))"
]
)
```


### Result


#### RAW CSV:
```
Indicator,Measure,Time,State,People,Error
Women in the Labor Force,People,2013-2017,California,15665864,2029.302693
Women in the Labor Force,People,2013-2017,Massachusetts,2901798,662.6542647
Women in the Labor Force,People,2013-2017,United States,131092196,6801.021339
Women in the Labor Force,People,2013-2017,Washington,2887525,879.0807952
```

#### Pretty Printed

| Indicator |Measure| Time | Location | Value | Error |
| -------- | -------- | -------- | -------- |  -------- |  -------- |
| Women in the Labor Force	| People | 2013-2017	| California |	15665864	| 2029.302693 |
| Women in the Labor Force| People | 2013-2017	|Massachusetts	|2901798	|662.6542647|
| Women in the Labor Force	|  People | 2013-2017	| United States	| 131092196	| 6801.021339 | 
| Women in the Labor Force |	 People | 2013-2017	|Washington	| 2887525 |	879.0807952 |




### Detailed Generation of CSV

| "Indicator" as string |"Measure" as string| dimension.name | "Value" as string |  "Error" as string |
| -------- | -------- | -------- | -------- |  -------- |  -------- |
| inidcator.name	| measure.scaledLabel	| dimensionInstance.name |	observation.value	| observation.error |

- Number of dimension columns will be generated based on the unique # of Dimension2.dimension.id from the selection
    - Column Headers will be the `unique(Dimension2Instance.dimension.name)`
    - Column Values will be the `Dimension2Instance.name`
- There will be a "Measure" column with the header as a static string
    -  Column Values will be all the `measure.scaledLabel` of selected measures
- There will be a "Value" column with the header as a static string
- There will be an "Error" column with the header as a static string



## [Example 1D - Multi Measure Chart](https://apps.livestories.com/charts/50052d11-49df-4d43-a0b3-e0c4d5cf5780)
![](https://i.imgur.com/Jtxe0Et.png)




### API Call
```typescript=
liveAPIClient.csvGenerator(
    measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0)", "M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))", 
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):LOCATION(T:NATION@1.0:C(US))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2012,2016))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X8(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2011,2015))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
"D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2012,2016))"]
)
```


### Resulting CSV
#### RAW CSV:

```
Indicator,Measure,Time,Location,People,Error
Women in the Labor Force,People,2011-2015,United States,15665864,2029.302693
Women in the Labor Force,People,2012-2016,United States,15665864,2029.302693
Women in the Labor Force,People,2013-2017,United States,2901798,662.6542647
Labor Force Participation,People,2012-2016,United States,131093350,6801.021339
Labor Force Participation,People,2013-2017,United States,131092196,879.0807952
```

#### Pretty Printed
| Indicator |Measure| Time | Location | Value | Error |
| -------- | -------- | -------- | -------- |  -------- |  -------- |
|Women in the Labor Force|People|2011-2015|United States|15665864|2029.30269317283|
|Women in the Labor Force|People|2012-2016|United States|15665864|2029.30269317283|
|Women in the Labor Force|People|2013-2017|United States|2901798|662.654264696942|
|Labor Force Participation |People|2012-2016|United States|131093350|6801.0213386832|
|Labor Force Participation |People|2013-2017|United States|131092196|879.080795185117|


### Generation of CSV

| "Indicator" as string |"Measure" as string| dimension.name | "Value" as string |  "Error" as string |
| -------- | -------- | -------- | -------- |  -------- |  -------- |
| inidcator.name	| measure.scaledLabel	| dimensionInstance.name |	observation.value	| observation.error |

- Number of dimension columns will be generated based on the unique # of Dimension2.dimension.id from the selection
    - Column Headers will be the `dimension.name`
    - Column Values will be the `dimensionInstance.name`






## Unanswered Questions
- What do we do about indicators? :crying_cat_face: 
- Do we want to include a measure unit column? Currently not implemented in Apex
*Note to our future selves: we may need this in the future. The user will be able to specify what the user would like to add to the CSV*
- What do we want to do about the file name? santizied by backend? Backend will deal with this.
- What do we want the user facing filename to be? Indicator name - special case multimeasure case
NOTE: UI will decide (look at character limit). Mirror download PNG download
- NOTE: Union of measures in multi measure case
- How do we want to deal with async download?
    - End point to see if `filename.csv` is populated
    - Initially if you get the file, start download
    - If it takes too long, put up a spinner


## UI Component Design:

- `<CSVDownloadMenu/>`
    - Exported component in `livepicker`

## TO DO:
- [ ] UI component design
    


## Implementiaon Thoughts from Antonia (take it or leave it)

I just used the following example to help me get an idea of how this many work and to make sure I wasn't missing anything! No need to use any of it as you all probably have a much more efficient implementation, just me hot take when I was thinking about it.

- COLUMNS: I would expect the general logic for the column headers to be able to leverage the logic used to create the response for the `metaData` call for the given Input `{ measures, dimensionInstances }`.

For example:

Using the `metaData` response for 
```typescript=
let selection = {
 measures: ["M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0)"],
    dimensionInstances:
    [
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):TIME(T:US.GOV.US.DOC.CENSUS.ACS@2.0:5YEAR(2013,2017))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(CA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:NATION@1.0:C(US))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(MA))",
        "D:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0):LOCATION(T:US@1.0:ST(WA))"
]
}

let metaData = liveAPIClient.metaData(selection)
let observations = liveAPIClient.observations(selection)

// lodash _.unique https://lodash.com/docs/4.17.15#uniq
let dimensionColumns = Object.values(
  Object.keys(metaData)
    .filter(dimensionInstance => selection.dimensionInstances.includes(dimensionInstance))
    .reduce((map, obj) => ((map[obj] = metaData[obj]), map), {}),
).reduce(
  (map, obj) => ((map[obj.dimension.type.id] = obj.dimension.type.name), map),
  {},
)

// {T:LS@1.0:LOCATION: "Location", T:LS@1.0:TIME: "Time"}


let measureColumns = Object.values(
  Object.keys(metaData)
    .filter(measure => selection.measures.includes(measure))
    .reduce((map, obj) => ((map[obj] = metaData[obj]), map), {}),
).reduce((map, obj) => ((map[obj.id] = obj.name), map), {})

// {M:US.GOV.US.DOC.CENSUS.ACS.DP03@1.2:X10(0): "Total (People)"}

measureColumns["value"] = "Value"
measureColumns["error"] = "Error"

// let indicatorColumn = TBD 
// NOTE: I am not sure how you will look up the Indicator for each measure as it is not included on the measure model :( sorry

let csvHeaders =  {...measureColumns, ...dimensionColumns}

// Using the result from observation call to map the csvRows
let csvRow = 

```